 Okay, so what we did, there'll be some people, maybe joining us for a new year, not a happy new year, but it is a new year, a new academic year, so welcome back to a lot of familiar faces and some new faces and faculty and staff and students and alumni and others all here, it's good to see you all. For those of you who do not know, I'm Autumn Hajran, I'm the Dean here in the College of Science in there. And we're very happy to kick off our fall semester, not only with our classes and sort of regular every day activity, but the first of them will be a rich series of seminars and talks from various folks from amongst our own faculty or students, alumni and often from our friends and colleagues from other institutions we've all received and so kind of with that broad introduction, I'm really happy to introduce to you Autumn Babikyan. Autumn is a PhD student at the McGill University in Montreal, there's like safe ways of saying this. So for the end of his PhD, he can see the light at the end of the tunnel and he's been working on actually a really sort of cutting edge technologies and science, algorithms and computer optimization algorithms and software behind ensuring the safety of autonomous vehicles and the title on the screen. And so thank you all for coming and thank you Autumn for making time in your what's called part vacation and part travels to Armenia to visit us and share with us some of your ways of research. That's it, so thanks. All right, thank you Autumn. Yeah, thanks for the introduction. As Autumn said, my name is Autumn, I'm from McGill. I have a slide about myself, so not going to talk too much. Yeah, it's a pleasure to be here. Yeah, part vacation, part work. I'm really happy to be here. Thanks for everybody who made this possible and we'll see hopefully if it goes well again sometime. There'll be an excuse for me to come to Armenia. Right, so today I'll be talking about what I do in my PhD research. So my research interests and specifically I'll be talking about self-driving cars. So autonomous vehicles are self-driving cars. My research interests aren't exactly related to training self-driving cars or just simply self-driving cars. They're more focused on the safety of self-driving cars. So what we're trying to do in a nutshell, we're trying to figure out new ways, better ways to ensure self-driving cars to be safe. So that's what I'll be talking about. I'll be going through a lot of different things, specifically model-based approaches. I'll be talking about that and optimization approaches. So I don't exactly know what the level of the background is in the classroom today. At any point you have any questions, technical, non-technical, you know, no problem. Any questions, feel free to ask. I was at the OVASP meeting a few days ago and one of the speakers said, Armysol Ansin Abakachigar. So Miya Mishnak, you have asked the questions. Please feel free to interrupt me. I like getting interrupted. So about me, as Aram said I'm a PhD student. I started in January 2020. I'm hoping to finish next April. So it's a hope, not necessarily an expectation. So the way it works in our research group or in kind of my PhD, we have a research group. We call it the Critical Software Intensive Systems Research Group. We do a lot of things related to safety, security, not only of self-driving cars, but a lot of things in general. And the way we work is we do a lot of kind of small projects with a lot of people, mostly undergrad students at McGill or at different universities. So I spoke to Aram a few days ago. So I can kind of at the end propose some informal collaboration opportunities. So listen to the talk. If there's anything that interests you, I have kind of a list of future plans that we have in our research group. If anything interests you, feel free to come talk to me. We'll try to set something up, something that you can contribute to some open source projects, or we can maybe even get a publication out of that. So if anything peaks your interest at the end, there's going to be a list. So just feel free to contact me and we'll try to set something up. So regarding my research interests, as I mentioned, I like testing self-driving cars, right? Testing autonomous vehicles. But in order to do that, there's a lot of kind of background, things that I need to know or theories that I need to apply to make self-driving car testing something that's valuable, right? So some of these things are model-based systems engineering, graph model generation, domain-specific languages, all of which I will be talking about today. But there's also a few other things I'm interested in, formal methods and reliable ML AI. I had to put ML and AI in there. But I won't be talking too much about that. So hopefully some of you recognize some of these words, these buzzwords. If not, it's not a problem. I will be going relatively thoroughly throughout these things. So yeah. So I will be discussing these kind of more background things as well as not only going to be testing self-driving cars. Quick-tape. Ooh, what is this? I clicked on the wrong button. Yes. Quick table of contents for today. So the table of contents, there's kind of four components. The first component is a short one. I'll give a bit of a motivation for safety of self-driving cars just to kind of get you guys interested a bit more if you're already not. Second, it will be a conceptual section, key concepts for AV safety assurance. So these days, the big kind of ideology, the big approach for self-driving cars testing is what they're calling scenario-based testing. So I'll explain what scenario-based testing is and what are the different kind of ways we can leverage these ideologies, these kind of conceptual advancements for the actual testing of self-driving cars. So the second part will be more conceptual. There will be more ideas. And the third part, it'll be a bit more kind of technical, not as technical as you would expect, but more kind of concrete research that we've been doing. So what we do in the context of scenario-based testing is we generate AV test cases, right? Test cases for self-driving cars. And to do that, we leverage model-based approaches, which I will be talking about, and also optimization algorithms, which I will be. Each one of those points refers to a paper that we wrote or were writing. So I will be talking about two of our papers. And one will be about model-based approaches, the other one will be optimization algorithms. So that's kind of the way that third part is going to work out. Finally, in the last section, as I mentioned, future research projects. So I'll explain what we're doing currently as a follow-up to what we have been doing. And I'll propose some things that are maybe we're planning to do in the next five years, three, four, five years, which hopefully it'll pique the interest of some of you and we can try to collaborate on that. Right, so let's get into the motivation. So autonomous vehicles, self-driving cars are gaining in popularity. So it's obvious these days to understand that self-driving cars are becoming more and more popular. In Canada, especially, it's a lot of self-driving cars on the roads. Of course, the levels of self-driving really vary. Some are only doing some kind of lane change assist or that kind of stuff. Others are a bit more self-driving. But these kind of varying levels of self-driving vehicles are pretty much everywhere in Canada. Even here, I saw a few Teslas, so that was nice to see. But in terms of the industry, there's a lot of big companies working on this. So I have a few on the screen. Crews, Waymo, Argo, AI, Teslas, it really looks like it's the future. It really looks like it's the future. However, it's not only gaining in popularity in the industry, there's also work being done in academia. So a lot of mostly university research groups are working on self-driving cars, but it's not exactly in the same way. So the objective of the industry kind of companies, their objective is to have people buy their self-driving cars. They actually build the self-driving car. They train it. They do a little bit of testing and they want people to buy it. So they market it really, really well with the objective of people buying it. But that's not the case for academia. Academia is more focused on the conceptual kind of research advancements. So academia detects issues in the current state and they try to address those issues. There are a few examples though where they're actually building vehicles. So Autonomous is an example from Canada. I think it's Waterloo. They actually built the vehicle and they're training it for some specific cases. So in this case, they're training it for Canadian winters. So Canadian winters are a lot of snow, bad visibility. So they have this project going where they're really training specifically for those cases. So that makes it a little bit more interesting, but as I mentioned, academia is more focused on the research things, the conceptual things. So autonomous vehicles are gaining in popularity. It's clear to see. However, there are real safety concerns. So this is obvious as well. I brought a few examples of news kind of news reports where there's casualties that are caused by self-driving cars that are being currently on the road. So these are relatively recent. This is an old slide, but there's one from 2022, another one from 2021 and so on. So these are kind of very relevant things to date. It's very wrong to say that self-driving cars are currently safe enough or casualties are a big deal. So when I can have four reports on the screen, it shows that there's some concern. So most of these are from the US. There are some in China as well. I don't know what the state is here in this part of the world. But generally, I mean, in North America, this is a big, big problem in China as a big problem. So what's the key takeaway from this? What's the key takeaway? The key takeaway is that, of course, we understand that self-driving cars are not safe, but we also need to understand that the existing approaches for safety of self-driving cars are not good enough. So it's not only that they're just not safe, that the existing approaches are not good enough. So that's the research that I'm doing. We're trying to improve the existing approaches to have better self-driving cars as an end goal. So we need better safety assurance approaches. Now, safety assurance, where do we start? Where should we start with respect to safety assurance? One place where a common starting point in terms of these kind of safety things are safety standards. So for those of you who aren't familiar, safety standards are kind of like a list of requirements. So what the safety standard does, or it says, is it gives a bunch of requirements to allow a vehicle to be placed on the road. And this isn't only for self-driving cars. So this is for regular vehicles as well. This is a simple example. If a regular vehicle has a faulty brake, like it can't press on the brake, that's a safety concern. So these standards are going to state that in these kind of cases, you want to make sure that your brake works, right? Otherwise you're not being safe neither for yourself nor for everybody else. So these are the kind of things that the safety standards are doing. And in case of self-driving cars, it's a bit more complicated because there's a lot of softer stuff that's going on in the background. A lot of people are not necessarily understanding exactly what's going on. So to kind of address the safety of this, it's a different challenge. And it turns out that in the case of self-driving cars, what these safety standards are proposing or prescribing are system-level safety assurances. So an example of a system-level safety assurance is really like an abstract statement, like a vehicle performs a right turn without crashing into the wall. Something simple as that that refers to the entire vehicle. It's not a case where the camera on the vehicle correctly detects an object. That would be a component test. So the system is something related to the entire system. It's not related to a specific component. That's kind of the safety standards, the things that are prescribed in the safety standards. Now, with this system-level safety objectives in mind, the idea is we want to come up with some approaches to ensure these safety standards, these system-level safety requirements. So I'm going to propose, well, not propose, but present three different approaches that are kind of currently relatively common. First one is in the industry, the other two are in academia. So in the case of the industry, what they're doing or they have been doing for a while is on-road monitoring. Also just a side note, there's a lot of companies who are moving towards more kind of cutting-edge safety approaches. But this is kind of historically what industry has been doing. So on-road monitoring. The way it works is that you build the autonomous vehicle, you build the self-driving car. You do a little bit of training, you train it, you make sure it's relatively safe. But the real kind of testing happens directly on real roads alongside real people, alongside real other vehicles. So that's kind of where the main testing is happening. Of course, there's different levels, as I mentioned. It can be just part of the AV that's being tested and so on. It's not as straightforward as that. But the idea is the main kind of testing is happening on the real road. Now, when do we know when our vehicle is safe? So the way the approach suggests that you let the vehicle ride for a certain amount of miles, like let's say n miles. And if it's able to safely navigate through these n miles, you can make the assumption that the vehicle is safe enough. That's kind of the assumption that that's working. But there's a few problems with this assumption. First of all, there's no guarantee that in the next mile you wouldn't have gone into an accident. So that's definitely one of the problems. And the other problem is that people hear about these issues. They did a little bit of research and they realized they tried to figure out how many miles do we need to actually write for our assumption to be valid or significant, statistically significant. And it turns out that number from 2016 it was 8 plus billion miles. That's a lot of miles. But these days you might argue that some of the companies who have millions of vehicles on the roads might be getting close to these kind of numbers. But the reality is that 8 billion miles really kind of depends on the type of miles. Let's say you're on a highway. If you're driving 8 billion miles safely on a highway, it doesn't necessarily mean that you're going to be safe on residential area. It doesn't necessarily mean you're going to be safe on tight curves. So this 8 billion miles, it's like a lower bound. Because you need to take into consideration all the possible areas, all the possible road configuration, all the possible scenarios. So this is kind of the main conceptual issue with this on-road monitoring. The other kind of more practical issue is that you're placing non-tested cars on real roads. And as a result, we end up with casualties, as we saw in the news reports. So that's the other kind of more practical problem with this approach. So this is historically what industry has been doing. Now, as I said, academia is trying to find better ways to do these things. So existing research, I said existing. It's really research from like three, four, five years ago. What they used to do is, of course, they wanted to test system level things, right? They want system level safety. What they did is they took the AV, they split it into different components, and they tested each component individually. So for example, you have a camera on the AV. You test the camera, whether it can correctly detect the vehicles, right? You can individually test the camera. But some research eventually showed that even though every component is working correctly individually, this doesn't really translate to system level safety. So when you put everything together, it doesn't really provide the safety assurance requirements that we need for vehicles to be considered safe. So there was a lot of work being done related to these kind of combining component level tests. But it turns out that in the end, it wasn't really an approach that was valid or valuable. Now, say to the art, what people have been working on in the past two or three years. They're calling it scenario-based testing. So the idea is very straightforward. You generate some scenario, some traffic scenario as a test case. You take your AV, you place the AV inside the test scenario, and you let the AV run. You give it some requirements. You give it some locations that it needs to travel, and you see what happens. You can do this in simulation, which is good, so you don't need to go on real roads. And you can monitor the results. Did it actually get into a crash? Did it hit a wall or whatever? And in the end, what you do is you can evaluate some kind of safety score. You can determine whether the AV has behaved safely or unsafely, right? You can have that kind of result. One example, so I have the description here, but I'll also add the picture. Now, let's say we have a scenario, and the scenario is described as follows. It's a scenario where the ego vehicle, so the red vehicle, is the vehicle we care about. It's the vehicle we want to test. So this ego vehicle is behind two adjacent non-ego vehicle, right? So the picture is pretty straightforward. There's the red vehicle behind two black vehicles. Now, the scenario is a bit more complicated. It claims that one of the non-ego vehicles, one of the black vehicles, is going to cut in front of the other vehicle, and the other vehicle, you know, to avoid a crash, it's going to break, right? A sudden break to avoid the collision. So let's just see it on the video. There's the cut in, and there's the break, and we care about the red vehicle, right? The red vehicle was able to kind of maneuver this scenario in a way that's safe. So what's the safety score? What's the result of the test? The test is successful because the red vehicle did not get into a collision. This is great. We're happy. You know, it might make us think that the red vehicle is behaving safely, right? Well, this is just one test case, and it's a successful test case. Now, what I'm going to do now is I'm going to give another example of the same traffic scenario. Here it is. We have a scenario. Ego vehicle is behind two adjacent vehicles, and there's a cut in situation, and there's a break, right? And actually, it's exactly the same track trajectories that they're following in both cases, right? Notice these exactly the same trajectory, but you might be expecting what's going to come. Red vehicle got into a crash. Oh, why did it stop? Oh, why is it not working? Anyway, everybody saw the crash, right? So what happened is, you know, it's the same scenario, but the outcome is different. This is a problem. In this case, we have a collision. The result is a failure, right? The test is a failure. So anybody who's a little bit familiar with testing can understand why this is a problem, right? We have a test case, and sometimes it succeeds, sometimes it fails, right? And we can't really make reliable test cases when we're dealing with these kind of things. So how do we solve this kind of problem? The problem with two implementations of the same scenario have contradictory safety results. How do we address this problem? One solution, one potential solution, is as follows. So we can take a look at all possible kind of scenarios, implementations that represent the scenario, and we can optimize or we can select the ones that we're interested in based on how potentially dangerous they're going to be. So if we take a look at the two examples that I showed, the one on top here, this is the one where it didn't crash. So just by looking at this picture, we can have an idea that it might not crash because it's far away, it can take its time to slow down, right? But on the contrary, the one on the bottom, it's a bit more susceptible for a dangerous situation, right? It's a bit more susceptible to get into a crash because it's closer. So what we do is we decide that we don't really care about this irrelevant one, the one on top, we just care about this. So we completely ignore these ones and we optimize for danger, right? That's the idea. In order to generate test cases that would potentially cause issues. Now, the question is, what is danger? How do we know what we want to optimize for, right? So in this case, it's relatively straightforward. We can know a scenario is dangerous by, first of all, looking at the distance between the vehicle, right? If the red vehicle is close to the black one, it's probably more dangerous, right? Anybody has any ideas what else we can optimize for? Speed, yes. Vehicle speeds, if the vehicles are faster, that means that they're more susceptible for danger. Yes, anything else? Yes, I haven't added that, but that's a good one. Yeah, a response rate of the cameras. Up hill downhill? Up hill downhill, yes. Yeah, a lot of kind of different answers. And actually, up hill downhill is a good one because it kind of takes into consideration the map, right? So we haven't talked about the map yet and the 3D kind of, exactly. So right now, I've just been talking about vehicle position on a 2D plane. So it's a lot of variables as you can think of. Is that a presence of a third vehicle? Yeah, exactly. So in that case, that would be a different kind of scenario. Or it might not be. You can choose to ignore that. But trying to stay limited to this, there's a lot of information that's missing. That's the point I'm trying to make. There's a lot of missing information that you can kind of intelligently decide upon in order to cause potential danger. The other one I had in mind was the cut-in sharpness. So if the cut-in is super sharp, then there's crash possibility. If it's not sharp, there's less of a possibility. So these are the kind of things that we're looking for to kind of have a better idea whether the situation will cause a crash or whether it won't cause a crash, right? And this is very common in current state of the art. A lot of people, even this year, a lot of papers are doing this kind of analysis. They're selecting a scenario and they're optimizing for danger. And we did something similar a few years ago. In our case, it was a different kind of scenario. The idea here, ooh, there's a duplicate person. Anyways, in our case, the scenario was the following. We have a case where, not this one. We have a case where we have two vehicles and the pedestrian. So the ego vehicle, the one we're testing is the blue one, right? We want to make sure that the blue is going to avoid collision. So the idea is relatively straightforward. The pedestrian is crossing the road. The blue vehicle is also crossing the intersection and there's going to be a crash. That's the idea. We want to make sure that the blue vehicle can avoid this crash. What are the requirements for this scenario? First of all, there are a few requirements in terms of positioning for the vehicle. We know that the pedestrian must be in this kind of area. We set that as a requirement. And the blue vehicle must be in kind of this area. So we set that as a requirement, an up your requirement. We're not optimizing this. You can also have some idea of path directions. So we know that the pedestrian is going downwards and the blue vehicle is going to the left. We want to avoid cases where the pedestrian decides to go back on the crosswalk. And there's no danger. Now, what defines danger? In this case, what kind of conceptually is defining danger is the fact that the blue vehicle cannot see the pedestrian while it's going into the intersection. That kind of makes it dangerous. And the other dangerous thing is we want to kind of ensure this collision. We want to be strategic in terms of positioning in order to make sure that the collision will occur. What we need to do is we need to look for interesting positions and speeds for the vehicles that are going to cause this collision. That's kind of the general idea. Now, has anybody seen something like this before? No? Even better. So the way we decided to go about this problem is by using some model-based representation, some model-based approach. So what you see on your screen here is a domain model. And this is a software artifact. It's not just an image. What you can do is you can actually generate some code from this. You can do a lot of transformations, a lot of code generation, that kind of stuff. So all you need to do is you need to represent the problem, and it's going to allow you to kind of facilitate finding the solution. So essentially what this domain model contains is all the information that's relevant for our test case. So for this test case, everything is relevant. So what's the first thing that we need? First of all, we need a scenario. We need a crossing scenario. A crossing scenario may have some attributes, size of the screen, maximum speeds for vehicles, that kind of stuff, global kind of stuff. And so we put this crossing scenario into the domain model. Now, what else does the scenario contain? What else does the scenario contain? So obvious. Actors and lanes. You need lanes. You need actors. Let's start with the lanes. You have lanes. You have two types of lanes. You have horizontal lanes. You have vertical lanes. So horizontal lane is this, and vertical lane is this. So the scenario, the crossing scenario contains a bunch of lanes. Crossing scenario also contains some actors. So actors have exact positions. They have length and width. They have speed, and they can be either pedestrian or a vehicle. In our case, they can also be a bicycle or a tractor or whatever. But in our case, we don't care about that. We just need to represent what we're interested in. So the actor is placed on a lane, and the scenario contains a bunch of actors. And the last thing we need is some relations. So relations are kind of interactions between components. So in this case, what we know is we want this pedestrian and this vehicle to get into a collision. That's a relationship. It's a relationship between two actors. We also want these two to not be visible to each other. That's, again, another relationship. So we represent these relationships based on what we need for our scenario. So here's the kind of full domain model. It was really quick explanation. Please feel free to ask questions. I'm going at it relatively fast. So here's a domain model. It basically contains all the information that we need. And what you notice is that there's kind of two categories of information. First part of the information is things that we know priori, things that we already know. What do we already know? Well, we already know where the lanes are. We already know how many actors there are. We already know which two actors are not seeing each other. There's a bunch of things that we know. And everything that's in red, we already know. So what's left? What do we not know? Well, we don't know the exact position of the vehicles. We don't know the exact speed of the vehicles. That's what we're trying to optimize. We're trying to optimize the speed. We're trying to optimize the positions to cause that crash. So very, very quickly, how do we find these optimal positions? How do we find these optimal speed? For that purpose in our paper, we use the combination of the V-actron model generator, which is an in-house tool, and the Z3 theorem provers, the D-reel quadratic solver. Has anybody heard of any of this? Would have been nice, but it's okay. I won't be talking into details, but I learned about this like last year, or the year I wrote the paper. So it's really kind of cutting-edge stuff, very specific stuff, so you don't use this if you're not in the domain. In a nutshell, the idea is that we have the V-actron model generator, which is handling the abstract relations as we discussed, and we have this other kind of theorem prover, this quadratic solver, which is handling the numeric things. We're looking for numeric things. So we're using a combination of this to end up with the numeric, exact numeric things that we're looking for. What are the kind of test cases we can generate? Here are three valid test cases. So as you can see, the constraints are being held. So the pedestrian is always on the crosswalk, different places, but it's still always on the crosswalk. The blue vehicle is always on the lane, although it's different locations. And we're defining the speeds to make sure that the collision would probably occur. There's a likelihood of the collision occurring. So we're defining the speed in that way. So these are kind of test cases we can generate, but things that we want to avoid are invalid test cases, are these kind of test cases where, yeah, there might be a crash here, but the original condition where the visibility must be blocked is not held. So we want to kind of avoid this because it was part of the things that we're looking for in the test case. Another case is this. So let's say, in this case, the blue vehicle is very fast and it's really close to the intersection. It's just going to pass by the intersection before even the pedestrian has a chance to come close to it. So again, this is something that we want to avoid because we're interested in that collision. We're interested in that collision. So our solution, our kind of approach, will not provide these kind of results. Make sense? All right, moving on. So of course, yes. So I realized that a lot of this is dealing with autonomous cars and their safety as far as in order for us to even allow autonomous cars and most likely the safety that could surpass the delivery of human. So then is that factored in here as far as here's a situation where it would have been impossible for a human to avoid the accident. But now we have to account for that and say now the car still has to avoid the accident. That's a really good question. Along with that would be questions about IR versus just camera. So at a conceptual level there's cases where, so there's cases that you're just standing and the vehicle is coming to, an adversarial vehicle just comes to you from the side when you're in traffic, like you can't do anything, right? So the interesting thing here is that we spoke about defining danger. There's also the idea of defining the concept of what can we do, right? What can the AV do? So there's a bunch of things that are allowed for the AV like in the case of an intersection you're allowed to turn right, you're allowed to go forward, you're allowed to turn left, right? Those are kind of expectations and we can't really even expect the AV to do things that are illegal, right? Even though it feasibly can do those illegal things. So that's kind of the idea of illegalness and what's possible, what's not possible. Those are things that has to be kind of manually told to the test case generator, right? The test case generator won't figure this out, right? So you need to, at least in this approach, you need to tell it a priori what are the different things you're looking for, what you want to avoid. That's one of the problems actually of this approach is that it won't be able to figure out and actually I'm talking about that in the next few slides. So that's one of the other things and LiDAR versus camera, again it's a similar thing, like if it can see all around, then the possibilities are different, whether it can only see in front, the possibilities, like the expectations of safety become different. But conceptually speaking, you need to tell it to the test case generator. So that's a really good question actually. Yeah, so just to conclude this kind of section of the presentation, we have the valid test that we generate and the invalid test cases that are not following the kind of the condition, the relations. Now, let's take a step back. We had the initial problem where we have a specific kind of scenario and we had two implementations with different outcomes, right? These two. And the solution that we proposed to this was to optimize for danger to just take into consideration this one, ignore it, right? But this sounds a bit suspicious, like why are we ignoring? Maybe there's relevant things in that test case, right? Maybe it's relevant for some other reason that we can't think of, right? Although it doesn't satisfy the danger idea, but it might be relevant, right? So the problem with this optimization for danger approach is that it's really limiting in the sense that, you know, you have one scenario and you're just generating a specific thing for that one scenario. It's very, very specific. So when we're interested in general AV safety assurance, when we want to kind of ensure the safety of AVs as an entire kind of concept in all possible scenarios, we can't really rely on a solution that only takes into consideration one scenario, that only takes into consideration a single definition of danger, right? It's very limited. The scope is very limited. Of course, as a side comment, of course, using this kind of optimization for danger, we can definitely state that it's safe in this scenario. Yeah, for sure we can do that, but that's just one case and there's infinities of scenarios that are possible, so we can't really list all scenarios and use these approaches. So the idea is we want to kind of have an approach that's not going to be specific to a scenario. Talk about that in the next slide. But yeah, the optimization approach is not necessarily generalizable. That's a problem. It's not generalizable. So what we're looking for is we want cases where it's not specific to a given scenario. It can be generalized to any input scenario, right? So here's, again, the same example. We add a little bit more information this time, right? We're adding here in this case that the red vehicle is far behind. Here in this case, it's close behind. Now we can differentiate these two scenarios. Now we can actually give value to each one of these. So we need an approach that is going to be able to directly handle this difference. It's going to be able to handle this additional information but without optimizing, because if we optimize, then we're completely getting rid of this, like we're ignoring this entirely, right? We want to take into consideration all the information and we want to take into consideration all possible scenarios. So that's the kind of approach we're proposing in our research and our latest research. We want an approach that can generate test cases for arbitrary scenario description. You give any scenario description as input, we need to be able to generate test cases for that. Of course, it sounds like a big, no, it's a big next step and there's a lot of kind of intermediate steps that we need to perform in order to achieve this. But while I'll be showing here for the presentation, just kind of one intermediate step. So I won't be talking about behaviors. I'll just be showing an example where we can use this kind of arbitrary scenario generation for static scenes, things that are not moving. So I'll give that example. So generate, I'm sorry. So the idea here, we want to have an approach that's going to be able to understand the scenario. It needs a language to define the scenario. So we're using the same kind of abstract constraints close, far, right, left, ahead, behind. We're using this kind of level of abstraction in terms of defining the constraints because they're semantically relevant. So in the previous example, this was ignored. Close versus far was ignored. Now we're giving meaning to it. We need to be able to handle this because they're semantically relevant. Close is not the same as far. We need to explicitly say that so we need this kind of language to define the scenarios and from that kind of scenario description, we want to generate actual test cases, right? We want to place the vehicles on correct places that's going to satisfy all the constraints. That's the idea. And we'll do this for simulation and for testing. So what I'll present now is our approach that's going from scenario description to concrete scenes exact kind of test cases. So what is a traffic scenario? How do we define a traffic scenario? We spoke about left, right, forward, behind those kind of things. So there's a really good paper actually from Ulbrich in 2015 that's talking about how we can define scenarios in a formal way, right? So the suggestion is that scenario is comprised of three different types of components. We have the scenery, things that don't move, tables, roads, houses. We have dynamic elements, things that do move. In this case we have three vehicles. We can have pedestrians, we can have bicycles and so on. And as I mentioned previously we have relation, right? So in this case vehicle C is in front of A, that's a relation. Vehicle B is on the road so we can have relations between the vehicle and the static scenery as well, right, that's possible. So this is what it takes to define traffic scenarios in a kind of general arbitrary way. And as I mentioned a few seconds earlier in the case study that I'll be showing it'll be about static scenes so I won't be able to discuss behaviors because that's kind of next step, next thing. Now, we want to go from the abstract representation to simulation ready. In the abstract representation they're calling it a functional scene. Function, you can think of it as a mathematical function, you know. We're representing the problem as I said, so C1 is in front of C2. That's a description, we're using qualitative abstractions that's kind of technical term for this kind of in front of, it's a technical term. Now, what does this in front of mean in terms of actual kind of test cases, right? What does it mean? We can represent this abstract constraint as a region on the map, right? So let's say we place a blue vehicle here. If the red vehicle, C1, is in front of the blue vehicle then we're making the assumption or we're defining it in a way to state that the green vehicle must be in this kind of triangular area. That's the definition of the abstract that's the intermediate level, the logical scene, intermediate level, intervals and regions and then at the end we have the concrete scene which is what we can use for simulation. This is exact numeric values, exact position, so as you can see, in this case we can clearly see that the green vehicle is in fact in front of the blue vehicle, right? So these are the three levels of abstraction that we're going to leverage. We're going to go from here all the way from here which is the input and one end up here, right? And we're going to pass through those kind of regions, those triangular areas that I described. So as a baseline tool, there's currently tools doing this but not too well but we're doing it. Scenic is a really good tool that we're kind of building upon that is doing this kind of transition but we try to do a bit better, right? Now I'll give a few comments for each level of abstraction. For functional scenes, what we can take into consideration is the abstract relations, close, medium distance, far, can see, left, right, whatever and we can have different categories, right? This is distance categories, visibility categories, position and any category you can think of, anything you require in your test case, right? We can add categories. For example, we can have a category for road placement, like is a vehicle on the road, is a vehicle on an intersection, is a vehicle, you know, on the sidewalk might be a case where it's interesting for the vehicle to be on the sidewalk, right? So we can add any kind of category that we require for our test cases but the idea is that this is expressive enough for arbitrary scenes and it's extensible. Here's an example. So this means that OG is close to OB and as you can see they are close. Medium distance for the other cases, anyways you can take a look, it's really, it's very intuitive, right? So both of these vehicles are behind the red vehicle and so on. Now, a little bit of technical things or conceptual things for anybody who knows about these partial models, what the particularity of our languages is not just the textual language, it has actual semantics, right? And it has mathematical properties that we can leverage and we can do things like validate the scenarios before even running the test case generation, like we can do those kind of things because there's mathematical properties. So we're using partial model semantics which is really cool actually, I really like that, but I won't be getting into the details so much. Now, we're in the middle kind of category of the logical scene. We have, as an example we have this kind of left constraint and that left constraint corresponds to this region, right? We spoke about regions, how do we represent this region? Well, we do this mathematical, so we have a really complicated mathematical formula that defines the region, right? So what this formula is saying is that O B must be in the region defined by this mathematical formula. So it has to be in that region. We can do the same thing for D ahead. Again, it's a mathematical formula, small modifications, but that formula is requiring the gray vehicle to be in that new region, right, in this region. So, as you can realize, this kind of scene specification is being mapped into a long list of mathematical constraints, right? And what we need to do is we need to satisfy these constraints. We need to have a solution that answers kind of these constraints, right? And to do so, we're using metaheuristic search which is the generalization of genetic algorithms. So I'll talk a little bit about genetic algorithms. Does anybody know about genetic algorithms? Yes, somebody knows. Good. So I'll talk about genetic algorithms as a quick kind of idea to give you an idea of what the approach does, right? So remember, we have a bunch of constraints, we need to find an answer to those constraints. So the way that genetic algorithms work, you start off by randomly generating some potential solution, right? We have no idea whether these solutions are valid or invalid, maybe they are solutions, maybe they're not, right? But you start randomly, start randomly generating a bunch of potential solutions, things that may be solutions. Next thing is you evaluate their fitness. So you have some objectives, the test case has some objectives, you have constraints that you need to satisfy, right? You test whether these potential solutions are actually satisfying those constraints. So you can get some kind of numeric value how fit the population is. And then you ask yourself, is the fitness good enough? Have we met our objectives? Have we satisfied our constraints? If we have, great, we're done. We have an answer. If we haven't, we need to restart, right? We need to generate a new population. And we need to reevaluate the fitness and restart. Check again. If until, it gets satisfied, right? Until we determine. Yes? Didn't we at first try to reach situations where we haven't where it was. Sorry, I didn't get that. We are now reaching the constraints. So if it reaches, we are taking this Yes. Yes. Yeah, that's a good one. So in this kind of mathematical formalization, the way we represent the constraints, we can decide what we want. So if you notice the constraints are a bit different because they're the constraints were things that are promoting the danger, right? We wanted to collide. So that's something that we, you know, in that case we want to reach, right? But you know, we want to not be safe. In this case, it's a bit different because we're not talking too much about safety yet. We're just talking about kind of relationships in general. And whatever relationship we have, like if I can show you this example here, we have a can see here. If your test case does not require a can see, what you can do is you can say it does not see, right? You can create another relation and use this approach to optimize for the opposite relation, right? The negation of that relation. So it's really, really flexible. That's what I'm trying to say is that although I'm presenting it in a way that we want to satisfy all the constraints, it's as easy to just reverse the constraint and then we kind of satisfy the opposite, right? So it's, you know, it's not technically easy, but it's conceptually like the idea is, you can just create the opposite constraint and do that. That's a good question as well. So in a nutshell, this is generic algorithms to, to generate the new offspring, select a new population and re-evaluate the fitness and we can and we can, from that, we can basically check again until we reach the termination objective that we're looking for. Sometimes it doesn't reach the termination objective, sometimes it times out so you have like a time limit. If it reaches the time limit then, yeah, you're done but you didn't succeed, right? So you can, that's another kind of case. Now that was genetic algorithm which is a kind of subset of metaheuristic so we're using these kind of randomness-based algorithms to generate these scenes that are satisfying all the constraint. And in order to kind of determine the fitness how fit it is we have different kind of objective functions, different fitness definitions for each category. So here's an example. The example is very straightforward. It's just a case where it's just one constraint. The constraint states that the functional relation, this is the functional relation that we need to satisfy, right? The gray vehicle must be in this kind of blue region, right? That's the objective. That's the kind of high-level conceptual objective. Now let's say we start genetic algorithm, right? We start the genetic algorithm in the first population it places the gray vehicle here. Is the condition satisfied? No, it's not, right? The vehicle is not in the region where it should be. But it's not, you know, we can do better than just saying it fails, right? What we can do is we can actually determine how close it is to the region, right? So we can do is we can say, oh, the fitness here is relatively decent, right? It's close to the place where it needs to be. So this is going to kind of help guide the search approach, right? We're telling it is close to the region. So basically the fitness depends on how far physically the gray object is from the red object from the blue region, right? That's what the fitness depends on. Now, we keep running. Eventually we reach a case where the genetic algorithm places the vehicle here. Is this better or worse? Well, it's clearly worse, right? It's further away. So we're not really getting better. We need to let the genetic algorithm know that we're doing worse, right? We need to kind of help the genetic algorithm out, right? So eventually we reach a point where it places the vehicle. In this case, now fitness is optimal, right? This is the best we can do. We don't really care whether it's here, here, here, as long as it's in the road, it's in the region. So in this case, we know that we succeeded. We can kind of terminate the whole process, right? So, but the idea here is that we're defining fitness based on a custom definition, right? How far it is from that blue region. Right, that's all for the kind of paper-related stuff. Do we have any questions before moving on to final parts? Right. So in the paper, so I have a link to the paper at the end. So if you're interested, you can take a look. You can read that. We do some interesting evaluations in the paper. So we want to generate test cases, right? We want to generate positioning of vehicles. Of course, with a lot of simplifications for now, because it's just a step in the kind of long road. We're doing evaluations over three road maps and these road maps are interesting. We have a simulator called Karla, very nice simulator. It has some maps that are included and the picture is actually from that simulator. So it's a cool kind of map. It has a lot of maps. We're evaluating on this map. The second one is a bit cooler. It's an actual real-life road segment in the world. It's an actual intersection in Hungary where we were able to kind of generate a digital twin of this intersection. So we kind of created a simulation copy of this real-life intersection with the tramway and everything and we were able to generate our scenes in this kind of real-life location. And the third one might be a bit cooler as well. So in Hungary, I'm doing a lot of collaboration with Hungary. So in Hungary they have this thing called Zala Zone. It's a test track. So what this is is it's basically a location in the mountains, not close to any cities. It's a kind of mini city, it's a small kind of city which is designed for testing self-driving cars. So what you can do is you can go there, you can bring your self-driving car, you can create some scenarios, you can actually test, and it's cool because you're not putting people in danger. It's a random location. You're avoiding those kind of real-world interactions. So it's a really controlled environment. Of course we didn't go actually there with the scenes there. We generated them on a simulation of this area. So we don't have the money for that. So these are the kind of three case studies that we did. And quickly going over the research questions, the result we're comparing the scenic kind of baseline approach. Yeah, the three kind of variations with our approach, MHS, a materialistic search. And what we're testing is we're checking our approach more successful. Does it succeed more often? And is it faster or slower than the existing approach? That's the first thing that we're checking. And the second thing that we're checking is how good is our approach? How complicated are the scenes that it can generate? So in the case of the first research questions comparing the existing baseline versus our approach, what you're going to notice is that the success rate of our approach, the purple one, is more accurate than the ones that are proposed by the current baseline kind of state of the art. However, in the case of runtime, what you're going to notice is runtime is a bit slower. So here the other approaches are doing well, but they're failing a lot of times. So the conclusion is that despite us being slower than scenic, we are significantly more successful. So significantly is a key word here. There's something called statistical significance, which kind of it's like a proof, not really a proof, but it's a level of assurance that we're better or worse, right? Certain level of assurance. So we have a certain level of assurance that we're performing better. So these are all the results. You can check that out in the paper. There's a bunch more results in the paper as well, actually. The reviewers didn't really like this, so we had to do more. So in the case of RQ2, what we can notice, which we went very time out, and we were able to kind of conclude that our approach can generate relatively reliably scenes with six actors. So you can have six actors. You can have a lot of constraints between these six actors, and our approach will be able to do that. And the kind of the median runtime is about 2,000 seconds. What is that? That's 300 is 5 minutes. Like, I don't know. Anyways, 2,000 seconds. Okay. So, but another kind of small technical detail is that the search space is growing exponentially, which means that, yeah, we do handle three more actors than the other approaches. No. Yeah, the other actors are limited to three actors, and we do handle up to six, but the constraints make the whole problem much, much more complicated. So if you take a look at the search space, there's 72 orders of magnitude larger. So although it's just three more actors, it's a lot more complicated to handle. So that's a big kind of result. Alright, that's all for the technical part. Now we're at the final section of my paper of the presentation, Current and Future Research Project. So, what I presented today is pretty much what we have already done. The second paper is almost ready. We're submitting it to the final kind of revision next week, but what are we doing currently, and what are our future plans? So, of course what was missing in the kind of the new approach was behaviors, right? We didn't have things that were moving around. We had things that were kind of static. We had things that were static. So we want to kind of integrate these behaviors into the kind of the approach, right? And for that purpose we thought of a new way. We're testing intersections. We're positioning the vehicles in kind of dangerous positions and we're giving them some paths, some trajectories for them to follow and to eventually reach these dangerous positions, and we're going to test whether these dangerous positions are actually dangerous, how well are the AVs working? So that's kind of the current thing that I'm working on, introducing these behaviors into this whole idea of constraint satisfaction that kind of stuff, right? That's the current work I'm doing. Another interesting thing is the influence of initial scenes, right? We spoke about initial scenes. We had these two cases where the behaviors are the same, but since the initial scene is different, then we have like contradictory results, right? So there's some analysis that we're doing in terms of figuring out what the influence of initial scenes are. What are kind of dangerous initial scenes? What are less dangerous? What are the things that we're looking for in order to create interesting test cases, right? So in terms of initial scene we're doing initial scenes because we're doing static scenes, right? That's the relationship, right? Initial scenes, static scenes. And another thing that's not really related to these kind of scenario generation is we're using our generated scenes in simulation to provide some test cases for image detection, right? We can simulate the scenario. We can get some screenshots. We can even have these running like behaviors, right? And we can check whether the image detection is working for some image detection algorithms, right? So in this case the algorithm was able to correctly detect the traffic lights, 73% accuracy. It was able to detect the cars, 98% accuracy. So these are kind of test cases that are kind of in another sector of software engineering, in another sector of software engineering. But these are kind of more what we're doing in the near future, currently in the near future. We have other kind of more longer-term plans. These are more simple projects designed for undergrad students, probably caps on projects and that kind of stuff. There's a lot of work that needs to be done for the input language, right? You can add constraints, you can add new cases, right? New kind of types of scenarios that we're interested in. And validation, right? We have the mathematical background. We want to implement the validation as another kind of next step. Scene generation is another interesting one. We can have custom algorithms. Yeah, we spoke about genetic algorithms that we can do a bit better, right? A lot of people are doing those kind of things. And simulation is the other one. We want to integrate simulators with real AVs. We want to kind of have as many real AVs as possible to have as many test cases as possible to make it more interesting. And we can extend the scenario with behaviors, with different kinds of behaviors. So it's, you know, I speak about it like this, but it's a lot of work and a lot of debugging. These tools are not very user friendly sometimes. Yeah. All right. I think I'm in time. That's all for my presentation. Just a quick recap. So we started off with the conceptual section. We talked about scenario-based testing. What is scenario-based testing and how we can leverage that for our approach? What are the problems with scenario-based testing and what are the solutions? We discussed one kind of approach that's a model-based approach for generating these conflict scenes, this kind of dangerous scenario. And we spoke about these domain models and everything, how we represent the problem, how we solve the problem again. And finally the last thing I spoke about was the three levels of abstraction and defining these scenes that are satisfying different constraints. Again this is an optimization algorithm-based approach. So we had to leverage these kind of abstract level. All right. That is all for my presentation. So I'm going to leave you all with this. This is me. A lot of people have been adding me on LinkedIn, so feel free to add me on LinkedIn. And I put my emails as well anytime you want. You're interested in self-driving cars. Just send me an email. I'm interested in self-driving cars. I'll be happy to receive that kind of email. I have the tool that we've been discussing these constraint satisfaction tool. Here's a link to that. And our recent paper which is a pre-print. It's not yet published. It's a pre-print. You can take a look at the paper as well. So it's an interesting paper. I like the paper. One of the reviewers is not liking the paper, but I like the paper. All right. Thank you very much. I'm open to questions. Great. Thank you. I have a big picture question. Not particularly technical, but I think having the technical insight that you have and the people you work with in different places have the close answer. Your answer will have that perspective. So when will we see... I feel like if I turn back the clock, 50 years or 20 years or 10 years, I don't know when driving this car is science fiction. First of year, 100 years, I don't know. In recent years, I feel like we've had maybe the last decade, we've had incremental changes. I'm using our incremental because if I walk out to the street now and when I look out in the air balloon in Montreal, I mean, you look out. You see people behind you. It looks like everyone's driving. Yeah, we have driver assistance. We have some driverless cars in some places. Or maybe someone's sitting there, but it's actually fully on what we call it. What year or day or moment do you think we're going to see a qualitative difference in how things go? What will be the cause of that? Do you think it's just been to continue to be incremental, incremental, incremental and in 10 or 50 or 100 years from now there won't be human drivers or something like that? I'm curious about how do you think and it's not only the technology of course, but how do you think what's to come? Yeah, that's a very good question. A question I asked myself as well. I think there's a good observation that there's incremental changes. That's a good observation and especially in the last 10 years, a lot of companies are investing a lot of money into this. So one thing that I would like to say eventually I'll do an answer to your question but the thing is that companies want to sell cars. So they're really advertising, they're working a lot they're advertising, marketing is amazing. A few years ago, Tesla came up with some full self driver, a full self driving plan that you need to buy. Technically speaking, full self driving means that there's no driver but the plan was nowhere new that like it was just marketing. People have the feeling that it's coming soon or we're almost there. Maybe technologically we are close, but there's a lot of other things that are involved that are kind of preventing that. Safety assurance is one of that one of those things. The vehicles can work and 99% of the cases, let's say, but that 1% is what's preventing everybody to get self driving. Of course there's the other case where it's costing a lot of money these days but I expect that to kind of get better as all the companies are going to start doing this. Eventually the cost is going to decrease but the case where everybody's just sitting and not touching the wheel with no kind of regulations I think that's very far and I don't think there will be like a trigger. So currently the things that I propose are really conceptual. So companies are working to implement these things but nobody knows if it works on a larger scale like nobody's tested that. So the kind of conclusion to all of this is we're still kind of not sure what's going on in terms of testing, in terms of safety. Until we reach that point where we're sure and reaching that point is a very complex process. You have to have an idea you have to test it, you have to test it in other places, you have to test it more again, again, again. The idea doesn't work, you have to improve the idea so it's a lot of iteration so the kind of comment that it's going to be incremental is by nature. We won't have somebody figures it out and implements it and test it on scale from one day to the other. So it's a big process that costs a lot of money. I think it will be an incremental thing. Will we see that in our lifetimes? I don't know. I don't want to say anything about that but there's a lot of question marks. Even though it sounds nice, it sounds like it's marketed well, it's still a lot of safety question marks with respect to this Yes. So maybe you could have a bit more insight on this. What do you think is going to happen to the problem of ensuring of the cars if full self-driving cars are available? I don't think we have that case here in Armenia yet but maybe it's a little different kind of So ensuring in the case that, who's at fault for example? Everybody has to have insurance, it's part of the law. Let's say if something bad happens, does that get on the company, on the software, on the person? Are you raised different, who pays for it? Maybe you have a little more insight on this. So I have a few friends who ask me about this and it's hard to say. So currently from my understanding in Canada at least it's the driver that's responsible because at the current level of self-driving is the driver has to be sitting down, hands on the wheel, aware of the situation. The responsibility of the driver and the reason why these are the regulations is companies they don't want to keep themselves safe. So companies they know that they're not super safe, 100% say the vehicle is not safe. So they set these requirements to make the driver the person to blame. So I don't think these kind of insurance things will change unless there's like a huge kind of everybody starts using this. In that case it will change but since it's still like at early stages companies are trying to kind of avoid problems. And drivers they know about this. They know that if they do some mistake, if they sleep on the wheel like this and then they get into a crash, they know it's their fault. So it's an interesting state but we're in a transition period. Who knows what's going on. In fact it is. As you said it's not very interesting but if there is more possibility there are more than two possibilities two or more possible accidents that are impossible to debate. For example human drivers are responsible for the crash they made and how is the responsibility of the set driving and how it chooses which one is the philosophical question how it chooses which crash to get into. So one interesting thing is that when we explain the problem as human beings you can clearly understand what the two cases are. Like let's say you have an old person and a baby you can understand a human. The vehicle doesn't understand these concepts. It just sees a picture and from that picture it measures a metric and based on that metric it makes a decision. So it doesn't really reason about these things it's just the way it's been trained. Maybe at some point during the training if it saw another picture or saw something else it would have made a different decision. But the one kind of comment that I would like to make is that computers know reason like humans computers reason at another level which is kind of more technical level. In the context of machine learning does it generate its core that it chooses the... Yeah something like that. I'm not too familiar with that but it's really the technical things I don't know but at a conceptual level you just think of it as a camera you feed the image to the machine learning or the feed the image the object detection algorithm or whatever and then you don't know what comes up and you don't know why it's coming the baseline is that it's still randomly spaced still based on training you don't know what the training is. Sorry for that question. Yes. Related to what you just said when the computers think they think differently they think differently Yeah. Do you think it will become easier or more difficult to have to program and to train or to troll the shoe autonomous vehicles as there are more of them on the road and then once we have autonomous vehicles on the road what do you think will be how will that be dealt with in terms of improving them or optimizing them does that make sense when they have to deal with human drivers are they thinking and acting differently than when they have to deal with other computer driver other computer driven That's a good question so actually I mentioned this a few days ago imagine a case where you have a city where there's only self-driving cars so in this case you can leverage a lot of computer specific things for example every vehicle can know the exact position of every other vehicle so it can ensure that there's not going to be crashes or it can prevent crashes but that's a case where you're dealing with other computers so in that case you don't even need cameras you just need a GPS but when you're dealing with humans it's different because you have that technological support so you need to deal with more rudimentary ways and if you take a look at this if you take a step back and try to understand what's going on in the world of self-driving cars it really looks like it's a computer is replicating a human you have a camera a human has eyes you have those kind of sensors you have the steering as if you're moving the vehicle it's the same operations as a human you're just trying to replicate humans I think this is kind of a general trend in computing neural networks neurons in the brain it's the same kind of trend so that's where the inspiration is coming from and you know what it's working so when you try to handle these things it's kind of an inspiration but you also want to leverage what the computer is capable of you understand neural networks as a concept but you can't replicate that with a pen and paper so you have to take advantage of the good things with relation to the computer but it's still really very much influenced by the human aspect yes do any of the models take into account the actions of other drivers let's say perhaps trying to avoid an obstacle on the road there may be a number of drivers and there may be one obstacle all lying to get into the same space to avoid a collision that's an interesting one so that's a multi-step process so let's say you're a self-driving car you have to detect the obstacle you have to detect the vehicle you have to think like the next step this person is going to avoid that I can't tell you a yes or no answer I don't know but this is certainly something similar to what I've been seeing we're not only looking at the picture as a picture we're looking at the picture trying to extract as much information as we can and making assumptions according to the vehicle so that's a good idea I don't know if it does exist currently if it does it certainly is at a very conceptual level just some proof of concept ideas but this is something that I know about it's an orientation that people are trying to go in I tried to do something related but it was more difficult than what we thought but I'm sure companies are working because if we can do we can try to understand intentions that's a big bonus currently we're not there yet but that's definitely something that people I think are working on so if there's anyone in the room doing a capstone this year or in near future years are at the beginning of the talk maybe at the end as well is sort of an outreach to us for potential collaboration in general I don't think it's good for all of us to get inspired by research that's happening outside of our four walls it's much easier now to collaborate with people outside of your university outside of your immediate field I don't need to list 20 different non-computers on its fields that have some connection to what we're doing so I mean collaboration across buildings and departments fields so anyway thank you all for coming thank you all