 Hi everybody I'm Langdon White and welcome to dev comp CZ if if you just came for me. That's cool I really appreciate it, but I really really hope you've been attending the other sessions. I'm sure You know, whatever you attended was great. I know the things that I went to have been really awesome We're here to talk about some research work that some students have been doing and It's particular. Sorry. It's a little project called profit And what it's about is trying to analyze microservice applications We have a bunch of speakers as part of this keynote But we're going to lead off with the kind of the professor who's behind the project Professor Tomas Chirney and he's going to come on and introduce the Project and then we'll introduce all the people who are speaking kind of as we go so Dr. Chirney do you want to? Share your screen or share your video for how everyone and So I'm gonna tell you something about a project that we are involved in it would jump to the next slide like that Okay, that's me and now about a project So I believe that most of you are here today because we shared the same passion about microservices We like to develop microservices So what we can typically do with microservices in nice is nicely divide the team So that someone is developing one microservice another team is developing another microservice and so on why we do that? Well, because this is the mainstream direction these days, right? So what do we know about microservices? They are self-contained and we can deploy them anytime we want Well, this has many benefits, but it could lead into some sort of issues, especially when we have multiple microservices So what could what could happen here? Well, I like to talk about it like a black box view versus a white box view So if you are the developer of a particular module, you of course know everything about your module But if you are talking to the team that is developing some module that you are interacting with you don't know much about it So you can see it as a black box, right? and the sort of same situation Is happening when you pass off your microservice to the sysops and they are deploying it They know everything about the deployment They know everything about the platform, but they don't know that much about the specifics and specific aspects of your microservices So there are times where we would like to have the holistic view However, microservices will not provide us with a holistic view. So next slide please Langdon and Naturally, the question comes how can we get the holistic picture of the overall distributed system? So there is something that is called Software architectural reconstruction and as you can see, this is sort of a process where we want to build from this whole Distributed cloud the holistic picture of what's going on. Well, we could do that manually And this is how it's most done most of the time these days But what if you would like to sort of make it automated? So let's look into what would the holistic picture give us if we have it. Let's go to the next slide So first of all, we understand the system, right? For instance, we want to modify something and we need to locate it first We would like to eliminate the deficiencies and there could be especially because this is a distributed system And it was optimized per module rather than holistically Conformance verification. So how do I know that one module is enforcing something like the other module? We would like to extract the documentation unless it exists But I believe it doesn't exist because we are developers We don't want to deal with documentation of ever a changing project, right? For module overview. So what are the dependencies between modules? Are there some relationships? We would like to do optimization Very common, especially for the distributed system We developed it scales we deploy it and then we get our first bill and we are not happy about it, right? How many of you have that experience? For security assessment very big topic these days and so on So let's look one more slide before I pass off to the work to my colleague So profet is the tool that we are introducing and that we are developing and experimenting So the idea is to automate the extraction of software architecture reconstruction For microservice systems that are distributed. So how do we do that? Of course, it's a holistic So we pretty much take each individual microservice We analyze it from the code perspective. Currently we do that for java platforms and from there We try to build a holistic model and We will learn in a second how to do that But maybe I will mention one more thing. How do we do these things? Well, we could do code analysis with the source code or with the byte code And we do both actually and it is a different use case One fits very nicely to the continuous integration for the software development lifecycle So if developer does something we can holistically assess the system If we have some functionality which we have and we will show you in a in a bit and the second Option is that we actually do that on the bytecode level So this goes to the integration for the Kubernetes for instance, or is still and the defaults Sorry, the sysoft could actually benefit out of this So let's move on and dipta is going to tell you something about how how the prophet works. So thank you and enjoy Thanks so much And so as we kind of moving through these we want to kind of talk a little bit to the You know kind of the next speaker So dipta tell us a little bit about what you're doing currently at baler or and You know kind of how you might have got involved in the project Yeah, so I am doing my master's at baler and since 2019 And since then I am working with dr. Chani and on this profit project Cool, and what is it that attracted you to the project like what did you find interesting? Uh, I think the most interesting part is Generating this holistic picture of the microservices some with some kind of automated process because it's Really important to automate the process for a large number of microservices Yeah, I yeah, I definitely agree. Um, you know, uh, why don't we let you kind of go on to talk about your component of the project? Yeah, uh, you can go to the next slides. So as uh, dr. Chani was discussing profit was properties merging the microservices so Uh In profit we have like three perspective of much merging the microservices and the first one we actually merged the data models and in the second step we Identify the remote calls or rest calls and merge them and in the first step we actually analyze the deployment descriptors or Kubernetes configuration files to merge the microservices So we can go to the next slides where we discuss about the bounded context. So we merge the bounded context which Which is done through the static code analysis for source code or bytecode and then For each microservice we extract the data models and then we merge the data models together into one large data models, so We do so, uh, by analyzing natural by, um, analyzing natural language processing or finding the similarity between data models the next slides, uh We yeah, yeah In the next slide here, we, uh, talk about the Margin the rest interactions among the microservices So here we also use the static code analysis, which is less costly and more faster And this generates us the service view of the software architecture reconstruction, which i think andrew will discuss more later And in this step, we have two phases in the first phase We scan the rest endpoints and rest calls of the microservices And in the second phase, we match the url Static type and parameters of rest endpoints and rest calls In the next slide, we'll talk about the containerized microservices So this is important because not always we have the access to the source code or byte code And what we have only the deployment files for example kubernetes deployment files or docker files So it is still possible for us to Generate this rest interaction graph for the containerized microservices So what we do for this we analyze the kubernetes Deployment files through a kubernetes client and we extract the container name into point path and service discovery after that We extract the executable jar files from those containers That is located on the interpoint path and we do so by mounting the docker socket Into a separate container which ensures that The same container is being analyzed that is running on the kubernetes cluster after we extract the jar file or byte code The next steps are the same as we did before So in the next step, I think andrew will discuss more about the software architecture reconstruction Thanks dipta And we'll give Andrew a second to to click the right buttons And thanks for being on the show with us And uh, why don't you tell us a little bit about what you're doing at bailar? So could you tell us what got you involved in the project? Yeah, so I started working with dr. Charny a couple years ago And it was just kind of an exciting way to look at things beyond the classroom And what we don't want we get to work with and then it gradually kind of grew into this thing that we realized Had real ramifications and industry and had a lot of great applications So Okay, so that's that's really pretty cool. Um, why don't you tell us a little bit about how you are interpreting? Sorry, what was that? Oh, I'm sorry. I think uh, we're getting some inbound microphone If somebody could check their mics and make sure they're not sharing their microphone So It's kind of odd, uh, so uh, yeah Hopefully the audience is not hearing this and it's just us Why don't you go ahead and talk about the interpretation that you do and I'll switch to the next slide So what dip does describing there is Really a process called software architecture reconstruction And this is where we construct a very simplified overview of an application Now as dr. Charny mentioned this can really be done In a manual or an automatic way and the vast majority right now is done manually And this really isn't applicable for a microservice application because they just kind of get way too Big and complicated and developers have a hard time understanding the entire system And this is done in four phases and to start with the extraction phase This is where you get kind of the biggest divide between an automatic and a manual system So the extraction is where you pull anything that you're going to need For the different phases that are left in sar and this can be Really anything. I mean it's truly unlimited. It's just a matter of what kind of views you're wanting to build from sar and so In a manual way you may pull things like uml diagrams or read me's or other You know brd's different documents about the application in an automatic system, which we're doing we kind of have to be A little more selective because we can't really interpret something like a uml diagram So we pull stuff like the code and the deployment The documents and we use those automatically Through some like, you know, like static code analysis dynamic code analysis and that lets us create these different views And so we move on to construction where we actually do that and we scan all these different source code and the related documents and we construct the different views for each of the individual modules At this point, we're just creating each module like its own little monolithic application But in the next stage, which is manipulation, we're actually going to start combining these different module kind of views in themselves And so we're able to build a way more holistic picture by starting to match the different aspects of the different microservices And so at the end of the manipulation phase, we have this huge overview of the application Which consists of a bunch of different views, which I'll get to on the next slide, but That lets us get into the analysis phase where The analysis phase is really just what questions do you want to answer about an application? And again, this can be kind of whatever you want What you build in the previous phases is really going to depend on what questions you want to answer And later in the presentation, we'll demonstrate a couple questions that we've been able to answer with them So Langdon, if you want to Yeah, so I want to talk briefly about the different views that we're going to be building And basically we have four that we focus on and we feel that this allows us to get the most bang for our buck basically and be able to Kind of answer the most amount of questions with this minimal overhead So the first one we do is the domain view, which is basically just all the entities of a system And then we kind of go through this process of merging them using, you know, some field matching name matching a little bit of nlp And we kind of start to find the overlap in them and are able to identify You know, kind of how they interact as a whole and build a giant domain model out of that Or kind of just see the overlap and see what are the commonalities like in figure one where it just pulls out what's the same We also have the technology view, which is where we kind of map all those You know api calls between the different modules and we're able to show the different dependencies based on api calls between the different microservices We also have the service view, which is basically um Sorry, I'm going to spoke the service view is all of the api calls between the different microservices the technology view is what did you actually use to implement those microservices and so this can be You know, the very thing about microservices is they can be implemented with whatever language you want Whatever version of the language you want, you know, whatever frameworks you want But there does come a certain point where you may be going too diverse And so that's something that we wanted to look at and so the technology view is just what are you using for You know your front end or your data storage or you know, the actual service layer architecture And so then service view was the api calls and then operation view is kind of The deployment of the application and so it's looking at things like the docker images docker files kubernetes deployment Uh configuration files and just understanding how it is that the different microservices are deployed Sorry, I also had to find the buttons to come back on the screen. Uh, thanks so much, andrew And uh, you know, hopefully that will be the end of our technical difficulties for the day Maybe for the conference, uh, but we want to you know, we want to hit it hard for you You know, or maybe one of our speakers is actually on the trip to mars and They got some background noise from that Vincent, uh, thanks so much for coming on. Uh, why don't you tell us a little bit about what you're doing at baler? Certainly. I am also a second year master's student at baler. I started in 2019 and I've been involved with, um, Dr. Trinity slab and the profit projects since the beginning of 2020 Cool. And what was it that kind of attracted you the project? What was interesting for you? Well, for me, it just seemed like a very practical project. Obviously microservices are pretty much the de facto standard for a lot of new development Not all that but obviously a lot of things now depend on that so anything to make that easier to understand and dissect especially for like Um onboarding new developers onto a project being that faster. I thought that was a really cool thing Neat. Yeah, definitely definitely a challenge. Um, so why don't we go ahead and get started with your segment? All right, so I'm just going to talk a little bit about Um, the project itself and the prototype we have deployed. This is just a uh, simple little demo of the project prototype that we have right now Um, so first off we have a couple of different, uh, test beds that we test against couple repositories the tms project what you see here Um, that's a simple in-house microservice benchmark that we created very simple just three services for a proof of concept type of thing um, the train ticket repository Is actually a more extensive externally made repository. It's got about 40 microservices in it. So, um It's going to be a lot more extensive in the kinds of analysis verification. We do on it We're going to actually use that a little bit later on in the talk later But for now, we're just going to use the tms for a simple demonstration So right now in the visualization aspect of this we're focusing on two different kinds of diagrams communication diagrams like you see here And then below that we also have context maps or domain diagrams Um, and this is really just visualizing the results of the software architecture reconstruction process itself So in this communication diagram, this is that service view that Andrew was mentioning You see that we extract all the different services. You see the cms ems qms Those are the three different services in the test bed and then we extract all the different calls between them So that's um, all the communication that happens among the microservices You see that we get um, the hdp methods and the methods that are called in the services Below that you have the context maps and this is just an extracted domain diagram For the entire system what we do is we find all of the entities or all the objects that are acting as entities within each individual microservice And we find its properties from relationships, etc Each individual service has a different view of the data so you can see below there We have domain diagrams for each individual microservice So we go into each microservice Find all of its entities and its relationships And then we kind of aggregate those together into a single combined view So we find what relationships exist Among entities across the microservices Um, and then we combine those into a single place And both of those diagrams again are just visualizations Of the software architecture reconstruction process. So on the next slide Um, I just have a bit more of a Expounding on what I was saying there about the domain diagrams And what we're doing. So if you have two different services that have these entities. So service one has Um specifically looking at the question and entity. It's got a lot of relationships To a lot of other things and then service to the question entity doesn't have nearly as many relationships But it has a new one that the first service doesn't have the question to exam relationship So essentially we're just Extracting those services or extracting those entities and then we're combining them together and the combined result on the right You have all the extra relationships that service one has but also the exam entity that was in service to That's just a basic overview of what we're doing there. So on the next slide what I want to talk about briefly is Where we'll be going next so if you can move to the next slide So right now as I mentioned, we're only Visualizing that bare-bone software architecture reconstruction itself just the those different views that Andrew mentioned In the future, um, what we're doing right now is analyzing different system concerns So, um, we're going to talk about security in just a second. We're going to talk about code quality through looking at code clones in a minute These are other things that we want to analyze and that we also want to include in the visualization process So we want to show a way to Share that information with the user We also want to look into some other formats one thing that we ran into when we were analyzing the 40 microservice benchmark Is that 2d graphs get very limited very quickly in the information that they can concisely display So we want to look into some other formats. Um, for example using 3d or some kind of ar or vr setup In order to help users kind of navigate these microservice meshes We also want to make it a bit more interactive of a process So allowing users to form some kind of query about what exactly they're looking for So they'll be able to narrow down the results instead of Well, they'll be able to start with the large overall graph But then be able to slice it down to what specific services are they interested in What concerns do they want to look at and what relationships between the microservices are they specifically interested in So those are the kinds of things that we'll be pushing in the future on this project back to you, Langdon So we can expect to see a vr version of this in the vr For the conference Is what you're saying. So if we go right, yeah, it's live right now You can you can go into the matrix right now and hack what that's all you want Right, right. Good to know. Um, well, thanks so much. Uh, and we'd like to welcome dip to back to talk about kind of Some example uses we're going to go through a couple of different Scenarios and dip is going to lead us off with endpoint security Yeah, so in the next slide, we are going to discuss about Some application of the software architecture reconstruction that prophet was doing so one of them is Analyzing the urbex inconsistency in the apn points. So we defined five Defined kind of violation or inconsistency related to urbex policies So the first one is missing role violations, which happens when one apn point do not have any urbex roles associated to it So it can be false positive because some endpoints are really public and that is intentional However, sometimes developer forget to Provide those api roles on the apn points And the second one is unknown access violation That happens when you define a urbex role on a apn point But that urbex role doesn't present in your role hierarchy as role as the role hierarchy shown in the left In the right and then the third one is entity access violation, which happens when Two apn points have same input and output. That means they have the same response type and request type But they have different urbex roles and these three Violation happens for the end point methods and the next two conflicting hierarchy violation and unrelated access violation happens for the intermediate methods So hoyn and intermediate method Have two different roles that are That are Ancestor of each other's for example user and advent. They are the ancestor of each other's When this happens, we call this conflicting hierarchy violations and hoyn two Different intermediate method have One intermediate method have Two roles that are in different sub trees. For example users and user and Moderator these two roles are in different sub tree. We call this unrelated access violation. So in the next slide we Talk about how we detect these kind of violations So first we extract the risk interaction graph and method call graph by using the static code analysis And we also extract the urbex roles associated with each apn points After that we combine the method call graphs by utilizing the risk interaction graph And then we perform reduction where we just keep the lowest role of each apn points And then in the aggregation phase We push the role from the root to the leaf node of the method combined method call graph And then we analyze each intermediate nodes and each apn point nodes of the combined method call graph for and look for those violations The next step the next slide we talk about how this combined method call graph is prepared So you can see that we have the application x and it contains all the jar files So we assume that each jar file Describe one microservices. So here we the discovery module. We have the microservice a and microservice b and we have the method called separate method call graphs of those two microservices And then after the flow match our module We have the rest interaction between these two microservices what is shown with the arrow and this is a get request as indicating in the arrow and then after the analysis module We perform aggregation and reduction and here you can see All the method call graphs are actually combined into one By analyzing or by using the rest interaction graph and in the next slide talk about how we are performing the aggregation and reduction So here you can see In the role we have the control method one which have role a b c and the control method two which have role pq and In the after the reduction phase, we just keep the lowest one. That means c from abc and q from the pq based on the role hierarchy graph on the left and then after the aggregation phase we put down all the roles from the root to live and we also combine the role c and q in the Repository method zero one here. So here the repository methods zero one have the two Two different roles c and q and we can see that c and q are in the different subtree of the role hierarchy graph So we can call this as unrelated access violation And in the next slide, we have a small demo how we are Running this application to detect against a test bed So London, can you just play it in the full screen? Yeah, so here on the left We are running our application and on the right we will compile our microservices So this is a springboard project. So we just run maybe a springboard run And it will run our project on the 8080 port and on the right We have the tms test bed. This is the same test bed. Uh, Vincent was talking about so Let's see. Uh, what do we have inside this project? So you can see we have four different microservices cms ems qms and ums all Have separate folder and there is a build script So we just run this build script and it will prepare the jar files for each microservices So it will uh, take some time to compile all those microservices only one and prepare the jar file for each microservices So as you can see we are building the cms and now it is building the ems project And then it is building the qms project. So all four microservices are Prepared for analysis and then we'll send just send a rest request And here we can see It is the past Part to the jar files where all the microservices or all the jar files are located. It's the same path here and then This organization path is actually used to filter out unnecessary or third party libraries because we just want to analyze our own project Not the third party libraries and then the security analyzer interface we define or we Uh specify the role hierarchy graph as we saw earlier and the output part is the where we actually generate the graph this file Here we got the response From our replication and all security method actually contains the meta data after aggregation and reductions Who we are not actually interested in this. We are mainly interested the constraints violation here So as you can see the constraints violation, uh, we have all those uh inconsistency or violation And here you can see the concrete configuration method have two roles admin and users and those are actually Ancestor of each other. So we call this uh hierarchy access violation and similarly In the find all configuration method We have moderator and user role after aggregation and reduction And these two moderator and users are actually in two different sub trees of the role hierarchy graph So we call this an unrelated access violation and finally Let's see the output path. It's in the download folder. So just move into the download folder Here you can see the graph this file is generated. We'll open this And here's the graph this file. We'll just visualize it And it will show the rest interaction graph Among the microservices along with all other configurations. So it is the same Uh thing as Vincent was showing with uh with a little bit more details So thank you for watching this demo. It's longer to you Apologies for that I'm not sure why it's uh Decided to act up on me. Um, apparently it's the technical difficulty day. Um, so uh, we'd like to welcome Andrew back to talk about code smells And this one is a true test for me. Uh, where we are going to go rapidly through Various, uh, code smell examples. So, uh, Andrew, go ahead and take it away Yeah, thank you. So It's always great to hear that they're talking about, you know, very applicable things like security our back role consistency Um, but I like to talk about things that aren't so perilous But equally as important and it's something that a lot of developers. I think Don't really take into consideration And that's code smells. So I know no one really likes to deal with code smells Because they're really just poor programming practice. They don't really impact The functionality of an application But they do impact the reusability the testability and the maintainability And so it's something that people need to be aware of But it's often difficult to kind of lock them down and even more so in microservices So next slide. Yeah, so microservices are kind of unique because not only does each individual module Have all of your traditional code smells. You also have microservice specific code smells that deal with kind of the interaction of the modules and not just the code within them And so traditional tools Really can't work on these because you need a way of generating a holistic view of the application In order to find these so that's where our sar process comes in and we're actually able to find them So I want to go through the 11 code smells that we detect And hopefully you recognize them, but don't think to yourself. Oh, this is something that I just did on the application I'm working on But I just want to demonstrate that these are things that are important and can be found with our process So to kick things off we have enterprise service bus usage I know esb's used to be fairly popular back in like service oriented architecture But now they are considered bad practice. They provide a pretty single point of failure Which is not something that we want. So you want to avoid using those We have too many standards kind of like what I was mentioning previously You know, you can have anything you want to implement your different microservices But at a certain point you may be getting too diverse and you need to kind of reign it back So that is more seamless for your developers Wrong cuts is kind of a tricky one It's where your microservices are cut along the presentation business and data layers Of an application instead of by feature So you really want to make sure that they're cut along the features and each microservice has its own presentation business and data within it Not having an api gateway. It's another big one Now if it's a really small application, you're probably fine But research has shown that over 50 microservices and developers are no longer able to accurately keep track of the system So at that point you want to start using an api gateway just to make it simpler on everyone API versioning You should be versioning your apis. It's helpful for everyone involved You know qa deployment users developers everyone benefits from having a versioned api microservice greedy is another interesting one. It's a little subjective It's where you have so many tiny microservices That it gets to be a little unwieldy and it would be better to just kind of aggregate a couple of them Into some larger microservices that are better suited and easier to manage Shared persistency is where two different microservices are accessing the same data in a database Hard-coded endpoints is when you hard code an endpoint that you call Um specifically around stuff like ip and port because once you start deploying that can always change And it's something that is way too brittle for a real enterprise application Inappropriate service intimacy is kind of similar to shared persistency, but it's a little different It's where each microservice has its own database But one of them is also accessing some private data within another's database And usually this is done just because it's the easy way out And developers are lazy and don't want to write the proper channels to get that data But this one in particular is really bad and you want to avoid it Shared libraries is another where if a lot of microservices are using the same library It might be better suited to be deployed as its own microservice and then could be scaled as needed Ciclic dependency is super self-explanatory. It's where your chain of api calls winds up making a cycle between microservices And that's all 11 of them again. Hopefully you recognize them, but don't think that you've implemented any of them recently If you want to check we do have a publicly open source tool called msa knows at our labs github repository, you know, and we always welcome Feedback and pull requests and you know anything like that. So With that i'm going to turn it back over to lengthen Thanks so much I don't think I screwed it up too badly. So that's nice And so what we'd like to do is welcome our next speaker So, uh, michael, why don't you tell us a little bit about why you're at baler or what you're doing at baler? Yeah, so i'm actually in my first year of the graduate program at baler university I actually just graduated there last may with my undergrad Uh, I was in the same class as andrew and so I decided to come back and Learn a little more from dr. Cherney and the rest of the people at baler do it all again So what attracted you to this particular project? Yeah, so um in my undergrad um, andrew and I we began study on The limitations of static code analysis And so I really got intrigued by that especially since a lot of people don't really use static code analysis At least it's not as much as they should be using in their software And uh, it's because these limitations that uh our presence and so from this, um, I was able to branch out and Kind of look at the limitations of these static code analysis tools over microservices, which is what I'll be talking to you guys about today Cool. Yeah, so, uh, why don't you go ahead and take it away and tell us about The work you're you're planning on doing for your I think for your masters, right? Yes, sir All right. Yeah, so as I just said with landon, there are limitations in static code analysis Here i'm focusing on the limitations of static application security testing over microservices So static application security testing is used to find vulnerabilities in software However, this foreign testing is not enough to gain coverage of security vulnerabilities in microservices So since the static application security testing only looks at the source code of software It can miss potential vulnerabilities that might exist due to the communication between modules that is present in a microservice architecture Due to this limitation Developers are required to use dynamic application security testing to find the vulnerabilities that the static form were unable to detect So the problem with this is that that is that the dynamic application security testing is a much more costly form of testing Than static application security testing due to its dynamic nature So that is why we've begun research to find a way to limit this need for dynamic application security testing So in order to limit this need we need to we have found that we can combine static application security testing with software architecture reconstruction In order to gain more coverage of the security vulnerabilities that might become present due to microservice and interconnectivity So we plan on combining the profit tool that we've discussed earlier With the fabricate analytics quality assurance tool in order to achieve this goal So the fabricates analytics the fabricate analytics quality assurance tool acts as a framework for static code analysis tools Which will allow us to customize our testing To get the best coverage out of the static application security testing tools that we are using Now in this study we intend on showing how this combination Can limits the need for the costly dynamic application security testing that I had mentioned earlier So like like landings landin said this is just preliminary research that we've begun over this and i'm excited and trying to figure out exactly how this is going to work And and uh writing up this for my thesis awesome, uh, thanks so much um So I think that was pretty much the kind of content we kind of had for you today And we are basically over time. So, uh, I wanted to Welcome, uh, dr. Churny back to the stage for a minute Just to kind of say hey, you know as per all open source Looking for pr is looking for more people to get involved. Please do And uh, you know, dr. Churny, do you want to kind of give your plea for how how awesome this could be? Oh, yeah, I would like to thank everyone that they joined us today and of course, you know Get your hands on I'm sure that you are bored with what you have been doing. Let's say for a couple of months And if you have a Friday afternoon of anything like that you could join us We would be very happy. We are looking for collaborators. You can participate on the grounds as a Some advanced collaborator who has the technical knowledge. We're exposed to Practice we would love to help Your help and work with you on some tasks that you perhaps can define So you can come and say hey guys, you know, we don't really have time to do that But I think this would be a great project. What we also have been doing is to Employ some red hat volunteers as a co-supervisor. So I have been working with multiple of you Already on some projects that you are doing and we had summer Surge that we are doing with red hat, which is very exciting and in fact Andrew and michael came from from that sort of incubation of being good developers eventually And we are looking for transition to practice. So all these things really Just reach out. We will be more than happy and there is never a bad idea. There is never a bad question So thanks so much for giving the talk As I said, we're pretty much out of time. So if you had any questions Please try to find us either on kind of the general chat inside hop in which I know is kind of rough Or if you can find us in the keynote questions channel in discord There's so many now instant messaging platforms that I have to think about which one I'm currently talking about So in the discord, I will at least be there or we can follow up with any questions that you might have later And I would also like to thank the presentation and power gods that we just had a couple of technical difficulties All of our speakers were in texas Which is currently having rolling blackouts in a lot of places So I am very happy that we were able to be this successful even if we had a couple of challenges So thanks so much for your time and we hope you enjoy the rest of devconf Don't forget to participate in the kind of Extra events, you know as well as going to the talks You know and get your yoga on and we'll see you around. Thanks so much. Thank you