 Joey, is the stage to see us. Okay, thank you. So I first wanted to just get a general idea, and I think that a lot of the people here are dev-in-developers, and probably everyone here knows what testing is, I hope. So I don't have to give AJ's talk, too. If there's anybody who isn't really clear on what testing is, could you please raise your hand and I'll go a few, a brief rundown of it. Okay, none, great. So as we know, testing is an automatically stabilized version of unstable, and it's pretty much unique among all the Linux distributions out there. There's really no other distribution than I know of, which tries to just take a program and look at the data that's available and generate a distribution out of that, like we do with testing. And it's actually worked pretty well, I think, in a lot of ways. We've actually managed to make, I think, two stable releases now using testing. Of course, there are some problems with it. One of the big problems with it, which came up as soon as we started producing it back in the year 2000, was that it's basically completely insecure and it's thought there's really no way to make it secure. It's basically insecure by design. So there's a lot of different problems which I've listed here. For example, any dependency between a library package and another package can block a security fix from testing. You have to wait for the library to get into testing first before the package that depends on gets in. And there's a lot of things that can hold those libraries out and you can get very deep library chains, especially if you have security holes in GNOME or in anything else that depends on lots of stuff. And one of the other metrics that's used in testing, of course, is the number of release critical bugs in a package can hold it out of testing. That's a real problem if you have a release critical bug which is really, really critical. But then you get another security hole on top of that. And now you have some maybe really hard to fix bug which is completely blocking some simple security fix from getting into testing. The next problem here is there's a testing delays. Packages don't reach testing until they've been tested in theory for 10 days. We can knock that down to two days but it still is a delay there. And it can be a bit of a problem if you're trying to just rush out the security fix for say a remote hole in the kernel or something. And so can the auto builders because before a package can reach testing, it has to be built on every architecture out there which is a big problem if an auto builder's down or if it's just slow or whatever. And the final problem here which I think is really probably the biggest one is that we don't try to really try to secure unstable either. We do our best effort at it when it's pretty good and the security team occasionally goes and checks to make sure that something's fixed and unstable before they issue a DSA for it. But we don't have a security team for unstable either. And so all these problems have basically kept people from using testing to any large extent or at least from admitting to using testing. And actually since the camera is pointing at me and not at you, I'd love it if you would raise your hand if you do use testing somewhere. Look at that. Until Sarge released. Okay, well that's fine. Let's say if you've used testing in the past year, let's raise your hand. So it's almost the entire audience. I think if we turn the camera around the other way, it would probably be half the audience. Well okay, well great, see? Gotcha. And of course, I think the reason you used testing in the past year of course is because we were trying to get Sarge released and it was a pretty good distribution and it just wasn't ready yet as far as security fixes and all that stuff which blocked it for a half a year. And so testing's really attractive for desktop users. I mean, I assume that there's a bunch of people who are scrambling right now to go install testings so they can go break their system by installing x.org because it actually went into unstable today. They're probably gonna, I mean in a week they'll be running to install testing. It's also really useful for custom Debian distributions who want to make a product that's based on Debian but that isn't as old as stable. For example, I worked for a year on the school Linux distribution which hoped to base its next release on testing or on Sarge. I believe that there's a couple of other distributions who have tried this, yes, question. I'll repeat the questions, I don't bother with the microphone. So a comment, this very building has machines in it that are running testing and that were apparently drive permit a year ago and I'm not gonna ask about how they're secured because I assume that the people here know what they're doing. But it actually illustrates a very good point which is that if you have a large organization and you wanna roll out Debian on a bunch of machines then you probably have the manpower to try to do some kind of basic security support on your own and you might end up using testing. And I know that that actually happened when I worked for VA Linux, I think. I actually forget what they ended up rolling out but they definitely had to do their own security support and all that kind of thing. So as I mentioned, I was employed by school Linux who wanted to base the product on testing and so they actually hired me and part of my job description was to try to figure out a way to make testing secure which is probably one of the most scary things I've ever had in a job description. And so I procrastinated it for a while and finally last fall I was urged by Petter who may or may not be here, there he is, hi Petter. He suggested that I actually get around to doing something about it and so I began trying to form a team of people to work on this and I think it's interesting that this team formed pretty much exactly five years after testing was introduced and for five years we had no security support and nobody dared to try to do it and I don't think that it's because I'm special or anything, excuse me. I think that it's more because it took that long before there was enough buildup deployments to testing and enough experience whether the people were actually comfortable with the idea of trying to do this. So we have right now a team which consists of, well there's probably about 10 or 12 people who are on the mailing list and five or six who commit on a regular basis. An interesting thing about this team is that even though we're working a secure Debian, probably two thirds of the team are not Debian developers. It's just an open team. Pretty much anybody who demonstrates their confidence can get in pretty easily and we have checks and balances in place that they can't screw anything up and so. Actually before I gave this talk I surveyed all the different team members to find out why they had specifically decided to work on it and I thought it was really interesting that of the responses that I got back, approximately half of them actually worked on either a derived distribution or they worked at a large institution like at a university or something and were already doing testing security and the other ones were also just as interesting I thought because besides the obvious reasons like making testing secure, some of them thought that it was actually fun to work on the testing security thing which I thought was amazing because personally I think it's a little drudgery to believe it. And another response that I thought was really interesting is that they appreciated it just because of the transparency of the team. Unlike the stable security team since we have members who are not Debian developers pretty much everything that we do is done in the public on mailing lists. We don't really have any privilege information. We're not on vendor sec and of course this has its downsides because there are security holes out there that we don't know about which other parts of Debian do know about right now but it definitely has its upsides in that we can get more people into the team. We don't have all the overhead of worrying about keeping this information confidential and that kind of thing. So when I came up with this team I actually started by having us go back and look at every single security advisory which had been published since the release of Woody. I think that was, I forget how many thousand advisories that was but it was a lot. Most of them didn't actually affect Debian but about a thousand or two did and most of those had already been fixed and unstable but probably five or 10 hadn't and some of them were security holes that were three or four years old at that point which pretty much just proved to me that we really needed some kind of a team who was adding an extra check to make sure that a maintainer doesn't just forget about a hole or just fall off the planet and the whole holes sit in Debian for ages. So in the process of going over all those holes we actually worked out some ways to keep track of all this information and to work together that I think have worked out really well as a team. It seems to be fairly scalable to fairly large number of people because we all just, we can basically work without communicating a lot with each other. We work in a subversion repository as far as tracking various holes. So I should actually speed up a little bit and talk about how we find holes. I don't know if the audience is generally familiar with the CVE database and the lists. If you're not you should become familiar with it because it's pretty much the way of the future as far as this goes. It's a list of all the security holes in basically all the software out there and the way it works it basically gives a single name or a number to every security hole. And from those we go through and look at all the holes. And actually I have Micah here who's a member of the team and I'd like him to just show us basically what we do on a day to day basis while I'm talking if that's possible. So let's see here. Here you go. So what he's gonna show us is we have a list of all the holes and the new ones automatically get added to this list as soon as MITRE gets around to releasing new CVE entries. And then we just get an email because we all watch the commit logs. Yes, Brandon? Micah Anderson? Yeah. Oh, I don't know it. So new holes get added to this list. Automatically we get the email and then we go in and each member of the team when they have free time we'll just claim a block of holes. And I think Micah's already brought it up and it's looking for the ones that he's claimed. Okay, so these are, you know, they're marked as to-do and he'll just go through them and figure out if they actually affect Debian, which on average they don't. On average they affect Windows or some piece of software that you've never heard of. And you're very glad that you haven't, generally. But it can be a little bit tricky sometimes to figure out if a piece of software in this list is in Debian. I mean, there's a lot of stuff in Debian and a lot of strange names. You have to try to map back into Debian package names and things like that. So it gets to be, take some experience, but it's fairly easy to learn. Exactly, he has no idea. So basically we work from this list and the list is then used to generate some web pages which can tell us what problems are actually holes in testing. What if we found a hole? We gotta use this xsoups on a Debian server. There you go. So we've probably just found a security hole right now. Yeah, so basically once Micah comes up with whether or not xsoups is actually a Debian package, he'll work out if this bug has actually already been fixed which it often has been and he'll figure out exactly what version number it was fixed in and then he's gonna add that into the file. And, yeah, yeah, there you go. So we actually have a bug report, well, yeah, yeah. So, and that's another thing we actually track is ITPs and make sure that we don't add things that have a known security hole to the distribution. Security box feed from before. Exactly, yep. So once we have all this data though, we can actually start using it in some fairly useful ways. Yeah, we can say, you know, I have a script which basically runs on one of the Debian servers and looks to see, you know, it looks just as simple version comparison to see if the holes are still out there and generates a page which the release team is fairly happy with which lists all the things in testing. Would you mind showing that page to them? Yeah, you have to go back. Yeah, so this is the page. It lists a lot of holes right now. If you look at the really deep red ones which you probably can't read on this display. Yeah, those are ones which are remote security holes, that kind of thing. Yeah, the ones in yellow or security holes that we haven't classified yet. We don't have enough information to know how bad the security hole is or we don't have anything about it. The ones that are in pale red are security holes which you probably wouldn't want to have in your system but they're not gonna be a remote route or something like that or they're not gonna affect too many cases. The ones which are in white are basically they're security holes in theory and they're more of them than anything else just because people don't really bother to fix them very often in my experience. And so these are all holes in testing and if you go down to the very bottom there, as you can see we have 55 unfixed holes in testing which are also holes in unstable, of course, because they all have bugs. And today we have 24 security holes which are fixed in unstable but aren't fixed in testing right now. And of course we have a certain number of items that we haven't checked yet and so on. So that sort of gets me to the point of how do we actually try to get, yes. Ah, that's a good question. The reason we have some advisories. Yes, I'm sorry. He asked why some of the advisories are CAN 2005-XXXX. Those are security holes which have been found by another means than the CVE list and haven't actually shown up on the list yet. One of the problems with the CVE list is it's updated about once or twice a week and in between that a lot of stuff goes on and we have enough people in the team now that we're covering pretty much every security mailing list out there that's useful. I mean we cover full disclosure, bug track, we read the WNBTS, we read DTSAs, we read Gen2 bug reports, we read Ubuntu bug reports, everything pretty much at this point. And so we do know about some holes which haven't shown up in the CVE database yet. That's great. So Matt Zimmerman just suggested to me that we can forward holes that don't have a CVE ID to him and he can get an ID and that's great because that's something that we've wanted to do but the stable security team doesn't really have the manpower or something to actually do that for us at this moment. I think you also said that you can get us to become an authority so we can actually have our own part of the namespace and assign IDs out of it. Is that what you said? Yeah, of course. Yeah, yeah. No, the way CVE works is there are multiple groups who can assign CVEs as I understand and I can't really go into details about that because I... Right. Yeah, exactly. The whole point is to not have more than one name for any given security hole. So yeah, we should definitely get together and talk about it. Thank you. So anyway, moving on to how we can try to get these security holes into testing a little bit faster and avoid the problems that I showed you at the beginning. One nice thing is that I am on the release team which means that I do have certain powers which I can use to try to force certain poles in, yeah. For example, if there's a security hole which is uploaded with, say, severity or urgency is low, then I can look at that and say this is bullshit. I'm just gonna upgrade it. It's gonna go in in two days assuming that it gets auto billed everywhere. I can try to not, if necessary, we can try to push packages out of testing. That doesn't really fix the problem for our users so it's not really worth doing, in my opinion. But the fact is that those issues that I listed at the beginning are design flaws of testing which make it impossible to secure it if you get right down to it. There's no way that you can say given this security hole which I have a patch for, I will get into the testing tomorrow. There's just no way. And so at some point, we've been hoping to get together a repository of our own and an auto builder network of our own so we can actually issue advisories, yes, I see. Why can't we use testing proposed updates? Okay, that's a good question. In theory, I think we could, assuming that it's auto built and assuming that we can then get the fixes from it into either testing or into something that's useful for our users. And that's definitely one thing that I'd like to do but it's been a little bit unclear to me as to whether it's gonna continue to work now that we've gotten Sarge out. And I'd really like to talk to somebody about that and see, it's being used. He says it's being used for security stuff. Okay, I haven't really noticed. Okay, I didn't catch the name of the advisory. That you said. Okay, let's see the issue there. I mean, people can upload to it but at the moment the test and security team doesn't really have any ability to deal with it. It sits on testing. I mean on security.debian.org which I don't even have an account on, you know. And if we have access to it then you run back into all the issues with a stable security team having uploads to this repository which are undisclosed security holes. I think that they upload them there first and they probably don't want random people who aren't even Debian developers to find out about them. So it's an issue. I mean, I would really like to use that. I think that would be the ideal thing but in the short term I'd be just as happy with a simple app repository somewhere which we put packages in and push them out and our users can put it in their app sources line and use it. So we forgot if oops is a Debian package with a security hole yet. Okay, great. So there's a few things which I would, if we can get back to the slide show here. Yeah. There's a couple of things that I would really like to just run through in the audience since there's so many Debian developers here ways that you can make our life a lot easier. And yeah. The most important thing and I misspelled it's not change logs, it's change logs but anyway, the most important thing that Debian developers can do is when you write a change log entry that closes a security hole list the can number in the change log. That just saves so much time and effort right there. If you don't have, if you have a security hole and you don't have an ID for it, then get one. And thanks to Matt, we have a way to get one and you can contact the testing security team also and we'll just do something until we get one. Very important, don't hide security fixes. Sometimes you may have a security hole and it hasn't been disclosed yet and you know about it and maybe in that case, you wanna make an upload to unstable and fix it and put in your change log entry, fixed man page or something. That's your call but it's just gonna make a lot of work for us down the road and we try to go back and figure out when did this actually get fixed and why isn't there anything about it in the change log and what is this fixed man page thing and where do we download a diff to see what he actually did and so on and it's just, I would personally think it would be better if you just waited until it was disclosed, uploaded to unstable or talked to the stable security team about it or something but it can be a real problem. I guess the next one's pretty obvious. It'd be great if everybody responded quickly to security holes. I think the important point is that you will be NMU'd if you don't. I have a list now. I know where the security holes are and whenever I have free time, I go through that list and make NMUs and there's zero day NMUs without any, you've got the bug in the BTS, I'm gonna NMU you. So, as far as I'm concerned, zero day NMU policy is in effect for testing for security holes until Steve or somebody tells me it isn't so. You may be waiting a while to hear that. Yeah. Of course the last one, the last way that you can really help is just to talk to us. If you see something on our page that isn't accurate or if you know about a hole that we don't know about which is publicly disclosed, please let us know about it. We're human too, we make mistakes all the time. We don't always catch the other person's mistakes in the team. So, the big one, put those CVIDs in the change log, put them in the bug reports, Matt. Yes, that is handy. Yeah, so Matt is pointing out that even though we have this new sort of list of security holes in testing, we do have the BTS and it does have the security tag in it and you should use that correctly and look over your package. If it has bugs which are security holes that aren't tagged as such, tag them and ping the security teams about it. I mean at some point, ideally we get rid of this temporary page which lists security holes in testing and we just use version tracking and the BTS to say okay, let's see which security, which bugs tagged security are still open in testing and then we can avoid some of this work that we have to do. Did you have a question? No, okay. So, the last thing I sort of wanted to talk about in this talk is how well we've done so far but it's a very hard thing to actually quantify in my opinion the big question which is how much more insecure is testing than Debian stable at the moment or the other way around. I've seen so many comparisons of the role of security of different Linux distributions and different operating systems and they're all just bunk in my opinion. So here's my try at it, which is also bunk. But, yeah, since the beginning of this year, every time that the security team, the stable security team has issued a Debian security announcement, I've gone and looked, is it fixed in testing yet? 50%, exactly 50% of them were not fixed in testing before the DSA came out. 37% or no, 33% were fixed in testing before the DSA, which is pretty good when you consider that we didn't know about some of these holes in advance and that the security team did. Then the 17% that didn't affect testing, stuff that was removed, stuff that, there's various things like that, not generally. This doesn't include packages that were already upgraded. Those are in the 33% number. And then I have two numbers which I have no idea what the correct value to put in them is how many of them, yeah, that's a good question. I have no idea what that one means. So, there's obviously a certain percentage of security holes that affected Debian in this year which didn't affect stable and did affect testing. And I don't really know what that is. Oh, I'm sorry, I know what this one, the affected stable with DSA, I believe that should be without DSA and the idea is that the fact of the matter is our stable security team doesn't fix every hole in stable. I'm sorry if this comes as a shock to you. But they look at, they prioritize, they sometimes miss holes. There's always a list of holes that they haven't fixed and it's basically always growing. Probably, yeah. Are they very obscure? Yeah, right. Like design flaws and stuff like that. And then there's also the ever-present bug in Mozilla which requires upgrading to the newest version of Mozilla to fix. And bug in Kernel which for some reason hasn't been fixed in six months, though, well. You know, huh? Oh no, of course not. I just don't run test, I just don't run stable on servers but I don't know if I should have said that on camera. I was asking if I'm bitter. Yes, that's true, my point is just as far as getting the numbers, yes, the question was shouldn't these be zero? Yeah, the last one. Yeah, sure. It should be zero in a sense. But my point is that there are security holes that didn't affect stable so it sort of outweighs the fact that yeah, we fixed 33% of all security holes that affected stable before the stable security team got around to it but the fact is that we had other holes that were affecting testing at the same time and so these numbers are bunk. And it would be great to come up with some better numbers, maybe actually look at the kernel security holes which are, you know, they tend to affect all versions of Debian pretty much. They tend to be pretty serious. They'd be a great thing to compare if somebody wanted to do it. Unfortunately, I didn't have the time to do it. He asked if I wanted to look at security or kernel security bugs, could I please involve the kernel security team? So Horms told me that if anyone here is working on kernel security bugs, please file bug reports properly, please contact the kernel security team. And we do do that in the testing security team because frankly, keeping track of kernel security holes is probably the hardest thing out there. It's a lot of work. They're very old defined often and so we pretty much just lean on the kernel security team for that. So, yeah. Scott asked if I file bugs on packages which the maintainer is clearly not aware of. Ah, well, yes. You'll see a few of which we don't, such as de-package. The issue that, yeah. There was a recent security hole in Zlib and it potentially affects de-package which embeds a static version of Zlib. This could potentially, I believe, cause de-package to crash de-package when it was queried or something. So we haven't filed a bug on that. It's just because we only found out this about two days ago and we haven't gotten around to it. I think we could do a little bit better at just filing the bug immediately. But, you know, yeah. That's definitely something we need to improve. I think there are a- And there wasn't and then we filed it with the bug number. I don't know if I need to repeat that but Micah said that, I mean, a lot of these bugs are actually bugs that we filed based on looking at the CV IDs and saying, well, the bug hasn't been fixed so we file it and that is generally our policy because otherwise it's just completely useless to have it in this list if there's no further information or if it, you know, if nobody else knows about it, it's just exercising futility. So, the next steps for us as a team are pretty much to try to get that repository that I spoke about up and running so that we can quickly push security fixes somewhere and to try to get the delay for security fixes down from whatever it currently is now which I frankly don't really know in general but in some cases it's been as much as 20 or 30 days for some holes down to, you know, three or four days would be nicer. And we'd really like to start actually issuing security advisories just like the stable security team does or even ideally in cooperation with the stable security team so that DSAs can say this is fixed in testing in this version. We're probably only gonna start out with supporting I-386, maybe a few other architectures and work up from there. And I think that's pretty much it. I guess if I have any questions from the audience I'll take them out and thank you. Does that say? Yeah, could you flip back to the slides? Sorry about that. Keep tearing Mike away from his work here. Okay, so this is our webpage. It does have a contact address. It's a listed alioth. Oh yes, there is a tool to missing. Well, I don't know latex, so sorry. Yeah, there's a tool before the JOEH, yeah. But yeah, this first page links to the second one so there's we, you know. The second one also links to the first one, but anyway. Any other questions? Yeah. Is it working? The scripts, the tools you have shown are generally for the whole Debian. And I would like to ask if there's any support for tracking bugs in a subset of packages which I have, for example, installed on my machines. That's a great question. A member of the team, do you remember who it was who was working on this? Don't forget to update the bug. I don't know what it is. No, it was already said. Okay, yeah, I don't remember either and I wouldn't be able to pronounce it if I could, but the basic idea is that we'll have something similar to App List Bugs which can actually go out and query our data and say these are holes which affect you now. Exactly, out of the packages you have installed here and if you're running testing or maybe even if you're running unstable since we also gather data for unstable, of course. These are the security holes that you're vulnerable to and since we have all this data in a machine-participal format, it should be fairly easy to do and it's already being worked on, I think, but if you'd like to work on that, that'd be fine. You can download all these files right off the web or right out of subversion, either way. Yeah. He said that there's an issue that on some machines he has untrusted local users on some he doesn't and that is the point. We don't actually, at the moment, differentiate between holes which are only remote holes and holes which only affect machines with untrusted local users. We started to move in that direction with the coloring here, but the coloring isn't particularly well-defined. It's just more of something, you really wanna work on the red ones first. So we could do that but it's gonna take more work on the part of the team as far as categorizing all these holes. So I think we'll probably see how it works out as far as adding these urges and maybe work up to more metadata that's more useful from there. So anything else? Anyone else? Question up there, Matt? So where I work, I'm often asked if Debian has a fix for a particular bug or if Debian is vulnerable to a particular bug and usually they'll give me the can number for that or some other number. And often I will go and I'll look at the change log and that kind of thing and sometimes it'll be in the change log but very often it's not and so I have to go in and actually look at the patches, that kind of thing. You guys seem to be doing a very good job of tracking down and figuring out the direct mapping for can numbers and that sort of thing too, the packages. Is there any way that you could somehow put that data, make that data, well I mean it's kind of available here but I would really like it to be in the package itself so that somehow I could look at the change log and know that for any given security hole that's opening in that package, whether or not the package is affected. Even if it just doesn't apply because it's not the right version or something like that, it'd be good to know that sort of information so I could very quickly determine that. Wow, that's cool. Yeah, you can also subscribe to the Commits List. Yeah, I wanna go to look at the package. I think the issue with putting it into the actual packages, often we don't find out about a security hole until it's been fixed by uploading a new upstream release of a package and so we can't really retroactively put it in but it is good to always put the can number in your change log because it does help but I really think that as far as your application goes it'd be better to just have a web front into our database and you can just query by can number then. And would it be bad to go back and edit the old change log entries? I think it'd be fine. I mean is that- Or just to add a new change log entry saying previous entry fix can blah, blah, blah. You know, it's perfectly fine to do that but it's sort of hard to enforce it on Debian. So the thing is that we have done in the past has been applied to the source if it's not listed in the change log. We might go look at the source and see oh, this has been fixed and then we just know. Michael said that sometimes we go look at the source to see if it's been fixed. I mean we basically go to as much length as we need to go to to figure out fairly certainly whether a hole's been fixed or not. So it can be a lot of work to figure that out. Yeah, I'm just trying to avoid duplicating that word because you guys are doing a great job of doing that. Yeah, and I mean we would also be happy if you are doing that work on any kind of a regular basis to give you commit access so you can go in and fix it if we get it wrong or added if we don't have it yet. So I think the pair's had his hand up for a while. Yeah, it's the one that put Joey to work on this task. I wanted to make a note in public that I'm really pleased with the results of my small job assignment to you. It's the result actually from a meeting in Spain we had, it's one and a half, two years ago where we decided that if you actually want people to use testing we need to get it secured and we decided in debut to get that done and try to get people to do it and Joey was apparently the right man for the job and it was fairly easy job description. Make a security testing team and make sure testing is secured. And while he hasn't quite succeeded on the last thing but we definitely know the state of it and how much work that's left to actually get it secure and it's a large leap forward compared to the previous state where everyone claimed that testing was really bad and secured and no one actually knew what kind of bugs were in there. So I would like to say thank you Joey for a great, great, great job making this happening. Thank you, Peter. I hope you're better than that. I'll give you a stand. You know, I also have to thank the rest of the team because it's not just me. I mean, I take a week off now and then it keeps happening and that's really good. And I think we have a question over here with Blar's. I'd like a list of the CVE IDs for a package linked on packages.qa.debian.org. That's a great idea. That's the perfect place for it. That's a great idea. And I think we had maybe one more question. We have another minute or two. I could, might as well just one, but whatever. I just wanted to say along the same lines is Peter's comment that when I first started talking about the idea of using package pools way back in 97 or so, AJ usually can quote the URL for that email message and I can't. One of the motivations for it in the beginning was not as a way to make releases easier per se but because I had the very definite impression that the vast majority of our net connected users wanted to run something that was fresher than stable and didn't want to put themselves at risk the way that an unstable of that time and an unstable since the release of Sarge can feel on certain days. And it's incredibly gratifying after what's now been what, eight years or something to finally see that part of the original package pools motivation and vision come to pass. So, dude. The testing security was this sexy, but okay. I find a way to promote everything. Okay, I guess we might have time for another question. Wow, great. Okay, I won't say AOL. Thank you too, of course. But my question would be, are there, from your point of view, are there any chances of integrating testing security with stable security and making use of the infrastructure which at least I cannot judge for the stable security and what exactly they use. I can see what you guys use and it seems to be working and maybe I think it makes no sense to have a duplicate infrastructure but maybe actually integrate this more and more. If you have a, I'm sure you've discussed this. Could you give us 10 seconds of... Yeah, I mean, I think it'd be great to integrate things both ways. I think that both teams have their strengths and when I settle to do things like this, I find it useful to do it from the outside and then work your way in. And I think that the fact that people now recognize that the test and security work is valuable is definitely gonna help convince people to make it more official and to integrate with the security team. But we do have the big divide here of disclosure versus non-disclosure and that's gonna just be an issue which is gonna cause problems as far as getting the two teams to work together well because the way we're set up, we really can't know about undisclosed issues. So it's definitely something that we need to work on going forward. That's all I can say. Okay, well, I think that I'm out of time now. Yeah, and how are we because we are taking the food picture now. Sorry, but thanks anyway for your quietness in the soup. For the nice talk.