 as people are trying to wrap their heads around this whole thing, I think it would make it simpler. Sure. Yeah, I agree. Okay. Cool. So we have an agenda and I think a very interesting topic that we definitely address right away is why are we out of credits in the AI? This seems a bad situation. Are all jobs failing right now? I think we have a few credits, like probably whatever the free tier is, but like I just looked at it in like seven in the last eight on master. All right. Well, what we should do is I've got an email going with our account manager. So after this meeting, I'm going to email and figure out like we should be on a credit card and that credit card should be paying for new credits. Right. It's possible that the card got declined or there's some other thing going on. So I will send an email and CC everyone right after this. Awesome. God damn it. Okay. So that's all for that. Cool. Dr. Chetan buffers? Yes. Yeah. I just wanted to see if anybody's been canarying the new buffer thing and if we can just pull the trigger and get rid of it. I know that I promised that I would and I didn't. So and I'm going to a conference tomorrow and then I'm traveling this week. So realistically it would be next week before I could play with that. I think some of the Pinterest folks had also agreed to do that. So I don't know if we want to follow up with them and ask if they had done that. So I'm Cynthia from Instructure and we've been using the new buffer implementation in production for a couple of weeks for about a week and a half now and it's been working fine so far. I haven't noticed anything too crazy. How much traffic are you putting through it if you don't mind me asking? Yeah. Right now our usage is fairly small. So we're I mean you know we're seeing peaks of about a thousand requests per second which is pretty small for us. So it's definitely not heavy usage. So I wouldn't take it as you know the only source. But it is it is more than zero. Yeah. I mean thanks. That's a that's a great vote of confidence. I would still love a couple more people to do some quick testing though I think what we could do right is like we're not going to delete the old implementation like we're still going to have the runtime flag. So I guess Josh do you want to just flip the default? Like I think that's probably fine. Sure. I can run up here to do that. Yeah I also think that that one might be worth an unboiling else too. Oh yeah yeah yeah like I would I would definitely put it in the release notes and send an email and all of that. But as long as people can revert it it seems fine to to flip it. Just verify that there are no outstanding fuzz issues with it. I think we didn't find any but we should just verify that. Is that something that you could just take a look at after the meeting? I'm just doing it as a space. Oh okay. Yeah on the on the bug Azra said that there haven't been any issues so far. Okay that's good. But that was 27 days ago. Oh that's what the fuzz is. Oh sorry I was hearing Azra. And Derrick Derrick Argueta said that. He's on the call. Yeah he's here. Oh Derrick. Are you here Derrick? Yeah we're gonna say we can try to canary the buffering patient in the next few days. That was also something on my plate which just kind of got pushed on the back burner. That'd be awesome. Thanks. Yeah it's something that I can possibly do in the next couple of days but just being realistic with my travel schedule probably not. I can definitely do it by next Monday but it seems fine to go ahead and just flip the defaults and if we find any issue we can always flip it back. Okay what else? Other stuff. I can't think of anything. Okay see that issue. So we are so we flipped the coverage over right so now we don't we don't use GCC for anything anymore right because it's too slow like is that is that what has happened at this point I think? The release builds have moved over then. Yeah yeah we've migrated coverage so yes. Okay that's fine. I mean if GCC is that slow I guess that's life but I guess my only concern there and this is not something that that we should action on it's just that I know that I've seen PRs from like people at Intel and other folks they keep fixing things that are breaking on like GCC9 and I don't I don't know how to reconcile this right because like we have we have limited resources so there's only so many builds that we can do it's like we're we're obviously never going to cover every compiler but right we didn't come cover GCC9 anyway. Yeah what I'm what I'm wondering if we should do and I know that you and I had talked about this at some point is and maybe we can just open an issue to track is to use GCC just to compile the server and don't run tests because the tests are what take forever and that would at least you know that would at least show up like obvious issues in production code and that shouldn't take that long so that might be something that we could consider. Yeah yeah yeah just open an issue to track that. All right I will make an or I'll put that in the meeting notes and we can figure that out. Hold on I'm making a note of that. Oh sorry it looks like you were typing also. Okay did you did you want to talk publicly yet about the runtime guard stuff Alyssa that that you had sent slack messages about or is it too too early? We can I think I think we're just in a state where we we think we're going to move towards encouraging runtime guarding visible like changes to the data plane. Yeah like adding headers or moving headers or whatever default to guarding it. I don't think we're at the point where we're going to put that in governance yet because we found a couple small issues and we just want to make it as easy as possible before we ask people to do it by default. Yep I think that's fine I think my only comment there is we're going to need to better define like during code review you know what is the data plane like what qualifies as a thing that we would want to run timeguard because I'm assuming that you know we don't need to run timeguard bug fixes but it's unclear right so like I think I think that we just need to have some guidance in terms of what meets the bar so that when people are doing code reviews like we can make sure that we're no I think we'll talk it up and I think this will be one of those launch and iterate things I mean we literally do guard a bunch of our bug fixes internally but like I think we've gone over the top right I don't want to encourage on where to go as far as we have right and that's start with something and then yeah and and I think that we've talked about this off and on over the last year or two and I have no conceptual objection I just I just want to make sure that we don't go too far over the top and then it becomes like a giant pain in the ass for not not that much benefit so I'm hoping that we can hit a happy medium and I think just having having some structure in place of how to think about whether we want to put the guard in or not like would probably be useful yeah and I think also maybe even hand waving about default tree versus default falls you know something with buffers you know we we want to default off and test carefully whereas you know like a something that we think is innocuous like removing headers which turned out to be disastrous yeah you know we could we could default through that people could roll it out if it caused problems people can flip it off and I also think that it's completely reasonable to like once we get the ability in the bot to require code reviews like for particular sections of the code I think it's reasonable you know to have a set of reviewers that must review either the tls code or the or like the codec code or something like that but even there I mean like we're just going to have to figure out like what code requires what and just to make sure that we don't block on any single person yeah totally so is that something that you feel like you would want to take a straw position on a proposal at at some point in terms of policy or like I've got one fix out that that the square folks found and I've got a fix that I don't love and I'm gonna send it out as a discussion point okay and then once that's in I'll I'll take a stab at governance okay okay great I think we'll start with like a encourages for a quarter and then again as we flush out more issues we'll make it stronger yeah I mean it's it's less even about encourages and requires I'm more just concerned about like what like what are we guarding and who who has to approve and I think as specific as we can be there and get that open for discussion probably the better will be okay cool and I and the fixes that you're working on just so I understand is the stuff that we had talked about so that we can use the static runtime guarding either in like the test code or the production code yeah so my my example cl which works great and solves a problem for envoy is essentially saying okay if you know for the integration tests if you're not a runtime if you're not a worker thread you can still check runtime yeah but then it occurred to me for people who have hybrid binaries including some people that I care about yeah that we'll still want it to work so people the PR that I pointed you out this morning that um empty headers right when we can catnate it just yeah yeah like that if if say google happens like that has not a way of course the headers it would have the same crashing problems I think like I want to land this as is to just fix a test code problem but that's fine you need to have a solution for people who have long worker threads that still want to access runtime so like yeah headers across the on the worker threads and then on worker threads right I don't know that looks like so again I'll send out the test CL yeah that's on there how to do that okay yeah let's let's maybe brainstorm offline like just thinking off the top of my head I'm wondering if since this is such an important case if we could have some kind of function where like a non-worker thread can register for runtime and then it would be hooked up to the same plumbing and then it would just work effectively uh that that seems like something that we could build I think and the question is do we want it to happen automatically which I think we do like I think basically if you ask for runtime snapshotting or not a worker thread you might we might auto-register you or something but again something like that right but like some some ability to hook yourself in even if that first time it requires locks or something else like I feel like we could we could make that work and that and that would be pretty slick because at that point I think like what I'm trying to get at here is that we have people that want to consume runtime in different ways like people that want to use flags or the disk thing or whatever and I think if we do this cleanly we can allow consumers to like toggle these flags in whatever way they want which seems optimal yeah okay great all right um I guess the only other announcement is we got the CFPs back for the conference they look amazing if anyone out there wants to be on the program committee and we have you haven't already raised your raised your desire please let us know we'll be starting to look through the proposals soon oh also um a shameless plug for sponsorships we're still looking for sponsors for the conference so there's opportunities for like things around sponsoring lunch or breakfast or stuff like that so if you're out there and you're interested in sponsoring that would be cool too I started to poke around to see who the people here are yeah um I'm I'm I'm hoping that we can find people out outside of Google to do that so I don't think it should be too too bad awesome and and it's it's approximately 5k that we would want for a sponsorship is that I you know that's more Chris's domain like I think there's different amounts I think that we could be flexible probably uh depending on depending on what people want to sponsor and probably what they want to get out of it um so I think just if you're out there and you're interested in potentially sponsoring um just circle back with us privately and we can we can figure out what's possible done I think so anyone have anything else okay see you next time yep