 Hi, folks, welcome to another stream. This one's entirely impromptu. I didn't announce this one in advance. This was more, I have been traveling or moving rather for a while now, and so I haven't been keeping up with my GitHub notifications. And so I need to do open source maintenance work. I have a bunch of open source projects that I want to make sure stay alive and stay healthy. And so I now have, I did some cleanup actually before this where I marked a bunch of notifications or I went through a bunch of notifications that were like read-only. Like I subscribed to a bunch of Rust issues that I'm just curious about progress on, for example. So I was at around like 500 unread notifications and now I'm down to 118. And some of these new ones on top are also just like subscriptions where I follow and a tracking issue, for example, they're not things that we actively need to do. But nonetheless, there's a bunch of just work that needs to happen in open source. And as a maintainer of some of these projects that means I have to go in and do stuff. And I was gonna do that this morning and figured out why don't I make it into a stream. This might be a different kind of stream. I've done one or two of these in the past but it's not gonna be a stream where I like carefully take you through a problem or an implementation of some kind. We're just gonna sort of dive into notifications and start working on projects that I know fairly well. But hopefully it'll still be interesting. If you feel like there's anything where you don't quite understand what I'm doing or more importantly why I'm doing it, then please ask and I'll try to give you some insight into this. But my hope is that this is maybe useful to people in the sense that we, you get to sort of see a glimpse on the other side of open source like not the contributing side but the receiving end and seeing the kind of work that goes into it. And hopefully it makes you interested in doing some of that yourself. But the other hope is that maybe it gives you a bit of a better understanding of where the maintainers come from in the projects that you already interact with. So it's a secret stream but it will hopefully be useful nonetheless. All right and because I recently changed my setup let me double check that I'm actually recording. I am great. All right, so the way that I go through notifications is I go from the back because those are the people who have waited the longest. There's an argument that you can do the ones that are newest because that way you keep the latency experience by the most number of people lowest in general. But I like going from the back. These three ones are work in progress. I'm gonna ignore those and we're really just gonna walk this list bottom up and see how far we get. I'm guessing I'll do this for about, let's say three-ish hours. We'll see how far we get. Great. All right. So what we'll do is we'll open a bunch of these and make our way through. Implement wheel support in Fontachini. Okay, so Fontachini is this project that implements the web drivers spec for browsers. So it lets you sort of like Selenium. It lets you remote control browsers. And this is probably just adding an implementation. It's pretty easy to add new features from the spec or just features we haven't implemented yet into the crate. Oh, let's get rid of these annotations. So they're adding, and just to show you what it looks like. So you have a bunch of different types of actions that you can pass to the browser. And currently you'll notice there's no mouse wheel action. And so what they're doing is they're adding a new type wheel action, containing wheel actions for the wheel device. And the kind of things I'm looking for here is like, does this fit the general structure of the crate? This is a useful addition. Does the documentation look about right? Wheel actions. But, oh right. So this is a wheel actions is a vector of wheel action. Yeah, so this is pretty common where you want to be able to do sequences of actions and send them as one thing. This is sort of a part of the spec essentially. The industry must be uniclated for this input source. That's fine. Pushes a new action. Links to the spec. Good. Pause is between. Offset of the scroll origin is pixels. All right, what about these field names? Oh, this is where is the cursor when you scroll? And this is how much do you scroll in X direction and how much do you scroll in Y direction? That feels fine. This is a private input that turns it into, we're using this other crate by Mozilla that encodes the like protocol messages in the spec. And so this turns the type that we have in our API into that sort of internal type as Millies. That's, well, I forget whether as Millies. Oh, that does include the seconds. Okay, great. And a scroll turns into the same with duration. So origin here we probably want to keep. So when you scroll, you can choose what the coordinates are relative to, whether they're relative to the viewport, like the entire view of the browser, or whether they're relative to absolute coordinates on a page. It feels unfortunate that this now doesn't allow you to specify any other origin. And input source for wheel actions, this is just so that you can chain multiple different types of actions. So you can say, move the cursor here, do a scroll, click, like as one big action sequence. Okay. So one of the questions that often come up in situations like this is like, do I want to block this PR on making this change? I think in reality, the change I want here is actually, I want this to be non-exhaustive so that we have the ability to add other events here or other fields to this, like the origin. So that's actually the main thing I wanna add here. So let's do this. Let's do, let's add non-exhaustive. And then I also like to, I like to link to the docs whenever I mention a feature like this because it's not always, like even though I know this feature very well, it's not clear that the contributor necessarily knows it. The attribute to this so that we can add additional fields or variants in the future without breaking that ability. Actually, I don't know if we can add non-exhaustive here because people do have to be able to construct a scroll. That's a good question, actually. Does it prevent, yeah, you also separately add it to variants. Non-exhaustive types cannot be constructed outside of the defining crate. I wanna see whether when pattern matching on a non-exhaustive enum matching on the variant does not contribute to its exhaustiveness of the arms, that doesn't help me. I forget what the rule is here. We might actually have to check this. So Kaganufu and we're gonna source lib RS and we're gonna have a non-exhaustive hub enum foo. And it's gonna have a variant bar that has field one string and then source main. And then I wanna see whether I can here construct a foo bar like is this legal or is that caught by non-exhaustive? That's legal. Okay, so, but if I add it here, then it can't, okay. So, so that we can add additional, so it's not true that we can add field, but we can add additional variants like maybe scroll from absolute to have a different origin without breaking back just compatibility. And then this is a knit. And I forget whether this is true. I guess I can test that out too. So if I do foo bar and I do cargo dock open. Yeah, cargo dock open. Yeah, so you'll notice the difference here between if I do this and then I have a different type, I guess Baz and I put an extra empty line between here. See how for Baz bar does not appear in the short version, but when you go into it, you see it. So basically single line breaks have no semantic meaning. They're basically ignored, they're treated as white space whereas double new lines means new paragraph. And so usually when you have documentation like over here, you want to have an extra extra new line between so that the first line is the summary of the documentation item. And the next line is just, will only be seen if you open it. Add an extra new line between these so that the first line is treated as a, as shown in the containing module type listing. Otherwise this will be treated as all being a single paragraph. So otherwise this looks good. The bump to web driver, I believe is fine. I think we no longer have any web driver types in our public API. And this was a very intentional piece of work that we did and it required a bunch of work to make this be the case. But if we hadn't made that change, then bumping web driver would actually require a version bump of Fantagini itself, of a major version bump, because if you have public types of this crate in your public API, then if I bump this, I'm effectively changing my own API as well. And therefore we can only do this bump safely within the major version because we know we've contained it. This is where really I would like to mark this as a private dependency, but cargo doesn't have those yet. Sorry for the very late view to this. It mostly looks good, just some minor comments. Main thing is leaving a space for us to potentially add a way to use a different scroll origin in the future. Request changes, submit review. If you're already blocking the PR non-exhaustive, why not have them allow customization of the pointer origin? I could do that, but I don't really wanna bog this down into discussing how we should handle origin. That's how you end up with PRs never landing or just taking enormously long to do so. Instead, non-exhaustive is I think a decent way to fix this PR and still leaving an open space for us to improve that in the future. Like this contributor has limited interest and resources as well. And if I make them go through a bunch of steps that they don't themselves need, chances are they're just gonna abandon the PR. Do not use the suggest changes button in GitHub. I sometimes do that where I highlight it and then make the actual change. In this case, I like to give the text. I vary whether I do it or not. Fantasini only supports Mozilla, right? No, it supports any browser that supports WebDriver. So it includes with Chrome, for example. All right, one notification down. Okay, so this is a suggestion for Erata for Rust for Rustations. There's a repo for the book. It doesn't have the contents of the book, but it has the website which includes Erata. And let's see, what are they proposing? You say the second workaround is to make each of you feels takeable. You can take an option by replacing with none. This approach is sort of for your mem take because you're talking about empty, clear separation between discussing option take and mem take. Yeah, how should we rephrase this? So here I'm gonna use suggest changes because I actually wanna rewrite it. So there's a section in the book that talks about cases where you have types that you need to, I think this is a context of types that you need to drop in your destructor. But more generally, like you have a mutable reference to something, but you actually wanna take it. You want an owned reference to it. And the way that you can do that is to make a field takeable. So one that can be replaced with a sort of empty value or default value by using mem take or mem swap for that matter or mem replace. And I think the current text isn't entirely clear about the distinction between using dot take on an option which leaves none in its place, which doesn't feel like a default value even though it is. And mem take which leaves, I mean, we can look at the mem take. So mem not tail take. Mem take takes a mutable reference to a T and it leaves in place a default value for T if T implements the default trade and then gives you back the T that was behind the mutable reference. And technically, these are the same. Like if you mem take a mutable reference to an option, it has the same semantics as option take. But I think that the current text is not entirely clear around that and that's sort of the complaint here. The that reads in part and take an option by replacing it with a none take a micro hash map empty values. I think that the real complaint here, right? Is that when I say empty here, what I really mean is implements default and for option that happens to be none which happens to actually be empty. Like arguably these are empty for many types like default for hash map is an empty hash map but it's not always empty. Like default for numbers is zero, for example. And so that's not an empty value. It's just a null value. Default really is the appropriate name. You can take an option by replacing it with none. For example, you can take a vector hash map, have sane default values like an empty map or a none. Because tedious if you're supposed to wrap nearly every field in an option and then modify every axis of those. Right, so that's not what I meant to do. This paragraph is technically correct but somewhat hard to follow because empty and default are not necessarily the same thing. Some types implement default but do not have an empty. But that value is not technically empty. Usize, for example. The sentence should instead read, have sane default values. So I think that's a better way to do this. In a, actually let's make it a little clearer. If you must wrap nearly every field in something like an option. Because it's not that you have to wrap them in an option, it's that you have to wrap them in something that allows them to be taken in something defaultable. I think something like an option is good enough here. So that's the suggestion I wanna make there. Taking so long on this, feeling about this proposed change comment. Great, moving on. IE mode of Microsoft Edge fixes 214. I think I remember this. The IE mode of the Edge browser. Right, so in Fantocini there is a way to, or in web driver in general, you can pass an additional set of options to the browser as part of connecting to it to configure additional aspects of how the browser operates. Like things like should it run without a browser window? For example, should it conform to the spec? Which is one of the options that Chrome has. And here, someone's trying to use it with the Microsoft Edge and they want to pass in this additional option to tell Internet Explorer to use Edge Chromium. And that doesn't work because no matching capability sets found. And that is because Fantocini currently always injects a couple of extra options. Specifically, there's a Google Chrome options where we tell it to follow the spec that we always inject into that list of capabilities. And that causes IE to crash because it goes, I don't know about that option. So the change here is basically to make it so that, previously we always inject this Google Chrome options W3C true. And this change is saying only if the browser name isn't Internet Explorer. And this is stupid, right? Like this is just a bug in Internet Explorer really because this colon thing here, I believe is a part of the spec saying these are extensions that should be ignored if you don't know what they mean. And I guess Internet Explorer just doesn't do that. So this is just saying, we'll still inject this always unless you're an Internet Explorer where we know that this breaks. Yes, it used to be that we only do this if the browser name that is passed in is Chrome. And I want to do it the other way around. It's that only if the browser name isn't Internet Explorer, which is the only one we know is problematic. Great. So this then actually looks fine now. I think I'm happy with this. They made this change. And I don't know why CI is being sad here, but I actually think I'm okay with this. This seems like an innocuous enough change. And then the question is whether I can kick this to rerun CI. Ooh, rerun CI. I don't really wanna, I'm just gonna merge this. Sometimes I think what happened here is I updated all of the CI scripts in this repo after when this person made the change. So the CI jobs that ran are all different ones and the new ones aren't run. But this change is straightforward enough that in fact, we can look at it right here. If I do W equals one to ignore white space, we see that the change truly is just this browser name change. Only if the browser name is not equal to Internet Explorer. Yeah, that's fine. Okay, so in this case, I'm just gonna overwrite this and merge anyway. And does this have lots of commits? It has three commits. It does really not need to have three commits. So let's squash it, squash and merge. Okay, let's make this a little more helpful. So we want this commit message to say, fixes to 14, i.e. Internet Explorer struggles with this. So don't pass it to i.e. Yeah, but there was no checks here for some reason. That's fine. All right, great. Thank you. And then I should do a release of Fantaccini, which I'll do in a second. I'll, doing a release of it, I'll do probably after the stream because it's not that interesting. And I wanna group multiple changes if there are any. Okay, next one, Inferno 2023, still fighting with Internet Explorer. That's very true. Are you okay with PRs not having tests? Usually I want tests in this case. I don't think we are even able to run Internet Explorer in our test suite. So it's not really a thing that we can do. Single stack detection could be wrong even if the event contains multiple colons. Okay, so this is an Inferno, which is our port of flame graph. And okay, so someone's running a Perf over a Go project where it looks like one of the functions has a colon in the name. Interesting. So in Inferno, what we do is we parse the output of Perf script. And we do that by essentially looking at the format and trying to tease out which things are function names, which things are program names. And sometimes that's kind of ambiguous. Like if you have this colon here and then try to separate this from single line stacks like this one, where you also have colon, colon, but these are not function names. So this is something like, it's like the executable name, the process, no, the time. I forget what this is. That is the process ID and that is the thread ID. And then this is the function, this is the top of the stack. So this is the function where the probe happened. And then this is the stack trace of that call. And this is the address of that function. So what's the difference here? This is a single stack case. So this is name of the program, time, that square thing is optional. Process ID, thread ID, single stack case. Let me look at these. So Perf script is not super ideal to parse. Some of them include a stack line if the stack has only as one frame. The first should not be handled as a stack or it's a ladder to both should. Trim until we counter a space or a colon, whichever comes first and then evaluate from there. I don't understand, that's gonna find this, oh, this colon, oh, by colons. What do you hear? Splitting by, splitting in three by colons. So that's this colon and that colon. And then this is looking at, looking for the first following colon and looking at whether it's empty after that. Yeah, and in this case, it's empty after this colon, in these cases it's not. And so in this case, that's problematic. No, you should be able to tell that this is that because something follows, because we're gonna split by this colon and then we're gonna look at everything that follows here. Poorly delimited formats. I think maybe the trick here is going to be to split by white space. You can't split on colon space because this line doesn't have a colon space, for example. So I think the thing to do here is to split on, yeah, the real problem is this split end. I think this has to be different. I think this has to be a split end two. So you split by this colon and then you split by, and then you are split by colon. No, because there's gonna be colons in the file name. This here is a file name and this, and I think that can also include a line number which has a colon in it, like it's file name, colon, line number. So I don't think we can R-split there either. I forget whether there can be fields in between here. There isn't a grammar for Perf script. That's one of the problems. Perf script just prints out. Like if I go to Perf help, I apparently don't have Perf. Let's make it so that I have Perf. I think it just prints them out one after the other which is entirely unhelpful, but I'm trying to see if there's a, yeah, I didn't think so. We could use regex here too, but one is that it actually slows things down. Perf scripts can be pretty enormous and regexes are gonna be a fair amount slower here, which is one of the reasons why we don't use regexes here in the first place. The regex also doesn't help you, right? You still need to know what to match on and if you know what to match on, you can write it without regex. The program name can also include a colon. That's true, but it can also contain a space which is all to say it's a terrible format. The question here is also what is more likely? Like for example, it's not uncommon to have colons and file names just be kind of disallowed like ffmpeg for example, if you have a colon in file names, it just like barfs on it and you have to give dot slash first or something, not that that helps us here. But let's ignore problems that already exist, like colons and file names will already not be supported. How can we fix this case? And I think the way you do it is you split n by two, yes, we're already skipping what comes first, right? We're already splitting, we're getting rid of everything up to here because we're with skip n, skip one, I mean. So we're gathering everything that's up to here. That's what's in event. And I guess the argument is that's not what should be an event. What should be an event is because the event can contain in colons. Also, I feel like it should trim here. Oh, we are split by space and then we take the last element. So that's how we get to this. Yeah, I wonder whether, okay, so what did this person propose who's looked into it? Without either trying to parse the rest of the line as a stack line best effort and allowing that to fail or trying to look at to see if the next line is a regular stack line or an event line. As I don't think you need to do that. I think that the way to go about this is actually an R split because I believe that the thing that comes after cycles here is like a sequence of flags. So I think you can just, when we do, when we split by the colons here, instead of splitting by colon in this way, you split one spike colon so you get everything after this colon and then you trim and you split white space and then you get the second field and then you R split by colon. So that gives you, N R split, R split N two by colon, which is R split once by colon. So that gives you, that splits by this colon. And so if the name contains a colon, then it won't make a difference. Okay, so I think that's the strategy. How about we change how this, how, where slightly, instead of split N three colon, something like pseudocode is easier, I think here. So we'll grab, let's just grab all of this stuff and see if we can't make it a little better. Yes, yes. Okay, so we would split once by colon dot map. All right, I guess this would be sure by colons. So this would be actually no longer by colon. So it'd be after colon, after first. And then event is gonna be after first colon dot trim dot split white space dot nth and we're gonna have the nth be, so looking at that line we had over here, here are a couple of other examples, that's fine. After this first colon, oh, this number is optional. That's good. That's great. Oh, no, this is fine, actually. What we'll do is, we'll do find field dot ends with colon. And then, so that's gonna be, so we're gonna split by the first colon down here. So splitting by the first colon and then we're splitting, we're trimming and then we're splitting by white space. So that's gonna give us this and then this for example, and we're gonna keep taking from that split until we get something that ends with a colon, which is gonna be this one, this one. And even if this number isn't here, it's still gonna find the first one that ends with a colon. And then we're gonna say, so that's gonna be the full event and then the event is going to be full event dot map. And we're gonna map that to, we're gonna map that to s.rsplit once on colon. Sorry, I mean trim end matches colon. We know it ends with a colon, so trim end matches colon is just gonna get rid of that one. Technically we could do strip suffix and we can expect that one because we know it known to end in colon. And then we are split once to get the next colon back. Right, so if we get rid of this one then we look for this one. And if there isn't one, then we know we have the event. Otherwise, we take what's left. So we do rsplit once dot map and rsplit once, I believe still returns these in the order you would expect. Rsplit once, that is, it returns a tuple where the first thing is what's before the thing you found. Yeah, you could imagine rsplit returning them in reverse order, but it does not. So we do rsplit once, we do map of event and flags, and they were only gonna return the events. And if there isn't ones, unwrap or else s. So if there is no other colon in the event then we should return the entire event. So I guess we can make this a little bit more helpful too. So we can say this perf script does not produce easy to parse lines. In particular, lots of fields can contain characters we might consider the limiters. We know that every line has at least one colon between the process thread info and the event info. So grab the process thread info usually doesn't have colons in it. Technically the program name can though. Can though we ignore that possibility for now to do. So here we extract the event info. The event itself has many space separated fields. We want the event name, I guess is really the, or is it event type? I think it's event type, cycles or function name, which is always, which always ends in a colon. So find the first field for which that's the case. And then the event type is can itself contain colons. But it also has an optional, in fact this might be this might break us. And this is maybe what the person was pointing out. It also has an optional, in fact, what even does that mean? If we go to the colon, where are the fields? If we go back to the example we had, right like the colon U here, it's basically impossible to separate out whether cycles colon U means cycles with the U flag or whether it means there's a function named cycles colon U, which makes me wonder what does U means? Like I wanna know the cycles field, where is that? Synthesize cycles, no, that's very unhelpful. Perf script event name, maybe. It's not gonna be helpful, is it? What is the meaning of perf script output? Let's see if Stack Overflow has some answers. Pids you in time, yes, but what are the other ones? Yeah, event is the one I'm after. I'm pretty sure, this might be actually somewhere else where the events, but what does the colon U mean? Yeah, so see this is only at the user level, user and kernel. Aha, here, this list, they're modifiers. All modifiers can be combined at will. But clearly it's also not just that because there's also colon P. And in fact, you see this in the example we have here where one of them has like, where was it here? Colon U PPP, what does that mean? Is there one that has P colon PPP, PPP, UP? Unhelpful, but it seems like these modifiers, like there's only a small number of them. I just wish there was a thing that told me which modifiers there are. Because then what we could do is, then what we could use is actually, we could filter out only those modifiers. Most common modifiers, okay, so Perf event modifiers. Event modifiers, the user restrict which events accounted U for U space K for kernel, H for hypervisor, G for guest counting, H for host counting, the P modifier can be used to specify how precise it should be and can be specified multiple times. Okay, so what that means is an optional modifier list at the end. That modifier list can only consist of the characters of U, K, H, U, G, H, P, where only P may appear more than once. We want to strip the modifiers at the end if they're there and keep any other colons intact. Is it possible that the example output you have doesn't comply with the spec? That is possible and we see this some of the time where for some formats, they just randomly get other things, but it doesn't seem like that's the case. Okay, so in that case, we want to take that full event string, event with mods, event with mods is this. And in fact here, we can do that map up here now that I think about it. Just to keep it closer to where it actually happens. So we want to do if, so let's do match SR split once. If that's none, then we return S. If that's some and there's a pre and post, if post.something, then we only use pre and otherwise, then we return the entire S. I think is what we want. So the post.something here is gonna be if it only consists of those modifiers. Seeing PSDW and E on different sites with man pages. Ooh, interesting. Where are you seeing these? Let's see if Google can be more helpful. Oh, there are more modifiers, aren't there? What happens if I do perf help list? What do I get? Oh yeah, I get more ones. Okay, C perf help list. So it's U, U, K, H, I, G, H, P, P, S, D, W, E. All right, so then what we're gonna do is if event is likely event modifier, post, then only pre. And then we're gonna create this function right here. Now there are a couple of ways to write this one. I think the way that I wanna do it is actually, I really want this to be not a thing we have to do on every line that makes me kinda sad. Like there are a couple of ways we can do this. We could collect all the characters into a hash map to get a count for each one. We could sort the characters of the string and that way we can just check them in order. But all of these feel like, I don't really wanna create a hash map each time because this is in a pretty critical loop of parsing. Walking the string is tempting, but we still need to keep some state for whether we see in a given character. So I think actually maybe sorting the characters of the string is the way to go. The problem, of course, is that this string, oh, actually we know that this string is supposed to be ASCII. So we can do if s.isASCII. I think there's a stir as ASCII. Yeah. Oh, it's a nightly feature, of course it is. Ah, there is an ASCII. Then return false. Otherwise, s.isASCII.asBytes. And I think this actually means that I need a mutable reference to it. Which is fine, it's not the end of the world to have to into back. The other thing we can do is if s.len is greater than one, two, three, four, five, six, seven, eight, nine, 10, 11. Did I do that right? Yeah, 12 minus one for P is 11. 11 plus how many times is P allowed to appear? It's allowed to appear three times. So if it's more than this, all modifiers plus three P's longer than all modifiers plus three P's. Otherwise, U size a bit set. We could do this as a bit set, you're right. It's a little stupid, but you're right. We could totally, all right, all right. Let's do this, let's be fancy, because of course. No, I think it's 11 plus three, right? This is, oops, this is 12 characters. So that means there's 11 without the P. So it's 11 plus three, that is allowed, yeah. So bit set is zero, and then we're gonna have to do, and then we're gonna have to do, we could do a bit set here, but let's do mod, I guess, well, this makes me sad, but this is one, zero, U, K, H, I, G, H, P, P, S, D. W and E, P isn't gonna be there. One, two, three, four, five, six, seven, eight, seven. Ah, seven, nine, and 10. And then we're gonna do four C and S, match C. So it's gonna be, if we see a U, then if bit set and U is equal to zero, then I have to be careful here too, because these are, I think I'm actually gonna call all of these mod so that they don't accidentally overwrite with single letter variable names. Bytes is probably faster, you're right, we can do bytes, that's fine. We've already checked that it's ASCII. So if that is zero, then bit set or equals mod U, and then it's this. This is also a place where a macro might be nice. Actually, here's what I'm gonna do. Watch this, record, K, H, I, G, H, P. Ah, I didn't put enough of them, damn it. All right, let's do this. Q, K, H, I, G, H, P, S, D, W, E. And then I can go over here. I can do the same thing here, like so. Love in macros, like so. And if it's a P, then I guess we also need a let mute P's is zero. Technically, we use the same bit set, but I'm gonna not do that. If P's is less than three, then P's plus equals one. Then P's plus equals one, and anything else, return false. And if we get all the way down here, then true, right? This is definitely something that we'll wanna test for. R split one's on colon, we've already trimmed away the trailing colon, and then the other thing we're gonna need is this bit. So the way that we detect whether something is a single stack line is whether there is something that follows this colon. So up here, when we're splitting on whitespace, we want everything after the first field that ends with a colon. And so the way we'll do that is, we'll let this be an iterator. We'll do event with mods is iterator find, and then we'll do rest is iterator.... There's a way, I think, when you do, if you have a split whitespace. So when you call split whitespace, you get back one of these, and it has a remainder. Ah, it's a nightly feature. I want this feature. Ah, would be nice to use, but instead we then have to do... We're gonna have to walk this one. That's real sad. Do you need the bit set? Do we need the bit set? Because if one character appears more than once, then it's also not a valid modifier set. Instead of all the ifs, wouldn't it be faster to set unconditionally and do an old bit set not equal to new bit set? Oh, instead of these? Probably, I don't think the performance here matters that much. Like, I don't think we wanna allocate necessarily in this loop if we can avoid it, but I don't think we also need to like, figure out the bit fiddling perfectly to avoid conditionals. Like branches in that loop, I think are okay. Well, this is gonna be really annoying, isn't it? We're actually going to have to, without remainder, this gets really annoying because what we really wanna get, right, is everything after this colon, wherever we first find this field. I really want iterator remainder here. I think what we'll do is, we'll do something kind of stupid. We'll say that if we'll do s.... So the full string.position, what's the... There is a way, I'm pretty sure, on string to find... I'm lying, it's on the other string, it's on this. I believe there's a way to search for a string, to search for the position of a string. Yeah, I want find, s.find. So this find is a string find, this find is an iterator find, and they're different because of course they are. So I wanna find event with mods, and I know that that's found because we found it above. And I wanna add event with mods.len. So that's the position, so rest is going to be s, rest.dot. And we're gonna trim that because it's gonna start with a white space. And now we can do some event with mods and rest. Great, so this was gonna find the first field that ends with a colon, and then it's going to find that field again in the string, find it's absolute position, find the end of it by adding the length of the string, and then strings splice everything after that point in the string into rest. And so that way this is going to produce both what we think the actual event is and the remainder of the string after that field. And so now down here, this is gonna be this and rest. And I suppose actually we can do, let event and rest is, actually we could even untangle that up here. No, I don't wanna do that. I wanna preserve it through here. And I suppose what we wanna look for here is if the rest is empty. If event.1 is non-empty, then we have post event. What if the event name appears as a substring of another field? It can, but this comes back to what the actual contents of these lines are. So in practice, that's not really going to happen because the order of the fields in the output here is actually, oh man, is actually the field before that is generally just a number. It's not a string. So it's really unlikely that's gonna be the case. Unlikely to have this find is unlikely to have false positive since the event name is usually only proceeded by an integer field, numeric field. I don't know if it's actually an integer. Make sure only valid modifiers appear and only in allowed quantities. And now what we'll go back is, after some digging, how about something like this? And I don't wanna just make this change myself because there are people who care about this problem and might be able to like fit over this, add tests, submit an actual PR. So this is more outlining a solution for them. How about something like this? How about something like this? To replace the first part of on event line inside, immediately inside the first if let some. So this way, hopefully they can continue to follow along preceded, not proceeded. Now, did I really spell proceeded? My scroll wheel is not working right today. Where's my preceded? So hopefully now they can continue to push forward on this while I go do other things like clicking done and moving on to the next issue. Let's close all these tabs. So maybe now you see why very often doing open source work ends up moving much more slowly than you'd like because each one of these require a bunch of attention. Make it more obvious to a new contributor that flame graph gets sub module needs to be initialized before running tests. So this is also in Inferno. In Inferno, we also run all of the flame graph tests and we do that by having the flame graph project be a get sub module. But if you forget to initialize those sub modules, a bunch of the tests fail because that sub module doesn't exist. Yeah, they get this error. No search file or directory, which is annoying. It should get more obvious errors. So I'm guessing they're gonna change the tests so that this is a great addition and improve improvement to the, sorry for the delay. Approve, what is it? Oh, it's coverage, that's fine. There are, this can be squashed, that's fine. Complain loudly if not, beautiful. Merge, done. All right, next. Okay, this is for EVMAP. This is a data structure. We did a video on this one ages ago. It's like a concurrency data structure that's specifically optimized for if you have many concurrent readers. I was about to start using this package because I have a single writer, a multi-reader setup where the write operation cannot wait for readers. So a traditional RV lock window or mute text can be annoying. The description says the event has lock-free writes but I don't see how that would work, both flush and refresh will block the writer until the readers are done. I would want a situation where writers can freely add operations without ever having to wait. And as soon as there are no more readers, the reader side is updated. But on the reader's time, not on the writer's time, it seems like lock-free writes, you're intending to say writes don't prevent reads. It's as ambiguous as I took it to mean. Yeah, it is true that EVMAP isn't really lock-free writes. Well, it depends because it is lock-free writes. It's just they're not visible until you call flush or refresh and flush and refresh are not wait-free. But point well taken. So the tricky part here is so the very basic premise of EVMAP is that you keep two maps, one that writers write to and one that readers read from. And when a writer wants to expose the writes that they've done, there's a single pointer that all the readers go through and so you swap that pointer to the other map. And so now all the readers start seeing your updated map but some readers might still be working with the pointer they previously read. So the writer now needs to wait for all the readers to shift over to this map before the writer now has exclusive access to this map and can start updating that one instead. And so it's that that wait for the readers to move is the reason why flush and refresh block. It's possible to add a mechanism here where you can like try refresh. So if all the readers have moved, then it returns immediately. Otherwise it returns saying the readers haven't all moved yet. So that's basically what they're proposing here with try refresh or try flush. The challenge with having the refresh sort of happen in the background, which is really what they're asking for here is that it fundamentally is a blocking operation, right? Like you're actually waiting for all the readers to move along and something has to block. We could have a separate thread that runs but the question is what does the writer do in the meantime? Like if the writer wants to do a refresh, what do they do if they're not blocking? Because when you do a refresh, the idea is that once refresh returns, you're allowed to do writes to the map. But if refresh doesn't block and some other thread is like going to do the refresh in the background, if you start doing writes, where did those writes go? There's nowhere for them to go. Technically EV map has a sort of operational log where those writes could go and then you just need to remember that you need to apply it to both maps but it gets really convoluted because now what if you call refresh again and the previous refresh still hasn't completed? I don't know if you want to get into this game. Could it be done with three maps? I think you just have the same problem because what if you refresh while there are readers in maps one, two and three then now you need a fourth map and I think that's probably not great. Yeah. Okay, so you're technically correct which is the best kind of correct. Writes to the map are weight free. One could argue that writes to the map are weight free because well, they are but the pure write operation is but once you take into account having to also expose them to readers then they're not weight free anymore. I think updating the description is a good idea. Just submit a pure. As for the larger feature this one is actually quite tricky. Adding a try refresh and try flush makes a lot of sense to me. Happy to take a look at PR if you have a chance. As for the larger feature of having of essentially scheduling a refresh to happen whenever it becomes possible. This one's actually quite tricky. The big question is what to do with writes with modifications to the map while such a tween when refresh returns but still hasn't but the readers still haven't actually moved on. And when the readers have actually moved on and one of the maps is available for writes you could stick them in the Oplog but handling the logic of that Oplog then becomes significantly more complicated especially once you take into account that another refresh could come could happen while readers are still now in both maps. Regarding the code to identify perf modifiers so the modifiers always appear in the same order. I don't know which is why I think it's reasonable to do what we did is just parse it with a bit set. Okay, done. Branchless index update and find GT. Okay, so this is a completely different crate. Arguably I should do this per crate. It might save me some stack space but I still like the idea of treating them in sort of queue order. Ward search is a crate that I wrote ages ago that is basically an implementation of a way to do, here I'll pull up the description. It's easier so I don't get it wrong. Approximate lookups and ordered collections. So given a set A of N values and a query value of X, it finds the smallest value in A that is greater than or equal to X. And so it's a little bit weird. It's basically binary search. So the idea is that you have a set of values and it's sorted and you're trying to find the smallest value that is greater than N, the binary search over a sorted vector. And there's a research paper that basically outlines what is supposed to be a very fast implementation of that algorithm of binary search using sort of vectorized operations and branchless operations and stuff. So basically if you have a modern processor how should you solve this problem? And in theory we could experiment with this inside of the binary search implementation in the standard library, but in practice it feels nice to do it in a separate crate and then improve it over time. And then maybe if it gets good enough we could incorporate it back into Stude. And unfortunately when I implemented this I found that it was slower than binary search. So if you look at the performance results here generally it's actually slower than what you would get with a vector binary search. And so I sort of abandoned in the middle. But someone else did some experiments with this where they basically wrote their own implementation and then found mine and then we started having a chat. And they discovered that there were some things that could actually be implemented that weren't implemented quite right in my implementation of this that makes a huge difference. So it turns out that my implementation of find greater than or equal which is sort of the core implementation of the library isn't fully branchless. There's a branch prediction is the main bottleneck which makes sense. The following expression is not optimized by the compiler. Oh, that's interesting. So one of the things that this implementation relies on or that the paper relies on is that on x86 you have these things called conditional moves and conditional moves are they're not branches so rewinding a little bit. One of the things that makes programs slow that there are many things but one of them is branch prediction. So if you have a branch like you've got an if then the CPU when it runs ideally wants to continue to run. It doesn't like if they're too possible if you don't have branches if you just have a linear line of operations the CPU can knows exactly what's gonna happen next so we can just keep executing. And that means that it can execute things out of order it can execute multiple things in parallel it can execute way more efficiently. If you have a branch the CPU has a problem because it doesn't know whether it's gonna run this code or that code. And usually there are two ways to deal with this. One is the CPU waits until it has all the information it needs to determine which branch to take. The other is essentially speculative execution. You have something in the CPU called the branch predictor which tries to predict which branch is going to be taken. And the CPU uses that to predict the branch and just start executing that branch. And if it turns out that the prediction was wrong you get a branch misprediction in which case you have to unwind the work you did and then continue down the other branch instead. And mispredictions are very expensive because you wasted cycles on doing something you didn't need to do and on undoing it later. So you really wanna avoid branch mispredictions but it's even better if you can get rid of the branches altogether. And these conditional move operations are not branches. They're essentially a way built into the CPU of saying evaluate whether this thing is true or not and depending on whether it was true or false take this value or that value. And so it's not really a branch it's something that the CPU can execute as a single instruction. And it doesn't have, there's no branch in the logic flow it's just a value that's conditional on some computation. And in this case you often write them as sort of a simple if followed by two different values. And then you rely on the CPU or the compiler really turning this into a conditional move operation which is a single assembly instruction rather than turning it into a branch which is indicated by a jump like the JAE and the JB here. And what this person is pointing out is that this code which should turn into a conditional move actually turns into a jump. And indeed usually the compiler is able to optimize these but for whatever reason it doesn't in this case solution is quite simple. So what we're doing here is we're trying to figure out whether to add one or two and in this case they're saying just do two plus one the two plus I plus one plus U size from X is greater than value. I see because X is greater than values that's the inverse of this condition is a Boolean. And when you do U size from Boolean you get zero or one. So I mean we can test this just to make sure. So if we do here U size from false we get zero and U size from true we get one. So this is a way to utilize U size from Booleans to basically get zero or one which is what we want to add here. And so now you see here the original versus the branchless. You see the branchless takes way less time here overall and in fact you can see the diff here in percentages you see the speed up is pretty universal. This one is slower which is interesting. I wonder why three benchmarks become worse could be unaligned memory access. We're now L one memory bound which is a good place to be in but generally the speed system things up a lot. And now half of the benchmarks are faster than sorted vector. So the binary search in the standard library, beautiful. So let's look at the diff. The diff is indeed exactly right. It's a great catch. Thank you to your other good write up approve. Excellent. Do we want to squash this? This is already nice. This is already just one commit beautiful. Merge done. A new remote to child own struct to avoid the self ref issue. Okay, so this is in the open SSH crate. The open SSH crate is basically rust bindings to open SSH. It's not quite bindings. It's really you spin up an open SSH Muxing server which is basically you let open SSH handle all the crypto like it sets up all the connections. This is for clients not servers. It sets up the connection and everything and then there's a protocol that you can use to interact with the Muxing server. So you can send it things like execute this remote command or read this from standard in or send this to standard in rather or read this from standard out from the remote process. And so that way this crate doesn't have to do any crypto. It doesn't have to do any authentication authorization or anything like that. It just needs to start and control the Muxing server. Remote child don't, okay. So the setup here is, let me pull up the open SSH. I forget if I have an alpha, I do not. Okay, so when you spin up a remote process like you want to execute something on the other end of a connection what you get back is a remote child which is similar to in Rust when you do when you spawn a process you get back a standard process child. Remote child is the equivalent of that type which refers then to the currently executing process on the remote server. If it is possible for SFTP to hold a session of spawn command. Right, okay. So the challenge here is that the remote child only lives for as long as the session lives. So if you have a connection to a remote host and you have a remote child running on that host if the connection goes down or if you close the connection rather the child should also not be usable anymore. And we enforce that using the type system in Rust. So we say that the remote child mutably borrows or immutably borrows the session. You can see this by the fact that the remote child has a lifetime associated with it. And you can call dot session on it to get back the session that the child was spawned from. Now the problem with this is it's actually it's really annoying to pass around these session things or rather anything with a lifetime really because now you need to like you can't give them to a different thread. You can't, it just, it gets annoying once you have this remote child that tied to a reference to something else because you can't move that something else because then the reference becomes invalid and the bar checker yells at you and you can't move the remote child to a place where it might outlive the session. And so I think the proposal here is that we have a type remote child owned which owns its session. And the reason that's okay is because if you look at session and you look at something like command they all take a shared reference to self, not a mutable reference to self. And in reality, if we go and look at the source, right, it's a session impulse. Let's look at the actual impulse. Process the one I want. That's not quite what I want either. I might have to look at the actual code. So we have two different implementations of how to talk to SSH. One is through the SSH command line tool and the other is through native implementation of the SSH protocol, the Muxing protocol. Realistically, we're probably gonna get rid of the process-based one at some point once the native Mux one is stable enough. But if you look at the process implementation and look at session, session is really just two paths because the Muxing client is just running in the background. Like it's not, there's nothing for us to really own. There's nothing in the session object except the paths to the sort of control for the Mux. And so this one is sort of trivially cloneable. There's a little bit of question about how we deal with cleanup, but you could imagine this being an arc session pretty easily. And I think the same is true for the native Mux implementation, right? If you look at this, it is also just a path to the control and the temporary directory that holds some essentially the state of that Muxing client. And so these sessions in theory are cloneable. I don't think we want them to be cloneable because then cleanup becomes annoying, but they're certainly easy to put in an arc, right? Because they all take shared references anyway. And so the proposal here, I think, basically lets, instead of keep passing sessions everywhere, let's pass arc sessions, and then we can still use references to have a non-cloning way to spawn childs and stuff, but you can imagine then that you could turn a remote child into an owned remote child, one that doesn't have a lifetime, by just having all the same fields and really just cloning the arc of the session so that that remote child can continue living. And the way that'll work, of course, is that the, even if the original session is dropped, like the original session variable is dropped, the remote child owned has its own clone of that session, which means the session won't be dropped, we're not gonna terminate the Muxer, and therefore the remote child gets to keep living. Unfortunately, I think this makes a lot of sense. The way to go about it is probably to start carrying around arc session instead of just S session, and then have a variant of remote child that clones that arc session instead of keeping a, should be a pretty trivial change, I think, beyond having to duplicate a bunch of the remote child code. So of course the downside here is that if we want a remote child and a remote child owned, now we have two types that have to, they have to have all the same methods, they have to have all the same contents and logic. And so now we kind of need to be like either use some kind of macro to avoid duplicating all that code or just duplicating all of the code and maybe deprecating remote child. If you can think of a clever way to, in which we can avoid duplicating much of the code that would be nice but not a requirement. Do you think we should also deprecate remote child with the plan to make remote child owned, the only remote child in the future? My main concern would be that they'll end up unexpectedly leaving sessions open because they're still storing a remote child somewhere when in reality they might want the borrow checker to tell them that that is the case. Is this relevant for modern CPUs? Oh, ord search? Absolutely, this matters. Especially because binary search is such a low level operation and it's often used in hot loops for anything that uses binary search, for example. Like you end up calling this over and over. And as a result, even nanosecond optimizations to this ends up having pretty large ramifications. And so like as you saw, like you can see a speed up of like three X compared to the binary search and vector and that matters. Where does the term MUX comes from? I've heard in T-MUX for multiplexer. Yeah, it's for multiplexer and it's used for, I don't know what the word multiplexer, I don't know how multiplexer turned into MUX but that is the general translation and it multiplexes multiple streams, usually input and output streams. Can I be evil and suggest that you open this stage to create to also work on Windows? The reason why it doesn't work on Windows at the moment is because open SSH doesn't have a MUX client, like a MUX daemon or rather the MUX daemon feature of open SSH does not exist on Windows, at least didn't until recently. Maybe that's changed in the case we would totally do it. Okay, done. Okay, we made it through the first couple of issues that I opened. We were at, what, 118 and we're now at 110. So, no, actually, I think there are more now because some of them are unread. All right, let's go back to the last page and now you understand how this works. Okay, so this one, I guess I forgot to mark it done. Okay. And now we've been going for what, two and a half hours or something? This is what open source maintenance is like. Oh, same person. Okay, this is back in OrdSearch. PRC needs two changes, I believe, and my b-track's more robust. The suggestions are debatable, great. I made no-doop methods as inline never. At the moment, they are not inline by the compiler, but this can change. Doop, I need to bring this back into, what is this? This is, I need to page this back in. I haven't looked at OrdSearch in a while. So for the benchmarking, I think doop is for duplication. Like it's whether there are duplicates or not. Which, why are they saying that doop should never be inlined? I don't think it matters whether doop and no-doop are inlined because all they do is test setup and then bench construction, which is down here. Yeah, like doop isn't used down here. Is it make? Like what do we end up calling here? We end up calling make underscore. The only thing that changes is the value of mapper. So the second to last argument of bench search. And second to last argument of bench construction, mapper. Oh, mapper is called in there. Oh, I see. So the thing they're marking as it's these that should be marked as, but these are already marked as inline. I don't understand this argument. These, oh, to make sure we have the same machine code for the setup. I don't know if that makes a difference. Oh, for the different sizes. I see, so you could end up with a benchmarking suite. You could end up with a compiler building, optimizing a doop test differently depending on the sizes of the payload. But I don't know if that's actually a problem because in reality, when you run the code, that's going to be the case anyway. Yeah, the compiler will generate different machine code for the different payload sizes. Although this can show better benchmarking results, this strategy is not representative of real applications because it's almost never the case. So we know the size of the payload at compile time. Size comes from cache.size. Oh, okay, I see where the argument comes from. Okay, so we pass in a cache here and the cache dictates basically the number of elements in the array that we're going to search. So the idea here is that we want to benchmark things that fit in the L1 cache of the CPU, the L2 cache of the CPU and the L3 cache of the CPU. If this is ever inlined, what you can end up with this, the L1 benchmark and the L2 benchmark, they're going to be, they're going to generate different implementations based on the known size of the cache, which is not normally something you know. So I think I agree with this. I removed the is sum from the following code. In this case, findGT is actually inlined by the compiler. In most cases, there's no significant changes. There is one where there's a performance drop. Okay, so this is, there's a bit of code where in the benchmark down, here, when we search for something, we call dot is sum. And I think the reason why I called is sum here was I wanted to make sure that the value inside of the sum was actually being looked at. But in practice, that's not really true. What ends up happening is that the compiler smartly realizes, well, you're just taking the whether it's sum or none, which means that the actual value inside of the sum doesn't matter, which means that I can avoid computing it. And so it ends up skipping these last part of the computation, because it does still need to compute sum or none, but it doesn't need to compute this value. So it only needs, well, it should still need J because it still needs to decide none or sum. But I think I agree that it's better to skip these sum here because black box is going to make sure that the compiler believes that it has to fully produce the value, including the value inside of some. So I think this is totally reasonable. One of them got a lot slower, which one, this one, L3. Okay, I mean, that's fine, that seems fine. The same benchmarks, certainly there's some systematic issue with the benchmark itself. I fully agree with both of these. Thanks. Now, hang on, that's in search. I feel like we should also do the same for construction. Shouldn't we also mark the construction dupe, no dupe as inline, never. I see you have just a few tens of notifications. Do you use a separate account for your work? I usually have thousands. No, it's because work for me doesn't happen on GitHub, generally. If I did, I would probably have a bunch more. Wouldn't it be smarter to just black box the input? Black boxing, the input doesn't help here because it would be, if it's inline, then black boxing it, I suppose we could black box the argument. So we could, an alternative here would be to black box this size here, would be to black box the two lines that read. Compiler doesn't get to make use of that knowledge. Maybe we should do that in addition to forcing no inline. Even so though, I'm inclined to merge this because the changes are still reasonable. It's just that we might make additional changes. Port stack collapse recursive. Okay, this is back in Inferno. Stand alone script, script to be run on some pre-collect input. All right, what does this do? Is there any way of collapsing recursive calls? I wanna see an example of this. Oh, I see, it's if you have functions that are calling itself, you want to simplify, so what's a good, let's just pull up a random flame graph here, flame graph. Show me, okay, so here's what a flame graph looks like. And for example, over here, you can see that we have calls stub, calls interpreter, interpreter calls interpreter, interpreter calls interpreter and then interpreter calls this. When you have recursive methods like this, there are two things that happen. First is just a lot of noise. Like you end up with very tall stacks when you don't really need them to be tall stacks. But the other thing that's important is, imagine that you have a recursive function that sometimes diverges. So sometimes like, let's say we have a function food that's recursive and sometimes it calls bar after three iterations, sometimes it call or three recursions and sometimes it calls bar after five recursions. Then now you're gonna end up with a flame graph that splits. So it's gonna have two sort of columns, one with three foods and then a bar and one with five foods and then a bar. And then everything above the bars are the same because they're probably the same sort of branching points. The inputs were just different. And so collapsing the recursion there basically making all the recursive calls be a single call means you're also collapsing those columns and hopefully getting a better end result. And so what this script does then is precisely that it turns a stack like this. So main calls recursive, recursive calls recursive, et cetera, until they call helper into just main calls recursive calls helper. So this person, I'm gonna PR up. Visual thoughts. So this is the issue describing and we'll look at the PR in a second. Yeah, core recursive functions would be nice to de-duplicate too, but it's, that one's trickier. Nice. Yeah, core recursion is tricky. I think the way you would have to treat this is like if you see BC, BC, BC, then you were to collapse it to BC. But the moment they start diverging, I don't think you wanna do that anymore. Okay, so they made it a standalone script. So Inferno comes with Inferno collapse Perf, for example, which takes the output of Perf script and turns it into these kinds of lines of the stack semicolon separated, falling by space, followed by the number of calls, the number of samples from that stack. So it turns Perf script, which is that long jumble thing we tried to parse earlier into that kind of regular stack syntax, collapsed syntax as they call it. And the argument here is like, we could either have it so that the collapse Perf takes like a merge recursive flag or something, or we could say you generate some foo.collapsed with this and then you run collapse recursive as sort of a post processor on that file. So that's what they mean by standalone here. And I think standalone makes a lot of sense. So I don't think we need that as flags. So what do they do? They added binary, that seems fine. Number of threads to use. I don't think this even needs to be using folders. So in Inferno, we have a data structure, kind of like basically a little component that makes it easier to write a thing that produces a collapsed thing. But I guess it makes, I think it makes sense to reuse it maybe. So you can construct one of these folders and then you basically write things into the folder and it produces the right syntax. And I guess this is a folder. It's just that its input is also folder or is also collapsed. That seems fine. This is just the binary interface to it. Collapse file to standard out from folder. And I think this is pretty common. I think all of our, we have a bunch of different collapses for different tools like VTune, VSProf, Detrace, et cetera. Perf is the most well-developed one. And I'm just trying to find the, oops, that's not what I want. I want binary collapse Perf. As you see, most of this is command line parsing and then it calls folder from. So the syntax of these is pretty similar. It's mostly just parsing the configuration. So this seems fine. It takes a path, the number of threads, okay. Collapse, direct recursive backtraces. Direct, oh, direct in the sense of a function calls itself. Staclize and merge direct recursive calls. For example, collapse is this into that. Okay, that seems reasonable. That's just a summary for the module. Okay, so the only option is number of threads. That seems fine for now. You can imagine here that we had options for core recursion, for example. A middleware folder that receives and outputs the folded stack format, collapsing direct recursive backtraces. Oh, so this is the folder, which is the thing that implements the trait that, that's fine. We can ignore that from options. That's fine. Why is zero, zero is equal to one, that's fine. You can't run a folder with no threads. Yeah, so this collapse private is the trait that we use for this component that handles some of the generation of this collapse format for you. It's already collapsed, so there are no headers. Collapse, single threaded. For a line in, okay, so it reads over the lines, parses out the parts, calls collapse stack. Every line is an independent stack. Yeah, the way that this format, unlike the perf script format is very straightforward, like every line is a stack. So there's just, there's nothing else to parse. It's just if you hit a line end, it means you're done with that stack. And occurrences here is this component that helps count. So this is saying insert or add. What we, you know, we take this stack line, we call collapse stack, which is what turns the one with the recursive calls into one without one, with the same count. It's not gonna be guessable. So guessable here is used for, we have this tool called inferno collapse guess. And with guess, you just give it, you can pass like perf script into it, you can perf pass vtune into it, and it'll just call the appropriate collapse for you. It's worth as questionable, I think, but it's certainly true that we never really want to guess that the format is to use the recursive Collapser. So that seems fine. Line parts is we just split by space and we grab the count and the stack. So this can really now be, this can just be R split once. Now I think, let's do a review. Collapse stack takes a stack. If it's not recursive, yeah, this is nice. So if it's not recursive, then we don't have to do, and we don't have to allocate a new string that has the modified stack in it is recursive, just splits by semicolon. And if the previous one is ever equal to the current one, then we have recursion, otherwise we do not. Great, looks fine. There is recursion, so we allocate a new string starting with the length of the existing stack that seems reasonable. And then we push only if the frame isn't equal. Remove the trailing semicolon he has about to say. And then we turn that into a cow. We have some tests, collapsing the empty string does nothing, collapsing single does nothing, not recursive does nothing. Nice, nice. This one is debatable whether if you have multiple spaces, whether the collapse format, dictates that this is the last white space, or whether it's like the last sequence of white space or the last single white space. The trailing space here is questionable. I genuinely don't know whether the collapsed format is drink of whites is dictates that the count is only ever offset by a single space. Maybe avoid testing that specifically here and just keep a single space before 42. And then test for collapse recursive. So the test collapse recursive thing, I wonder actually whether Flamegraph has tests for this. Test probably does not. It does not, okay. Yeah, so I'm guessing they added some, yeah. So there's some test files here that just has stacks and we're expecting this one, for example, to turn into this. So main not recursive turns into four, main recursive helper one and two turns into main recursive helper three. And then we check that this works when invoked with one thread with two threads, whether through the API or whether through the command. This looks great. I like this to minor nits. And then I think we're doing this. Beautiful, beautiful, beautiful, beautiful. Done. Couldn't the pattern matching avoid duplication by adding a guard? Let's go back and see. You mean the pattern matching in, you mean the pattern matching in here, right? This, you can't have a guard on something that is only present in one pattern. So you can't write none or some L if L. I don't think that works. I think the guard is handled for the entire arm and not for a sub pattern. Unfortunately, this could be optimized a little bit like you could do, you could do something like last dot map or technically you could reduce the duplication here with something like if last dot map or true, not equal to frame. Not terribly important though. But yeah, it's a good call. Okay, order search. Benchmark payload in the case of ordered collection and B3 set contain references while sorted VEC operates on primitives directly. This is incorrect because it punishes ordered collection and B3 set with one more level of memory interaction. In this PR, I've removed references from Make This and Make B3 set payload generators. Performance improved significantly. All results are reported including changes from 5. Wait, I'm confused. Now ordered collection. Oh, as in this. The ordered collection here is this. Across almost all benchmarks. Nice. Okay, let's look at this. If this is copied, this can just use dot copied, I believe. But that's fine. Interesting. So previously, the argument here is that previously the benchmarks would let sorted VEC have a vector of T's. Like if we had a vector of U sizes, for example, it would actually contain, it would be a vector of U sizes. Whereas B3 set and and R type here, R would be collections of references to U size. You end up with an additional indirection that's entirely unnecessary, which unnecessarily penalizes us too. Good catch. Faster across the board. Should probably update the read me as well. Now that we have such different results. Approve. I don't think the copied thing is worth blocking on here. Single commit. Single commit. Great. Merch. And then what you can do here. I think this could also use dot copy. Oh, no, I know I can't use dot copied. Rust copied. It can't use dot copied because it doesn't need operates on operates on an iterator over shared references. But what we have here is a because they call into here instead of iter, what you end up with is a mutable iterator over vector, which yields mutable references. And so the type signatures wouldn't work. I think you can simplify this slightly by using dot iter instead of into iter dot copied. Okay, or search. This is from someone else. Nice. Let's see what they say. Criterion is a more full featured statistics driven benchmarking tool and creates nice graphs. That is true. I like criterion. Regardless, I think it would be interesting to re-bench this code in a variety of computers and 2023. See how things stand. Or search sorted back and be reset. I agree. That is going to be interesting as PR is a work in progress. Let's see this PR emerge for several reasons. Yeah, I do. It does make me sad that the yeah, I do kind of want to move to criterion. I totally agree. This execution time median is stable within 100 milliseconds and stops execution. Still want to see this happen. So if you're up for happen especially with all the improvements that has made recently. Any chance you have time to pick it back up. This is also something you'll see pretty often is here, I could just do this myself. I could just port the benchmark suite of or search to use criterion and it wouldn't take me all that long. But I would rather the person who originally started this do it instead for two reasons. First of all it's not clear that it's the best use of my time. As you can see there are a lot of things to go through here that require my input specifically. Not just anyone who knows Rust but I have to go deal with a bunch of these things and no one else can do that work for me. So I'd rather spend my time there than here on work that other people could do. And the second one is it encourages someone who already has shown some interest in this codebase to continue to develop that interest. And so hopefully they might then become a contributor and eventually maybe a maintainer. Because for many of these projects I am the only maintainer and I would love to see other people come in and help and essentially not take them away from me but if I can let other people have the agency to do things with this crate then it can get better without me being a blocker for it. So I very much here want to encourage other people to do that work. It's not really selfish at all even though it might come across that way. Done. This is in Factory RS. Okay, so this is there is a this is a job server so something called Factory Factory is a job processing tool. It's basically a job queue that there are a bunch of previous implementations of this sidekick is the other one that many people know about. Factory is basically the replacement for sidekick and many years ago I wrote a client implementation of the factory protocol so you can submit jobs to factory and you can also implement workers over factory and I never actually used it myself. I just found it interesting and I like implementing protocols and here this has already been fixed. Oh, I see and someone just left a note about how to do it in case someone finds the issue but didn't see in the documentation. Okay, great. Thank you for sharing your knowledge. Nothing for me to do. This library is fascinating actually. This library I've had to do very little with and it seems to just kind of take along. Like I haven't modified it since September of 2022 and I think people are using it. It almost certainly could use some dependency updates. So if anyone wants to do some dependency updates on factory feel free but otherwise it seems to be ticking along finally. Is this a good project to help people learn Rust? Which project? I've gone through a lot of them so far. Rust IMAP replace NPC sender receiver by VecDec or VecDQ Okay, so this is a long discussion I've had in the past. Okay, so this is in the IMAP crate which implements the IMAP mail protocol the client side that is and in the in the client protocol based on the RFC it's possible for the server to send messages to a client that aren't related to the client's request. So if you're writing an IMAP client generally you're going to send commands to the server like show me all the messages in the inbox and the server is going to reply with a list of messages so far so good but the server is also allowed you to send unsolicited messages so when you send you know tell me about messages in the inbox it might start sending you a list of the messages and then randomly send you a message that says like new email arrived and then continue with the listing and that new email arrived message is an unsolicited message and those can arrive at almost any point during your connection so as a client library we need to handle those the way we currently handle them is that we have a we have a sender and a receiver channel so whenever we detect one of these we send on the sender and we give the in the client API we give out the receiver so that a consumer of the library choose to handle these unsolicited messages however they might wish now it turns out that because we store the sender we the client ends up not being send and sync and I forget exactly why it's not send but it's certainly not sync well for some reason the client ends up not being send and sync because we hold on to this channel oh it's because we hold both the send and the receive side and then it ends up not being send or sync and this is really annoying because you really want the ability to take a client and like give it to another thread for example and so the solution here is it turns out that because of the pattern in which we use this channel it doesn't really need to be a channel at all it just needs to be a queue and when you have a queue when you want a queue you can just use the Vectek type library so at the conclusion of this discussion was let's just replace the implementation or replace the use of this channel with a Vectek instead so hopefully this shouldn't be too bad that seems fine cargo lock I'm going to ignore source client so we're going to keep a Vectek and anywhere where we currently pass in a mutable reference to the sender we're going to pass in a mutable reference to the Vectek these annotations that all seems very reasonable where is the place where we give out the receiver oh yeah that's just a public field on the session so that allows people to read out of this but that means this is a breaking change because of course it is but we're about to cut a we have been about to cut a new major version of iMac for a while so I think that's okay in case we want to change this again in the future I wonder if we should stop having this be a pub field and instead have a get unsolicited method or something like it maybe one that way the exact type we use here isn't exposed in the API and thus can be changed so we would want something like impulse session and then we probably have pubfn next unsolicited which returns an option unsolicited response and we would have a all unsolicited which returns a vector of unsolicited response what do you think? start to read in the types of everything has to change because they're now getting a different type in this is oh this just creates one, this is for a test so it just creates one internally for the test that's fine this changes looks fine that looks fine alright what we got in parse this is all just type changes type change this send becomes a pushback that's fine wait this should be a pushback now I'm paranoid let's go back and see that these indeed are also pushback this is an extension that just passes it elsewhere so that's fine this also just passes it elsewhere so that's fine pushback pubfront is none instead of is receive that seems reasonable that is indeed how the API has changed try to receive is pubfront aren't type changes like this just the best great this looks fine is factory a good one to factory might be a good first place to start with Rust actually the code is not super complicated so if you're looking for something to play around with factory might be a good one I don't know Fang yeah unsolicited messages have tripped me up a bunch of times both client and server side missing semicolon on a line I don't think they're missing semicolon see I would catch that okay this is fine this is fine whoops this is fine this is fine I think it's only the parser which is the one we looked at that does pushing pubfront push back takes from the front of one queue and pushes to the back of the other which is equivalent to appending the queues great pubfront pushback and this is a deprecation in chrono which is fine I'm okay with taking that change as part of this reply this one bit in there but I think is wrong otherwise looks good done still can't run multiple tasks in parallel using a session and that is because every session borrows self as self mute yeah so this person wants to concurrently use an I map connection across multiple threads and that indeed will not work because we only have a single TCP channel as ultimately you need to synchronize on that TCP channel because you can't really have multiple in-flight operations over I map over a single connection because the the I map protocol like if we look at RFC I map 3501 if we go look at some examples from the protocol like if you do select in box these are the messages you get back and notice that if we had concurrency here if one thread sent this and then the server sent like these two and then the client sent from a different thread sent another operation there's no way to tell that the response whether the next response line is for the first connection the first operation that's done finished or for the new client operation from the other thread so these like stars which are just sort of line continuations wouldn't allow you to operate concurrently anyway fundamental to the I map protocol it doesn't allow for multiple concurrent requests so you'd need to either use a mutex or a separate for each client unfortunately ok fresh ok so this we finished this we finished this we finished Chrome driver not respecting headless that browser name it won't process the argument all otherwise ok interesting we need to specifically tell the browser that specifically need to say that you want chrome in order to pass headless that's interesting retainer access to all crates interesting yeah that seems fine openness is a trust this is one of those projects where I who am I writing this person this is one of those places where I haven't fully handed off openness as yet but nobody zoo is doing a lot of the contributions to the crate these days and I still do reviews and what not and I still do the actual releases but we created a github organization specifically to make it easier to have multiple of these sub crates for example and you know they're doing a bunch of work on the crate that I'm now not doing and I'm ok with that like I want other people to to help with this because I don't have the capacity to do it all myself so let me this is always a tricky thing when you're a maintainer which is like here's someone that I trust to make changes to the crate who's asking me to add another person that I haven't directed with to give them access to make changes to the crate I think in the world of open source like in close source world you would never do this right but in open source world I think you kind of have to like you need to generally be willing to bring more be lenient to bring in new people into the fold it's sort of a trust but verify kind of situation where you know I'm still going to get notifications when there are changes to open SSH crate for example I still make the releases so I can sort of monitor how this goes and if I feel like this person should also like is fine then I'll stop thinking about it and otherwise I'll be paranoid for a little while but that's ok so invite member this person invite and security key I have a security key can you hear the keys jingling remember great wouldn't extend be more efficient to concat vik that is an excellent point let's go back and do that so it's going to be IMAP floor quests where does that live that lives over in I think that was actually in the test now that I think about it but it might be in fetch it's not in fetch somewhere with a few more lines maybe I don't think it was in parse but maybe it was not in parse yeah this is just in tests so I'm fine with that I don't think that's important oh is the IMAP for second revision RFC nicer I should look at that I've done too much supply chain work oh yeah I'm paranoid about giving people permissions to things rest free restations it's incredibly information dense so this is an issue filed for the book however it's incredibly information dense and touches on many niche corners of language with descriptions and advice only is arepa worked examples hi there oops glad you like it no there is not currently any such repository the book is intentionally quite dense I wanted it to act more like a series of pointers for what to explore and learn more about rather than an exhaustive and inevitably eventually outdated reference if you want to start an example repository though I'm sure many people would be interested in collaborating on it close with comment we're under 100 sweet and there's a bunch of dependabot things let's do this one first this crate I'm not really maintaining anymore okay this is deprecations so I don't really want to land that one as is I don't know what to do about this one I'm torn because this is it is a really cool crate so it basically implements the MySQL client protocol so you can write a thing that looks like a MySQL server so anything that speaks MySQL is able to operate with your thing and you can build whatever you want behind it so it is really cool but it's sort of languished because it's not something I've really been working with if someone was to take a stab at picking up this crate it is a really cool crate you can build cool things with it it mostly just needs a few updates to bring it to the latest MySQL version and other things too if I remember correctly yeah MySQL the OpenSSL one shouldn't really matter I guess I can take that why does it even want to bump OpenSSL here OpenSSL is the thing that has the deprecation so in that case I'm fine taking the MySQL one too except of course that it doesn't pass tests which seems problematic or only on certain platforms it doesn't pass minimal versions and it doesn't pass Mako's latest minimal versions fails on something with BindGen and this is almost certainly the OpenSSL bump so this is fine I'll take I'll take that one so this really means I should do a a a new major version release of that because MySQL is in the public API that's fine bumping OpenSSL here I think is also okay and now it only fails on MakoS and Ubuntu Beta why does it fail on Ubuntu Beta because of the deprecation okay great that's fine I will merge that too if someone wants to take a stab at the CI failures in MySQL server then please do delete branch quick check in a tone okay so a tone is a different library a tone is a library that is like a vector or more like a Vectek that so the way that these data sources often work is you know they have a limited capacity because you allocate a vector that's like this long and then if you push more items that are on the list it allocates a vector that's twice as long and then it moves all the elements into the new vector and now you can push the new things and you have more space the problem with this is that you end up with latency spikes anytime you have to double the capacity and move all the elements you get delayed by more and more time because like imagine you have a thing with a million elements and now you allocate a thing that has space for 2 million you have to copy a million elements over that's a lot of work to have to do and so you start seeing latency spikes in your application that happen less and less often but take longer and longer time and so what a tone does is it essentially instead of moving all the elements at once it keeps both the small and the large array around until and then moves a few items at a time every time you push and then only when the small element array when all the elements of the small array have been moved over only then does it deallocate them so it allows you to smooth out that latency cost let's see quick check nice no breaking changes in quick check that's interesting is that actually true and read me no change log not a change log huh releases there aren't any releases tax version 1 that commit bumps the version numbers okay that's relatively unhelpful that's as of January 8th 21 I don't think I'm that far behind there's 092 move send bound move rand as a public dependency MSRV bump not an compatibility okay sweet I mean it doesn't pass why doesn't it pass yeah that's what I was worried about I wish there was a an actual change log for this upgrade okay so something more here has to happen in order to do this bump this also is a good thing for someone to try out if you're curious about quick check because here there already are a bunch of quick checks like property tests that you should be able to tweak slightly to make them work with quick check 1.0 so if anyone wants to pick this one up I'm not going to do this one on stream I think if anyone wants to pick this up I'll put it in chat it might be a good candidate it does not prevent the allocator from just resizing where it's possible it does but the latency spikes are too costly for applications that are latency sensitive oh for two factor off I use a just a Yubiki one of these things sweet so hopefully maybe someone picks this up I'm going to mark it as unread uh this is on Funtouchini updates hyper rustles to 024 native TLS oh yeah native TLS should bring in open should bring in OpenSSL why as it was it was forcing any user from Funtouchini create to depend on OpenSSL even if you weren't using the native TLS feature interesting that shouldn't make a difference but alright that seems fine to me oh I see what happened it's because it wasn't marked as optional down here and now it's marked as optional I think this isn't even actually needed because the OpenSSL entry there but it's because cargo requires at least one feature to name any optional dependency so that's why uh okay when I use perf script with detrace this is because you're trying trying to use perf script with the detrace Collapser you'll want to use the perf Collapser instead so perf okay yeah so someone posted it pointed out in chat like this has been a 4 hour stream and we're still at 96 we started at 118 and we're now at 96 notifications and I'm sure if I went to the front page some of the ones that I commented on now already have more comments on them that I haven't dealt with yet oh um there's more imap ones previous to 251 I was able to call pars to create mock response objects using my application testing I can write tests for their code that calls the imap prep without having an imap server set up oh I see um so this is someone who wants to test their code that runs on top of the imap crate um and the challenge is that they don't have a way to construct values of the imap types um which you need in order to test your program you might need to like generate a random like fetch response um and we don't want to make the types fully pub because they need to be non exhaustive sometimes we add more fields um having a way to expose builders isn't a bad idea I think it makes a lot of sense to expose builders for generating objects for test purposes we just need to be careful that we don't um open up potential backwards compatibility hazards um where we won't be able to add fields to structs or change to their internal composition in the future allows building response objects for testing applications uses crates okay so it adds a testing module to use add a dev dependency imap extension adding the future test helpers uh right we should probably add that also to um I forget we already have that here actually do we already say all features on docs.rs uh we don't do we um nope tokyo I know tokyo does this why I'm pulling it up so there's a nope there's a little section you can add to cargo toml for docs.rs to tell it to generate your documentation with all features enabled um and I think we're gonna want that let's also add this thing so that docs.rs will document the crate with all features enabled uh okay so that part's fine this needs uh code block around it with I don't know how you would do this like I guess it's mark down and then I want to nest this yeah oh can you use quadruple backticks in the outer block nice uh these feel like they're probably not relevant to the instructions we want to keep the example somewhat simple for people there's no need to include all this extra complexity um right so this just lets you call the parsers arguably what we want is something better than just you can give us the raw input and we'll parse it for you right it'd be nice if there was actually a way to construct these as well but I guess this is a decent start um put this and all the other examples in mark down so you're saying I can do this and that'll work nice works so that they run as tests and then we should also actually invoke the function and check that the response is what we'd expect heh there's more things that shouldn't be pub um although I wonder whether they even should be pub but maybe they um this is a good start although I feel like uh what we'd really like to get to is full builders for all these so that folks who want to test uh don't have to generate full imap protocol messages and can instead just use a builder to construct the value they need for testing definitely something for another PR though um I think I want these it'll be nice if these were in sub modules instead uh so that we have a place to add builders later I'm thinking something like mod capabilities pubfn parse input vac u8 I also think we should uh use impl into vac u8 instead um so that byte strings will just work and not require dot into and finally since this is just for testing and to help ensure that's the case what do you think of always dot unwrapping inside of these rather than exposing the result makes it less attractive to use them for anything else uh what's your take on doc test versus separate test which one is better um they serve different purposes doc tests are intended to be visible to users so they should be testing things that uh they should be demonstrating something of value to the people reading them um and anything else should go in a unit test um anything that is like testing internal invariance anything that's testing you know multiple different combinations of things where the user doesn't really need to know or care about all the combinations all of that should not be doc test because it doesn't need to be doc tests are for demonstrating use uh or corner cases of interest to the user so that they serve different um they serve different uses um picking this up some notes so you used quick test and your resources and all these different ways to test code and where to learn about them how to use them um i mean i talk about this in rust for stations you could read about it in there uh but the the basic premise here is like there are a bunch of different testing strategies and you use them for different things i don't know of one resource that talks about all of them my book does to some extent um but you know the these are more like categories of testing right so you have um simulation testing like something like um uh loom for example uh you have regular testing where you exhaustively specify the behavior of the tests um you have chaos testing which is where you just sort of run the test multiple times and you modify semi random parameters or parameters in the environment to see whether you can generate it to do the wrong thing like run this test 100 times with like random inputs and then you have fuss testing and the advanced sort of property testing where you um have a framework for generating inputs and you check that the assertions of the code don't get triggered by it um and all of these are valuable they're valuable in different ways it's not as though you should do one or the other um add in a client builder that abstracts a way connecting to tls or non-tls connections and what tls provider is used um allows a more transparent and versatile usage of the library as one can simply compile it as is and then use the builder to configure where we connect and how we connect without having to be concerned about what type is used for a client and session yeah this is a long debate we've been having uh I don't even remember where this ended up oh this has a lot of context for me to switch back into I think I'm gonna not do this one on stream basically the premise here is this is a summary is um in the imap client today uh when you want to connect a client uh you have to pass in like a tls connector and you get back a client tls stream tcp stream so it's like the fact that you're using tls and which library you're using for tls is encoded into the type um that's nice from a performance perspective and it's nice kind of from a security perspective because it means that you can enforce a compile time well you get the benefit of uh monomorphization for the the IO and also you get to see from the type and check at compile time that you are in fact using tls the downside is that it's really annoying to work with now you have different types for different types of connections if you want to change tls so now you need to be generic if you want to build a library on top of imap you need to be generic over this internal type um it also means that it's kind of harder to do the right thing a little bit because you need to configure a bunch of things it doesn't just kind of work you need to choose an implementation choose a type um and so the proposal here is to instead basically get rid of this generic inside of client um and have a client just be a client and not be generic over its inner connection type um and then have that essentially be a an enum internally in the client for what kind of connection do we truly have um so that the user doesn't have to think about it and there's been a lot of debate about exactly how we should do this um and it ended up like we needed to this is this has been going on this PR for like a year a year and a half and so we've constantly have to like re-base it on top of other changes to the library um at the core of it is essentially dynamic dispatch like we have a trait that implements the things that we need uh and then everywhere we just keep a um let's see if I can dig this up uh I don't have an example of this right now but we basically keep a connection which is like a box din of uh box din of this thing um so that allows us to read and write and do things like set timeouts on the connection um and that way yeah done here box din in imap connection um and so that way we're just doing dynamic dispatch which is usually okay anyway because when you're when you're doing io like this you're not really you're not compute bound like you're not bound by whether you're doing monomorphization the io is the bottleneck and so it's okay to use like uh type erasure here the the box din is not going to have a meaningful performance impact um but doing a incremental review of this I don't think I'm going to do on stream because there's as you can see like a lot of context to catch up on uh so I'm going to leave this one as unread for now um all right let's do one more and then I need to go have lunch uh the two more because that I think is just a user request might be said to deal with haphazard has a lot of context to it so I'm going to ignore that this is for my configs which we can ignore um this I kind of don't want to deal with those right now differential tooltips are okay fine we'll take these all right and then those are the last three we're going to take today and then I'll have lunch um what is your criterion for doing things on screen or not there's not really a criterion it's more that I need to eat uh and so I don't want to take anything I don't want to do anything now that I know I'm always going to take a while because then I won't eat for longer uh it's currently 2pm where I am so it's way past lunchtime uh do you think cargo will get support for custom testing frameworks I hope so but it's uh it's something we've been wanting for a while and it's actually kind of tricky to land something that works well yeah I can see youtube chat uh returning prototype dot array console log but nothing from println uh where do we have execute sitting in the kromby bugger after I click the icon I see empty result lines from the fantocini oh who knows man it'd be all sorts of things I don't think this is a bug in fantocini it might be a race condition in the code where um the value is empty by the time you get to the return uh it might be that the value is empty at the time of the return but by the time you look at the console it has resolved um next thing to do would probably be to enable debug verbose mode in your web driver to see what actually goes over the wire and I'm going to close it because this is a support request um okay these are talking about differential outputs in flame graphs so differential flame graphs are let's see if I can pull up an example of them there's an example of these in here though that's what I want to see okay so a differential flame graph looks like this so the idea is that you give in um you give in two flame graphs one or you give in two collapsed really one before and one after and the flaming graph that gets rendered shows you how much faster or slower each stack got um so like this is showing you that uh main got or whatever that red line is got this much slower than the other thing um they can be a little hard to read which I think is what the point here is um so if you have something where base only both okay modified so this is the before and the after right so when you generate a differential flame graph for this there are like tooltips over each little bar um that shows you the backwards looking I have to page a lot of this back in um I think backwards looking is um I don't know what they mean by backwards looking I think it's whether you take this as a and this is b or this is b and this is a um okay so what actually happened here what actually happened here is that uh base only went down by 50 units so it went down by 100 percent both one stayed the same both two went up by 25 so it really went up by 100 percent and changed only went up by infinite percent both one 12.5 percent oh because it's computing based on like the total number of samples represent overall time but that's not what most people care about they care about absolute change yeah reasonable semantic is multiplier from old to new consistent across both directions an example above base two would be new equals two times old and we need to know if you care about backwards compatibility my feelings the current behavior is useless enough just removing it would be fine but this looks like maybe the scaling factor option is broken um oh that's okay so that's that separate issue all right um I think you're completely right that this is confusing and I like your proposal to make it um multipliers instead and consist consistent across directions consistent inverted across directions um totally fine with uh removing the current info and replacing with this better info is there a diff these two parent first child 10 parents second child 10 parent 230 and parent first child 30 second child 30 parent 230 so as parent 2 is unchanged even though it's actually gotten larger yeah this is that's interesting um this is also a good issue for someone to try to tackle actually this is probably just a simple bug somewhere in the collapsing code let's go see if we can spot that real quick so that's over in source differential mod uh somewhere over here in right stacks I think I need to see what the um the diff produces here so if we do uh all dot prof and we do new dot prof and then we run diff where we get out 30 to 30 10 to 30 10 to 30 okay so that seems reasonable uh so then it's going to be in the in the actual flame graph generation yeah it's over I think I know what's going on I think I know what's going on it is um so when we render these what we do is we um we're sort of accumulating data as we want so you can't render parent until you've rendered both the child or until you've parsed both the children because the children tell you how large the parent is um but I think when we render the parent we only keep the the sample number we don't keep the we don't keep both numbers um and I think that's up it's under merge actually um and samples differential okay so here we have a delta and a max we split by semicolon and then we call flow and we pass in delta okay so in flow where's flow flow delta common prefix oh right there's like a super funky algorithm here for dealing with these files because you like detect whether you have a common prefix with a previous line that you parsed so here um you detect the prefix there's no shared prefix between this and the line before and only at that point do you emit the contents for parent 2 and then when you get to here you notice the shared prefix is the same between parent between the previous line and this one so only here do you emit the stack frame for first child and then you emit the stack for second child when you get to the empty next line and similarly that's also when you generate the um the frame for parent so there's like a yeah this is where we look for the shared depth we iterate onwards from where they share huh could be here could be wherever this thing is um where's the thing when it closes something I'm missing here where's flow mutable temp and mutable frames okay so frames ah yeah so this is where it loops over all the things that uh are previous to where we are yeah it's in here where we need to add to the delta not set I'm pretty sure it's this one um I'm not going to try to actually fix this right now but um I might be able to point this person in the right direction so more I think this isn't the rendering but the actual um diff collapse computation specifically I wonder if it's over it's related to this comment okay I think that's where I'm going to stop for now we we're down to 91 that's pretty good I need to go eat so that my brain will work again um but I have as you can see I have more open source stuff to catch up on so um I might just I might start just doing more of these whenever I want to continue to catch up on notifications if you all found this useful I mean there are a bunch of you here so I guess I'll I'll ask you all do you think this was useful because we've been going through a bunch of different random projects doing all sorts of different things mostly smaller tasks and keeping up but was it interesting useful um helpful inspiring um then let me know okay some people seem to think that it was twitch people do which means youtube people will tell me in about 10 seconds um okay I'm glad to hear it well then I will try to do um more of these going forward when I have to catch up on open source stuff I probably won't do it again after lunch I need to my brain needs to relax um but there will be more of these at some point in the future okay thank you you all um thanks for coming out I hope that was interesting and if you're watching this on demand maybe there's also another one of these that's already out otherwise I'll see you later