 So we are finally ready to start this little very short now five minutes short to talk about new Features in this trace. I'm Dmitry. I am a maintainer of stress for quite some time and today I'll give a short talk about new features just just a few of them because well most of Stuff we are doing is quite boring and nothing to talk about But very important for these features and other new features to work But if you're really curious, you can have a look at these notes and find dig out all the boring stuff is there But today I'll stop Remind you about fault injection. I was talking about last year and we'll talk about new features some of them are already included and already released and Some are still being merged So this is a recollection from the last year For the injection if you just happen to miss this new feature it's very unusual for traditional stress that you can tamper with system calls like in this simple example or Well, you actually can inject any kind of fault in any system call you specify like in those invocations you want and This is also from last year's talk. It's a real bark was found in Python that's So already fixed now, but you can see that it just failed to check return code from OpenSeas call and from Reese's call and that resulted to sick falls because then just accessed and Fixed upset from an old pointer another bark was fixed in Dynamic linker in Glypsy Pretty harmless bark, but still it was a bark was ignored Setting protection Some portion of memory expected to be inaccessible, but it failed to check this and it remained accessible This is not good But it was fixed also So this is just a quicker correction from the last year's talk if you are more interested you can just Have a look at it and play with it. It's very useful feature. You can use it for like regression testing We we actually use it in this unit of a stress itself, so you're welcome and We implemented more features in this direction So there is a new option called inject that can inject not just false to see schools, but other kinds of things like Signals to school vacations other kinds of return codes that are not that useful and also Delay injection. This is last. This is a quite new feature. It's not yet merged that you can inject delays in invocations on entrances call on any existences calls and Like if a stress wasn't slow enough You can make it even slower in specific cases as you can see in this example like home Odors of magnitude is slow one to like three orders of magnitude slower. Just those specific schools you want to besides this silly examples where you can also Well silly I Actually been asked several times that people wanted to slow down some schools to throttle network traffic For example and those tools they that don't implement this like slow down rates on or sends or some other schools another kind of use for this feature is like in fault injection case is for Regression tasting where you can Specifically slow down some process that's expected to finish some to complete some operation very fast And if another process or thread is waiting for it for a fixed time, this will fail So this way you can do that kind of inject some delays that affect control flow Let's do this my talk consists of small blocks and if you have questions When you see this thing like five slash five it's the end of block and you can ask them right now if you want If not, let's move on to another block so This year actually last two years. We've been a project Mmm It was a Google some of code just a project of two years that implemented parts of for netline traffic The first year was a pretty ground very groundwork And last year we implemented the most popular between protocols starting with the route and you can see like nine of them and And more are going to be implemented this year thing Hopefully How does it look? This is a very simple routing table to to make this out in table you you have to Create like a basic container with no network interfaces just a little back interface If you create this you will have just four kernel kernel routes So how does it actually look what? IP tool sees from the kernel Let's this is going to be a very impressive picture be prepared That's what's going under the hood you see this is very structured output. I Did some coloring it's manual. Sorry. I did it by hand. So but you can see that I mean The tool it doesn't see any colors. It's a structures Oh, sorry It's just very simple. You will see schools but without all these structures. It would be just String or pointer depending on what kind of as you you know, we've sent to it would be instead of this Instead of this structure that starts here Yeah, we carry brackets. It was it would see what? Sorry, it would see just in all the stresses. We would see just a string Binder thing that means nothing but here you see here like a Structured output almost No, this is a little crowd, but are also supported You can see on this None of them So it's almost like a nothing debugger without probably Can't really inject all this stuff inside the program, but you can see all this Gory details, so then let's go on Another set of features we implemented is related to Cisco specifications as traces you probably know have a lot of Filtering like it can filters is calls by classes This time we decided to add some more classes, but to avoid Collisions with system calls. We added prefix. So all new system call classes started with prefix Sign and all all traditional system System call classes are also have this prefix. So these are traditional and we also did like For different start family syscalls You can see in this example how it looks like what does it mean two persons just the way to describe all Start process family syscalls Why so so many? How do how many f start system calls do we have in the nooks can do a thing? Five Why five well 64-bit architectures usually have two of them 32 bits also too, but different and there's also an engine system call that modern architectures don't have But you don't have to worry about all these details just use the system call class name That's all as you can see in this example On the 20 on the on the 27th year of the project we decided We found out what we don't still have support for regular expressions How could that be? So what did this and you can see that the same example from a previous slide could be could be Like Using you can do the same thing using regular expressions. I don't think it looks easier to use but With the regular expressions you can do all different things. Oh, thank you The time is going back I Another feature of system system call specifications is conditional expressions as you probably know System calls are added like every or almost every new Linux kernel release and They are not added to all architectures at the same time. It's not normally the case for example the recent From the stat family called stat X. It was added for many architectures, but not for all and for example, you you write a somewhat portable script using a strace You will get into the situation that it works in most cases, but doesn't work on a Monish architectures like Italian where statics up to this point is not yet right up so to Using this conditional specification you can work around this and this is a real case example you will probably Seen this or we'll see in future that GNU lip see switched from open to open it internally and also to for implementing open Library call so if you used to trace by open it no longer works starting with with glip see 2.26 But if you switch to open it you won't be able to trace Older glipsies. It's not good So you can use the list But it's not portable because open is not available on some architectures So how can write a portable stray script You can use a regular expression or you can use a condition so There are two examples which one is easy to read. Do you think a regular expression or condition? I Think addition. That's why we added condition despite that With the regular expressions you can describe the same thing, but it's usually looks more complex Yes It's it it says that this is called Specification doesn't happen to match any real system call So you can write full bar and it will stress will not trace any for bar because there are no for bars is cool Yeah, if you Just type SS call or regular expression that doesn't match any CS call the stress will not run It will issue an error as you can see Yes, sorry Why not make a condition the default behavior was the question First of all, you won't detect any accidental errors people do errors Sometimes It will be a change of behavior and for the project of 26 or 27 year old It's not a good idea actually to change Behavior in a backward and incompatible way for not a Not really good reason Okay, the last topic is advanced is called filtering syntax This was implemented also as a part of Google Summer of code project last year and it allows even more flexible specification of these calls for example You can trace by file description number Without well the traditional syntax you can just either trace by specification as you can see who could see before or you can dump by For description number now you can combine all these things you can combine actions Default is trace you don't have to specify it. It's written here Trace but you can omit it as you can see on following slides. You can also combine Different kind of things like you can as you can send this example you can trace read syscall and Dump only those syscalls that access a specific file this way you will see that in slightly artificial example You can see that some program accesses Different things but we dump on the laws of them that are actually interesting why dumping them the zero could be interesting They think well, I've seen once it was not a device, but a file in real life It was not a good idea to replace the zero with the file. I think somebody just with a Super user privileges are directed output there and Some also did something like this stupid thing and well You can use logical expressions like and not brackets This way you can do very flexible filtering As you can see you can select only of those syscalls that Look with files, but doesn't take Descriptors as arguments or and doesn't return descriptors is written as their return value or you can all kinds of combinations logical expressions As you can see in these examples in the first example you don't see anything Just As you could expect cut Works with a file it opens it it does something with it, but you don't see all these syscalls on the doors that are specified this is another example of that kind of Combination you can use Like Before this you you couldn't specify what kind of Syscalls you want to stack trace So all syscalls in traditional syntax all syscalls selected for tracing would be Sectorized if you request it now you can select which of those syscalls that are traced are actually the stack traced So you can see This program accesses description number one, but we're interesting. We're interested only in those Closes called to have a look where it goes from as you can see It's it's from exit hunter. So what cut probably does it installs an exit handle that closes Standard output and it's actually what's happening as you can see in this example So that's most more or less all This Who's this living this is a basket over a stress Up to last year had no musket So we're very happy that we have one meet our new musket If you have any questions Yes, please I So the question was why one might want to filter something in a stress rather than Pass a stress output right well Tracing makes things slow If you don't need something then you don't trace it you don't pass always Imagine how many syscalls you you have to issue to fetch all this memory From tracing if you don't need this you just don't that's why Yes, please Yes, it's not really net link tracing it's It's advanced parsing of the Traffic that goes Because by default if it's treated as a string It means nothing or almost nothing It's a parsing. It's not additional tracing capability. It's additional parsing of things So this is cause are the same or It could be any kind of syscall it could be just read right Yes, please Other things which are sometimes very helpful to understand We don't have parsing for DNS protocols and I'm not sure because it's not Something you can distinguish from a socket for example once the socket is a net link socket you can find it's out So I can at least I could filter by potency make that trace filter whatever Would be a nice feature and now I see that it's really impressive and some more Final blobs getting So it probably would make sense to implement some kind of plug-in system to So one would so one would just implement a posture for this chunk of memory the states would fetch it and Give to a plug-in to Destination whatever Sorry, and then have a set of rules from Tracing something at that part that's destination that system called something use this plug-in for analyzing code. Yeah, for example It's not really a wire shot it's a straight For a complex problem and you only want to see what's going on the network before you're made for example, that's your things Yes, this also could make sense. Yeah Thank you And the wheel just make an interface to I want to have the feature in this place finally just get this one of our for Yeah, this makes things Not to replicate all this work making why shark and to speed up and what whatever else? Yeah Any more questions We can choose something for a name Yes, I think it's it starts. Yes He's nice, isn't it? Any more questions no, and thank you