 All right, welcome to the next talk in our series right now We've got Muhammad Mokbel up here to speak on the SPF shell framework. So please welcome him to the tour con stage So in case you're wondering what the Word-forming element the first character in it the character f white is capitalized It's literally for no particular reason other than the fact that I want an acronym That is three characters minimum. So that's it Yeah, I come from Toronto, Canada and I promise that I come in peace and I'm bringing no cold. Otherwise would be all frozen by now So that's that is there anyone from Canada here. I can all by myself and Okay Better information about me I'm secure session and trying to micro the big point. I'm a member of the digital vaccine lab I do reverse engineering primarily malware vulnerabilities and write filters for tipping point next-gen IPS I'm a former student senior security investigator at CIBC one of the top five banks in Canada So I was an L3. I was doing an investigation reverse engineering I also was former reverse code engineer and malware researcher at the security lab for about five years I'm mainly interested in reverse engineering and malware research intrusion detection prevention systems and I have a special interest in C++ as well as compiler and software performance analysis and In general, I'm interested in information security and what comes out underneath it and As part of my involvement in reverse malware reverse engineering I got to see a lot of interesting exotic communication channels slash protocols so that that alone worth the separate talk so This is The first slide in the talk. I promise that the talk is not about wash our display filters So don't get this courage But I just want to make sure that you know What are display filters in white shark, right? Can anyone tell me what this display filter is supposed to do? Okay, so all it does is that it's checking for HTTP post request With the fact that you have a content length header and the request set to zero yet. There is a payload Does it make sense? Of course, it doesn't make sense, right? It doesn't even adhere to the RFC specifications This is just like a bit of introduction, so this is the kind of display filters you write for why shark and now we have T shark T shark is the console version of white shark, right and And this is the this is how you would invoke it are Takes pick up file and why is where you write your display filter as simple as that and Here remember I'm talking about display filters. There is a difference between display filters and capture filters So display filters work with a static pick up file that you're already analyzing Capture filter works works in real time as you're capturing traffic. You're filtering it. So that's different talk Now that we know what what is wire shark? What is T shark? Let's go to Windows command processor The holy grail of who runs and controls all okay again. The talk is not about CMD I just wanted to set the stage for what what's coming next and To to automate some of your tasks in CMD You would use what's known as those key macros, right? It's extremely limited and it doesn't give you much of Features to automate your tasks in a generic way, right or at least in a democratized fashion So what is the motivation for this talk? I Know this sounds weird the first point But literally I wanted to explore the new feature in C++ 11 and 14 and what is better than? starting a new project from scratch, right and testing different stuff and I have a user of wire shark. I literally use it on a daily basis except of course for Saturday and Sunday, otherwise, I wouldn't have a life and I wanted something more than Add a display filter button I'm sure whoever is familiar with wire shark So when you write a display filter if you want to save it for later on you just have to the right There is a plus sign what do you save it under a name and that's how you automate your Invocation of display filters And of course it is a pleasure on working something in you starting from scratch From designing architecting engineering testing and documentation like the entire process This is personal project The agenda for this talk is as follows. I'll be introducing the framework itself why I created it Features internals how it works and how to write constructs. This is what I call construct I will see later on what they mean and And how and in most importantly how this framework will help you achieve the following simplification of repetitive tasks Automation of exploit kid detection. Well, I don't provide Solution for that it is for you to come and write different display filters And of course The framework itself helps in building self-contained and easy to manage self explanatory units slash constructs Yeah, and the rest goes along So what is SPF? Internet shell SPF is a shell framework that provides Substituted abstraction layer with seamless interaction for t-shark and windows command shell interpreter It features a New custom developed Language it's a declarative language called errors that I developed for this framework And in case you're wondering what errors sense for it's literally in reference to the Greek God of love procreation and sexual desire That is I just like Greek stuff a good history. So that's why I chose this name and of course you have Framework you have a shell and you have language, right? So how would you interact with the framework and the shell? Of course that is a set of built-in helper commands That allows you to interact with the language itself in a dynamic way so how this interaction was t-shark and CMD is accomplished This is a come through two unique constructs that are part of the language and I define them under the following names SPF is for t-shark and Win is for interacting with CMD. This will become clear later on in the slides I know now it's very abstract and These constructs constitute the knowledge base of the framework So the more you write of these constructs the better it is the framework in terms of what you can achieve with it so first we'll address the first construct that is SPF construct and This is the skeleton of the construct Okay, so first you have SPF Then you have the name you have to give it a name right and then followed by the logic definition The logic definition takes Display filter logic So you have to be good at writing t-shark display filters the better you are The better is the SPF contract Don't it's what's supposed to do and yet and you have to write is semantically and syntactically correct display filters because The language itself does not verify The semantics of the display filter that you write through the framework. It is being verified by t-shark itself and not the framework info The keyword info is literally you are giving this construct a help Statement that's all and for tag. This is when you want to search for different set of constructs in your framework Like in a non-hierarchical fashion So you just give it a type for example you give one one construct extra to be the author and what's supposed to do and The tag Keyword is optional. You don't have to have it in the construct whereas the others They have to be present and This is in a nutshell SPF construct. It's as simple as that and mind you each of these keywords They are accessible from the shell from SPF shell later on You can inspect each one of them and you can update the logic one at least dynamically Sorry in memory and therefore when construct it is the same as SPF construct. Oh, sorry I'm moving way ahead. So this is an example of what an SPF contract looks like, right? So I give it a name first one is get your eye, right? The logic of this construct is the display filter that tells it, okay check for all extra to be request methods that are get and print to the console the fields That is the request your eye. It is as simple as that, right? So I'm automating it To do this for me So I don't want to type it every time I use this particular display filter, right? And I'm giving it a help and attack It is as simple as that on the other side for the wind construct It's the same as SPF constructs except that at the evaluation and execution steps They take different Paths like this is internally speaking But it is the same as SPF construct and of course You cannot write T-shark display filters in these constructs because these gets executed by Windows CMD, right? Whereas SPF constructs are handled by T-shark and with wind constructs You can do whatever you want whatever you can run Via your CMD Anything here. I'm simply calling this WMIC command which gives me the US architecture As simple as that. Oh So we have these two constructs. Okay You can write as many constructs as you want, right? They are available from the shell But how can we generalize their use in such a way that when I call a given construct I want to be able to Input an argument that would influence the output of the construct, right? So I want to parameterize it to do that Here through the power of errors language We can use a set of input operators, right? so as Of this version The language supports couple of input operators that allows you to parameterize the logic of Your constructs So first I have the input operator arg Arg is simply CN if you're familiar with C language just a simple CN and takes whatever value so You can place this arg operator anywhere in the logic definition and when you invoke that construct The construct the shell itself is expecting you to give it a value That will be Put in place where this arg input operator is located in the logic definition This will become clear later on and now I think this is the most powerful input operator So the arg one takes only one value, right? It's always from the shell Now I need something more generic more powerful something that will execute different contents All at once for me. So I introduce this list operator It takes the counter of a text file so When you use this construct this operator in your construct, what's going to happen is the following First you give it file name before your data is stored on disk and every time There is an execution of that construct that references this list file It's gonna pull one entry from the from the file and execute it one at a time So it's gonna be executed in an iterative manner and Of course It is reasonable to say that they can exist only one instance of this operator per construct, right? Otherwise it doesn't make sense And furthermore you can generalize it. So the file name need not be hardcoded You can input it from the shell Should you choose to? Via the arg input operator. So that was a natural conclusion technically Of course So you have this operator, but the data and the text file that you're referencing Every time you get a hit on any entry in The shell in the output you need to know which one it is hitting on right so to do that you can Assign to every entry in the text file this particular print operator With a specific message Okay, so every time You get a hit on the data part The the message it's gonna get printed in the shell and gives you the output it is as simple as that So the data part contains the data part that will be put in the logic definition of where the list operator is referenced As we'll see later on so what Can you do with list operator? What possibilities it's open for you simply you can use it you can store list of suspicious or anomalous me user agents Because you don't want to in your construct you go on a hard code and the entire list in the logic definition and you can Have a list of malicious slash suspicious SSLTLS certificates if you know malware that's communicating with a bad side And of course in case you're wondering Recent versions of t-shark they allow you to parse the entire certificate every field previous versions You aren't allowed to do that at all. You would have to extract it manually and then do all the stuff and of course grouping so if you have a malware family and each variant of that family has Different communication protocol or they differ slightly So of course why not group them in one list file instead of creating a construct for each variant? okay, so now we know About SPF and win construct and we have a list of input of operators to interact with these constructs from the shell Or at least to generalize their use Now how can we even? Make it better if you want to build One construct on top of the other right just like in a any functional language, right? You call functions But this is declarable language, right? So I need to introduce more operators This following operator is called the call operator So you have a construct right that performs specific function, right and you have another construct But That construct the new one Inherits some of the logic in the other construct. So you don't want to write it again So I started through writing it You just call it right as simple as that and this is done through the call operator Call and you give it the SPF construct that you already wrote and the same it goes with the win construct and mind you from SPF You can call win constructs, but not the other way around why is that? Because when construct gets executed by Windows CMD whereas SPF constructs again gets executed by T shark So from execution perspective They are handled differently So this is the call operator So and again to point it out Call operators contain a fully functional They reference sorry they reference fully functional constructs. They are meant to reference fully functional construct That can be executed on their own not as macros Okay, what if You want to have just macros that cannot be executed on their own But just for documentation clarity Invisibility in your code you might need something else other than call operators for that. I Also have what's known as global auxiliary logic definitions So these definitions they are global with the universal lexical scope They are none executable named statements That can be used of course with SPF and win constructs for as building blocks. So again You cannot Execute these logical definitions on their own they meant to be referenced By other constructs. They are just macros literally as simple as that and this is the syntax for a Logic definition for example, if I it always has to start with an L capital letter followed by dot followed by the name of the Auxiliary logic followed by the logic as Simple as that so how would you reference this? Auxiliary logic in your construct. How would you call it? Right? So I need another operator for that for that. I use the insert operator Just like in your SQL statements or whatever other declarative language So to reference a given auxiliary or logic definition we use the insert operator So whenever SPF parser Finds an instance of this insert operator statement and the logic definition It's gonna be replaced of whatever it is defined Okay, so Let us say you have a set of constructs. You have ten Right that you execute one after the other always, right? So you go to the shell Let me call get your eye construct fall by another one another one another one, right? And you keep doing this again and again and again. So you want to make this process, right? So how would you do that? You do it through? What's known as multi-command unit? So this allows you to group The calling of different constructs In one I'd say like in one statement and they will get executed on your behalf one after the other It is a just a matter of grouping them and of course since The execution happening through Windows CMD you can use whatever Windows CMD provides and All of these constructs again. They are Accessible from the shell as we will see later on So here's here's an example of an MCU unit So for example, I give it the name test Right, and here I'm printing to the console this message the echo message and then I'm calling the construct the SPF construct get you uri with the argument get and Printing another message to the console and calling the win construct arc So I go to the shell SPF shell and I execute this MCU unit under the name test and It's gonna call all these It's gonna do all these things for me It's just a way of automating your task. That's all that's to it. I guess we're all and also to generalize your Right up off constructs if you want to group them in different translation units by that I mean different files like one for malware one for vulnerabilities one for suspicious user agents or whatever You can also use the include preprocessor directive right Just like in see right now the language You are the liberty of using this directive anywhere you want and there is no restriction at all You can use it in an acid way or whatever. It's fully supported. Okay Now from the shell You need A way to interact with these constructs, right? How would you do that? That has to be a set of commands Implemented in the shell in the SPF shell in particular and these shells. They are known as helpers Not help so you can just go type help in the shell and expect to get a list of all these commands because if you type help it's gonna be it's not part of SPF commands is gonna be handled by Windows CMD and then you're gonna get the list of help by supported by CMD. So that's why I'm making it helper and These helper functions as a concrete interact with define SPF CMD and the shell itself in a dynamic way So before going any further into the constructs, there are two important files in the framework. We have the configuration file and the main SPF command translation unit, but first we'll start talking about the configuration file and The list of options that contain and how it can influence the execution of the framework Depending on whatever value it gives you give each of the options. So this is the list of Options in the configuration files and the first one is we have SPF command file path So you literally just give it a path to your main command file where you have your construct stored We'll see later on why this option even exists And then we have the path To your t-shirt executable and then we have a path That you can set for all your pick up, right? So this will be like the default path all pickups. So you don't have to set it every time you Want to work with a given a pick up from the shell? It's already set for you and In case you have a default pick up name that you always work with or every time you download a pick up It gets saved under this specific name. You can also give it a default pick up name Now the option load command SPF file So this one Allows you to load and parse Everything stored in all of your construct that are stored in the CMD file at runtime So at the time when you execute the shell the framework itself It's gonna load all the constructs in that file parses it validated and makes it available for you At runtime so you can turn it on or off if you turn it off you're gonna have to do it from the shell through another command and then the history This also takes a boolean value Well here first the first option is this is where you want to start all your Historically executed commands you can sort them on disk the load history file This one allows you to either to load it around time or not so So how would you use the shell that so since we've introduced all these constructs and then for operators and language and everything So how would you use it? Like what would be a typical workflow when you get a pick up for that? We have the following workflow, which is facilitated by the helper commands so first When you execute the shell the SPF shell You have to Give it the name of your pick up that you want to work with right and to do that just set Beacop or get pick up. It's just a matter of off inspecting different variables the get pick up path is Does not change the value in the configuration file only in memory Sorry the the set pick up path the set pick up get pick up. I think they are self-explanatory except When you want to Execute a given construct against multiple pickups, right? So you're not gonna execute the same construct against every pick up one by one, right? so for that you can set the pick up name to Start AF which stands for all files fold by another star then every time you execute a construct It's gonna go check every pick up in that directory and this is where What I'm talking about regarding the command file where you can either Part it and validate it at runtime or from the shell so why this is important because You're already inside the shell and you want to update some of the constructs, right? So you're not gonna close your shell and then execute it again just to get everything validated again So but just by typing load command file it's gonna Redo the same thing again for you Get all commands is literally a response of showing you all the key the constructs you defined and You have lists of other The commands that allows you to inspect every keyword every keyword of the construct the name the logic the list of tax supported and other things and Also, you have commands for inspecting the history of all commands you typed and you can even execute multiple historical commands all at once through the helper command EXH and If you still remember what is an MCU is the multi-command unit This is how you would call a given one through the command EX MCU and get MCU list Gets you a list of all defined MCU. I Know this is all abstract now Unfortunately, is there isn't an easier way to Mixed accessible Until I show you like a demo Okay, so for SPF construct What did you know how it looks like? We know how it works. We know its functionality Let us say I define an SPF construct But I have no reason at all to show it in the shell So when I say when I type get all commands, which gonna give me the list the list of all construct that I define in the in in my file in my CMD file I Want some of the construct not to show up? Right because they either have no functional interpretation Or they are of no use for me. So for that there is another Thing I call specifier height So the specifier height allows you literally to hide this construct from the shell You can still call it reference and do whatever you want you with but whenever you call get all commands It won't show up in the list. It is just as simple as that and this is for clarity for management purposes Okay, so now we'll start with examples which should make it easier to understand So for example this SPF construct is supposed to Detect some parents of a given exploit kits right and in this particular example we detect If from what I recall Which what it was exploit get Let me say I forgot actually But we'll see later on Okay, so in the logic definition. We have the display filter Actually to be a request of your eye. So here I'm inspecting what the uri and Against what? against the list of entries Defined in the text file e k regex, right? Because I'm gonna have multiple regexes So I'm gonna I'm not gonna store every regex Here in the logic definition This is not manageable So what's in that text file? This is what it contains. So in this case Here I have the following entry in the text file first. I have the print statement if you still remember Which can a print to the console the message We have a match against Angler your eye test pattern okay Every time I get a match Against the following regex. So this regex will match against Angler exploit kit URI pattern, okay So let us say you have a nuclear exploit kit your right pattern, right? You simply add another answer to the text file you give the message First you use the print statement you give it a message nuclear your itas pattern and then you give it the regex pattern Against that particular exploit kits, right? And when you invoke This construct e k d is gonna test each one consecutively in sequence, right it is as simple as that and here's another example I'm not sure how many of you come from a corporate environment And you all know that You can just set your DNS server to use a public DNS server Right, you can just go and use a Google right usually you have your own DNS for water, right? so malware What did they usually do it? When they send a DNS request they use Google DNS server instead of going through your network DNS server This does like one of the evasion techniques they use, right? So let's say you have a set of peak apps and you want to check every DNS request where the DNS Server is any of those in this list DNS server of text file So I grabbed that list from this URL and the guy seems to keep it up to date actually Like as of August he has updated it which is interesting So the logic definition here is clear. I'm checking destination host Right, it has to match either of those in this list and the protocol is DNS and and to the console Print to me the frame number that matches this DNS entry the source yep source IP destination IP and the DNS name the actual domain name and for example, this is what the DNS server the text file would contain right for Google DNS server I have these two entries, of course it contains other also more examples. I have this example called if a Merrill port less than 1024 I'm not sure how much you're familiar with the RFC but the source port For your connection whatever you make a connection is absolutely randomly Assigned or generated I should say right and from zero to ten twenty three. They are reserved right Why I'm talking about this in first place Because I did a small experiment where okay, I have like about I had about 400 malicious speak apps and I wanted to see if If any in if in any of those peak apps, I have the source port That is less than 1024 Right, because it's not RFC compliant So for me to automate this process it is as simple as using this display filter Right by writing this construct and here what I'm checking for just the TCP source port has to be a little 1024 and This check is only valid when I make Sync connection, but it because it can be any syndrome quest, right? Syndrome quest can be sent outside of the TCP handshake So it has to be through the connection And of course the logic also can be done with the other option that I'm listing in the in the comment So you can also use comment with your construct But it's just the other way around That's as simple as that and to my surprise I found actually malware that The source port is less than 1024 and why turns out some of the malware family especially the the ones that Perform denial of service attack Especially when they want to perform some of the sin flooding right they construct the packet byte-by-byte, right? And these people usually don't care about the specification This is one way of identifying it and you have another example where I found out they had I had peak apps where Some of them are generated by fake net are you familiar with fake net so fake net is literally just to Simulate some of your traffic for example if you're making get your quest So instead of going to the actual server just replies with whatever is provided by fake net for you So it doesn't reach the internet at all, right? So for fake net Also, it has such anomaly It's its own peak apps. It doesn't even have The physical the frame header as well, right so he also technically it doesn't Adhere to the specification another example to give you Some of the usage of the of the framework And some of the operators Have you ever heard of the black nurse? Do you know the service attack? You've heard of it. Okay, awesome so This one is literally as simple as sending I see a pure quest with type 3 and code 3 This is all it needs to succeed. However, the limitation is You would have to send it at a rate of 40 to 50,000 per second And it affects hardware devices from firewalls to routers Let's say have big app. I don't want to check for this Particular attack, right? So, how would you do it? It's as simple as writing this construct Right, so here first. I'm creating this first two macro macros and Auxillator logic definitions, if you still remember Where I'm giving For the destination on reachable one the value of three and the port unreachable the value of three This is just for adding clarity to your construct, right? Otherwise. I would have used just these Numbers literally the numbers right in the display filters. There is no macros for these numbers, right? So here I'm just adding clarity to the construct and in the logic definition if you see here I'm not using the white operator So this is the difference between this construct and the other constructs Here I'm using the Q and Z Options provided by T-Shark for me, which allows me to inspect some statistical information about the packet Okay, so here what I'm checking for all ICMP of type 3 and code 3 and Counting the number of frames that matches this Display filter logic or this filters logic. I shouldn't say this filter this filters logic, okay, and For what interval? the interval is to be given through the input the arg input operator Okay, so when I go to the shell I type black nurse So all it's expecting from me is the interval value, right? For that I give it give me the list of All frames or not. We're not the list of all frames the number of frames that match this specification This filter in one second, right? Because as I mentioned for this attack to succeed you would need to send the same Packet ICMP packet at a rate of 40 to 50 thousand right so this is a way for you to test it It's as simple as that and here I'm using of course the answer of the insert operator to reference The auxiliary logic or the macro that I defined outside of the construct as simple as that now for collaboration As I mentioned at the beginning I Said that you can use the include Pre-processing directive and this gives you a lot of power actually so the scenario I envision for collaboration is as follows first You store the cmd file the master cmd file on a network share right You start on a network share and in the configuration file Where you define the path to that cmd file? You point it to that cmd file where you placed on the network share, right and from there in that file You use the included pre-processing directive pointing back to your directory to where you have SPF stored the actual framework, right? and everyone else on the network can use the included pre-processing directive in that master file pointing back to his own directory and then once you have that entry pointing back to your Path your local path then you can use The direct the include directive again pointing to other cmd files local to your own directory I know it's a bit complicated But it is literally simple one wants to start using it and then once you define a construct It becomes available to everyone else right so this is a way of sharing it and democratizing the sharing of these construct It's really that simple So I don't need to implement any fancy stuff in terms of networking or anything like that Of course, it's not over the internet. It's just local to your network, but at least you can Share its usage and mine you I know this might sound like a bit complicated here There's good documentation on how to use the framework. So you need not worry about The absence of any description for example for the slide a bit of information about the internals of the framework so for example the SPF Constructs they are stored under separate data structure. The same thing goes for the wind construct as well So I make a differentiation between external and internal so external is what you define Whatever construct you define in the cmd file and internal is what I define in the framework at the language level like At the C++ level as as part of the code and what why is that because sometimes There are certain constructs that you cannot implement just solely based on these are display filters You cannot achieve that whatever it is that you're trying to achieve. So at the code level I have the privilege to write those powerful construct and make them available for you and Things to note and reporting for these constructs like whenever you try to validate them from the shell Is limited So however to verify that your constructs working as expected When you do the get all commands and it shows up in the list of all commands supported then It is guaranteed that it's gonna work like this is the ultimate conclusion that's gonna work Okay, and other things that you need to be aware of the order of evaluation of all these input operators It matters a lot again all of these are listed in the documentation file. Okay in details This is for your own reference in case you're wondering. Oh, why my construct is not working, right? So you go to this particular paragraph Performance, I mean this goes without saying the performance is dependent on t-shock Why because I'm not implementing the dissection for these pro calls in my framework I'm leveraging the power of t-shock to do this work for me, right? So every time I call a construct it goes through t-shock. It's not natively implemented and there is a drawback for that, of course Future work This is what I'm planning to implement a DGA based detection algorithm That is not to be done through an external construct, but an internal construct and make it available for you as an disposable object Detecting different vulnerabilities that you cannot even do with Your typical IDS device whether it is snort or any other fancy ideas you're using And open sourcing the framework as I mentioned this is a personal project I was exploring different features of C++ So I would be literally demanding the same function, but in a different way just explore the new feature of C++ And that's the reason why I haven't outsourced the framework as of yet So I need to refactor the code and do a lot of stuff for that And I'm actually planning to implement natively some of the protocol and in particular HTTP so that I have a greater control over it instead of just invoking those display filters via t-shark and For you to know All your wire shark fanatics It is buggy I'm not sure how much experience you have with wire shark Especially the dissectors and the parsers these are different things. They are buggy. They miss some of the fields Even some of the display filters don't even work depending on how you capture your pick up file And of course I'm working on this This feature that I I'm already working on it and But I haven't finished the implementation yet I'm not gonna talk about it. It's gonna take at least a minimum of three minutes It's not that easy to implement as well. There's a lot of thinking behind it I was hoping to finish it before the conference, but it seems not gonna happen and Framework availability is Can go to my son and I'll load it There's I fixed some bugs, but I haven't updated the Version that's already on my site demo, unfortunately when I have time And for summary, yeah talking about the framework We introduced a new language. I showed some of the case studies and references If you want to learn about display filters wire shark filters these are the different things except the Semantics and t-shark credits just the icon I use at the beginning of the slides the first slide I got it from here just For real reasons I need to mention this and thank you all for attending my talk