 So next up is Quentin Monet, who will be presenting Stateful Packet Processing and Open State Implementation with EBPF. Quentin? Okay, thank you. Thanks for coming. I'm Quentin and I work at 16 and I've come to take about the Open State Interface for Stateful Packet Processing and about the possibility to implement it with EBPF among other solutions. So the initial plan was to talk a lot about EBPF and to present it. And at some point, I somewhat changed my mind. So it will be a bit less about EBPF and more about Open State. So just before we get to the technical details, we are working in software-defined networks. So we have physical hosts who have virtual machines, who have programmable switches and controllers. So controllers are used to configure the switches in a centralized way. We have the data path. We have two paths for data. This is kind of a shortcut when all packets are processed by the programmable switch. And for packets that require no processing, that's more processing operations. They are usually raised as exceptions and sent to the controller. So this is the case for every packet belonging to Stateful Flows. By Stateful, I mean Flows for which the forwarding decision will depend on the traffic that has been previously except the switch. So the idea behind this talk is what if we could take as many packets as we can from this standard path and move them to this faster path, to this shorter path. So can we find a way to make the switch smart enough to do that? We are keeping this centralized way to configure everything. So this is also the idea behind the BBAP project, which is a European research project that I have started in January 2015. It's expected to close next month, actually, so we have learned of the project. The idea is to get very speed reactive control and processing tests inside the switches. We are keeping this centralized control while being scalable, but everyone wants to be scalable, so it's nothing new. And we want also to be a platform independent, so not to rely on any specific hardware or vendor solution. So as for the partners of the project, it's a research project. So we have business companies, we have academics partners on the right side. And all of those partners have designed what we call the BBAP architecture or BBAP switch. And it's made of three components. There is OpenState for stateful processing. We'll talk about it just as a wrap. There is something for in-switch package generation, but we will not see it in the presentation. There's also OpenPacketProcessor, which is an extension based on the two other components. And we see a little at the end of the presentation. So OpenState is for stateful packet processing. And the concept is to forward packets depending on what we have seen already. So what we need is to keep states for the traffic. So when a new packet arrives, we need to perform a lookup for the state the flow it belongs to is in. And then we need the second lookup somewhere to find what action we must do for the state. And then the first step of the process is to update the state of the flow to a new value that we need to obtain from the second lookup as well. So for this we need two tables, the state tables, the first one on the left, and the XFSN table. It stands for extended finite state machine, which is the mathematical model implemented as this process. So here's a case study for seeing how it works. Concretely, we try to do partner key. So usually partner key consists in a server telling to everyone that its port, for example SSH port 22 is closed. And only when a client sends a correct, secret sequence of packets, then the server says, OK, it's open for you. You can come back to this port. So it's a security measure. So to implement this sequence, for example, in my case, packets and UDP port 111, 222, and so on until four. To implement this, we need to register somewhere the state wire to remember the previous packets from this sequence. So this is the diagram, the state machine for this example. We start in the initial state here. I want to put it closer. When we receive the first packet for this sequence, we move to state 1, 2, 3, and then we open the port to the SSH port. And if we find the packet in between, that's not part of the sequence. We go back to the initial state. So this is the state table for this example. At the beginning of the program, it's nearly empty. We just have a rule that says that for all flows, we are in the different states, so that's the initial state here. So when the packet arrives, we get state different initial for this packet, and then we perform the second loop in the XFSM table. So in this table, depending on the packet we receive, we will match on a state and on an event. For example, the event is UDP destination port 1111. In this case, if we have the different state and the correct port, we will return an action, which is drop the packet. We don't want to forward this packet. It just in the switch, we drop it. Then we change the server. And we move to the next state, which is state 1. If the packet is not part of the sequence, in other words, it's not the correct part or it's not a state we know. We just have a different rule at the end that re-initiates the state of the packets. So we get this value. We perform the section of the packet, and we update the state table with new states we get for this flow. So after the update operation, for example, if the packet was part of the sequence, we have new data for this flow as a new state we found. So that's quite simple. So we have a variety of implementations for this in the project and the questions we try to answer for my part. Let's do this with ABPF. So ABPF, I will give you the details again. We have had two good presentations already thanks to Daniel and Thomas. So here is again the same diagram that you saw with the CBN presentation. So I'm using TC hooks to attach program inside the kernel. I can compile my program from a C subset of code. And I can also communicate with the user programs through maps, and that will be very important because by default, so the ABPF program, when it runs it processes the packet and then once it's finished, it is everything about its state. It's discarded and then when the next packet arrives, we start a new with nothing. So we have maps and as Thomas said already, we can keep states with these maps because they are persistent between different instances of the program. They can also be shared amongst multiple programs. I don't care, but they can also be shared with user space and this will enable us to initialize the maps. So let's use the hash maps for the open state tables. So the code corresponding to this is in the GitHub if you want to have a look at these one details. So I use the first file which is my header file, openstate.h in which I have the key and value for the state tables, the key and value structures. I'm using ether type, IP source, IP destination and my state table will return a value for the current state of this flow. I also have the XFSM table below so I will match again properties I want for the event and also the state I got from the first table. And then this second table will return both an action to perform on the packet and the next state to edit the first table. That's the file for a program. So a part of this file. Obviously I skip everything about header processing. So I just jump to the state table lookup. So I have to set a key and then to perform my lookup with one of the ABPF helper functions from the cannon that make it able to accept the other maps. If I get a return value from these maps then I'll go to the XFSM lookup otherwise but I should not reach this point. I go to the NF program and returns some kind of error code. This is the next part. I have to perform the second lookup in the XFSM table so I set up again a key. I perform my lookup and depending on the value if I get something I will first update the state in the state table for this flow. So I will use the next state I got from the XFSM table to perform the update. And after that I can just switch on the action I got from the XFSM table. So action drop then I save the code for TC action shots for TC that he has to shut down the packet and otherwise I can forward it. So this is nearly all for the code so we can compile it with Clang. That's the command line I use. We can attach it to a QDisk as a BPF fighter and the only thing we have to do after that is to initialize the maps so I have not provided the code in the slides. We can use a user space program that relies on BPF system calls to fill the maps from user space. Also there are tools known as BCC tools, BPF compiler that use Python Wackers to do it in a probably easier way but it's not necessary so it can be both. Okay. So I don't have time for a demo but this is just a screenshot of what I got for part-making. It's not really impressive. So I send packets from port 22, then the correct sequence then again packets from port 22 and only the last packets P7, PAP9 pass through the part-making application. So after part-making I just have another second use case which is a token packet which is maybe more interesting from the industry by the view of PAP9. The thing is with this open state interface I cannot exactly implement a classical token packet with tokens and each time a packet arrives a token is consumed and when we have no more token we cannot forward a packet. It's a wait-limiter use case. So I cannot do this as a bucket with a counter for tokens. I have to find another way to implement it because of the operation for open state. So using the equivalent algorithm is a time in a day that shifts to the right one slot at a time. And I have three cases. So if the packet arrives between the time boundaries of this window it's a clear, it's not a traffic. I can forward this packet. And then I move the window from one slot to the right. If the packet arrives to make for the window it means that we have not seen a lot of packets during the last period because we have not shifted the window much so it's okay we can forward the packet. And we initialize the position of the window to the current position of the packet. And the last case is in case of heavy loads. When there are many packets arriving in the same type slot then we have shifted the window too much on the right so there is no token left for this packet so we drop the packet and do not move the window. So there are no things on the right-wing side we have to use the extension of open state. It's called open packet processor so it's an extension so we will use everything from open state plus global and perfluent registers which are like variables. And these registers are evaluated with a set of conditions. These conditions, the result of these conditions will be used for the match in the XFSM lookup. So for token bucket we have two registers T-min and T-max that correspond to these values here and there that define the boundaries of the time window at every moment. So we can have conditions with these variables and we can evaluate these conditions and depending on the results through a force of these conditions we will match or not in the XFSM table. So this is the full diagram of OPP architecture without entering too much into the details. First we process the packet to extract any information we need too much and we perform the first lookup in the state table. So on this packet metadata we get the state for the flow and also a set of registers. With those registers and possibly global registers we evaluate the conditions that we want to use and then the second lookup is done on the result of these conditions. So state we are in and the packet events for example is the UDP port for networking and it returns as before the next state for the flow the action to perform as a packet and also an update function that will be used to update per flow and global registers. So on the eBPF side we have nearly every few minutes already to add those registers, those conditions. Also for the token packet we need the arrival time as a packet there is an internal helper for this so it's quite useful. And the conditions can be defined in each program so for example in my program I first get the arrival time as a packet I perform my state lookup and then I evaluate conditions so I've implemented a wrapper to do so to check whether the current time is good or equal to the routine, etc. And then I perform the second lookup on the XFSM table based both on the current state those return the conditions. For the tables it's the same we just have the registers in the tables. So again just a screenshot for this application I send 100 packets from the client and on the server I got the first six packets my token bucket had a capacity of five tokens so first packets come through then we have to wait each time that a new token is generated again so that the next packet may arrive all of the packets have dipped. So this is just a list we have some additional use case for OpenState and OPP for the project we use it to do quality of service to do failure detection and recovery and also there's an application I appreciate which is the denial of service detection and mitigation so I won't describe it here for the details I have some blog articles I want to read them on some of these use cases and to conclude I really think that EBPF makes a nice target for the blue bar architecture so both OpenState and OpenPacket Processor we have some limitations with these limitations for example we have no web carding mechanism yet in EBPF so if I want to say okay now I want too much packet with anything for the EBP destination port and I just want to focus on the source port currently it's a bit difficult to do with BPF there's also the program for concurrent accesses if I want to run my state-full application on several cores at a time the packet won't necessarily be processed to the maps in the same order as the live but otherwise it looks quite well so we have more implementations and this OpenState architecture for the blue bar project EBPF is just a small subset of what we've done so if you want to go and have a look the reference implementation is done of a virtual switch which is also a switch it's accelerated with the PFQ framework there are also several other implementations and acceleration prototypes I know that some partners are running OpenState on top of the remote driver for DPDK on FPGA so also everything is available on GitHub if you want to have a look at this so that's it, thank you if you want to see more details about some of these points including the deliverables of the project you can go and see them no problem, if you have questions I will be glad to answer them so what's the real performance depends on what implementation you want for EBPF I take it I've not finished the performance test yet but it uses EBPF facilities so it will be the same as EBPF with TC I think the features that are available for EBPF was for simple drop with TC EBPF TC it was something like four and something million packets per second kind of three that's seven gigahertz processor I think so it will not be as fast as this because we use maps, we use hash maps we have to hash values and of course it will be slower than this but still I think we can get pretty good performances I think I forgot the points for the next steps so next step include trying this with XDP hooks so as to obtain better performances hopefully any other question?