 Welcome everyone. So our next speaker is Diego with SNAP, a toolkit for user space networking. Hi. Hello, everybody. Hi, thank you very much. Thanks for attending this talk. On this talk, the title is SNAP. I'm going to talk about this toolkit for programming users, networking functions in user space. It's called SNAP. Basically what I would like to achieve with this talk is three things. Introduce you to SNAP, explain a little bit how it works, and what kind, show you what kind of things you can deal with SNAP. So let's go for it. What is SNAP? SNAP is a toolkit for developing network functions in user space, and it's mostly aimed for high-performance networking. And I need to clarify two concepts here in this definition. One is what is a network function, and another one is what is high-performance networking. So a network function is basically a program, any program that manipulates network traffic. These programs, usually what they do is a series of basic operations such as read a packet, forward it, drop it, modify its headers, or create new packets. And combining these primitives, you can build any network function you can think of. For example, a firewall essentially is an application that gets incoming packets compared to a table of rules and executes an action depending on the result, either forward the packet or drop it. A network address later gets incoming packets, modify these headers, source IP, destination IP, recalculates checks and forward the packet. And it's an alien application, gets incoming packets, creates new ones, embeds the old one into the new one, and send it through the network. So these are the types of applications that you can do with Snap. And what is about high-performance networking? So the networking field has changed quite a lot in the last five or six years. What has happened is that the hardware has become much better and the prices have dropped. So nowadays you can get a 10G NIC for 100, 200 bucks depending on the vendor. So NICs of 25G, 4G are affordable. Still the appliances, the equipment that you find in data centers are still very expensive, these high-end routers by Cisco. So people start to have this idea of maybe I can build a piece of hardware that is equivalent to these high-end routers using commodity off-the-shell pieces. Maybe you can build a piece of hardware that is equivalent but you still need to put some software in that hardware. So what would you put? On the natural choice it feels like it will be Linux because Linux will conquer the world, it will put Linux everywhere. But the point is like Linux doesn't fit well in this scenario because the way Linux works is a general-purpose operating system and whenever a packet hits the NIC there's going to be an interrupt and then the kernel has to handle this packet, it goes through all the networking layer of the kernel until it reaches user space where we are running a network function. What happens is a network function is split into user space LAN and kernel LAN. There is a cost of this packet going through the networking layer of the kernel that at this speed is significant. For example, if we are using 10G NIC dealing with packets of 550 bytes, so every 440 nanoseconds a new packet is hitting the NIC. If we want to process all these packets with one single core at 2.5 GHz that means we have 1500 cycles per packet for a network function. That's our budget. And this gets worse if the packet is smaller. In the smallest possible packet of 64 bytes we are getting a new packet every 51 nanoseconds. If operations that happen in the kernel such local logs or an L2 Cosmis has a cost that it will eat almost the whole budget. And there is a great article of a presentation by Jesper Brouet who is there where he explains all these things. So people find out this problem in the kernel and they start to think of solutions and there are many solutions to this problem. This in the kernel has improved also it's worth mentioning that. But one of these solutions was to do a kernel bypass. And a kernel bypass means you tell the kernel hey don't worry about the NIC leave it to me I know how to handle and then you program driver in user space. Which is quite hard but that's how you do it. Luke Gorey the originator of SNAP and my colleague Deliver a talk about how you can write driver in user space yesterday. And there are other two kits that follow a similar approach. There is a toolkit very well known called BPDK by Intel. There is a collection of drivers in user space by different vendors. And there is also BPP now renamed to FDIO by Cisco it's a project on the Linux foundation and it follows philosophy very similar to SNAP. So let's go with SNAP. SNAP was a project that started in 2012 by Luke Gorey. It's mostly developed in Lua and SNAP means fast in Swedish. Fast is something that characterizes SNAP because SNAP is fast because that's a kernel bypass. It is fast because it's written in Lua and network functions are written in Lua so that's faster than coding in C it's very easy with SNAP to go from idea to code to production. And it's also fast because it runs on top of LuaJit. LuaJit is a just-in-time compiler for Lua. It's a extremely fast virtual machine it's regarded as one of the fastest virtual machines out there and in addition has very good integration with C thanks to foreign fusion interface that is only available in LuaJit. It's not implemented by a standard Lua. And in SNAP most of the data types and data structures are actually defined in C. How it works in SNAP a network function is essentially a collection of apps connected together via links and once we have this graph of apps we pass into the SNAP engine that is going to run it in units called breads. So here's an example of a very basic program in SNAP what we do is that we start the configuration then we instantiate in this block the apps we need we need a NIC this is the driver that's going to read packets from the NIC and then we have a filtering app that is going to filter packets with source port 80 and then we have a pickup app where we're going to write down packets to an output file. Then on the next block we connect these apps together whatever comes out from the NIC is going to fit the filtering app and whatever comes out from the filtering app is going to fit the pickup app. We pass down this configuration to the engine and we run it this is how it will look like as a graph whatever comes out from the NIC it goes to the filtering app the filtering app does something with the packets and whatever comes out is passed to the pickup that eventually writes down the packets to a file so I said that the SNAP engine process packets in breads and a breads has two steps it has an inhaler step that is going to put packets a batch of packets actually into the graph and it's hellish step that is going to process those packets. On the inhaler step the method pool of the apps is executed it feels fine and on the exhale step the method push of the apps is executed this is how the pool method of the internal driver looks like well it leverages on another class called def but what it does it loops and what can't receive packets is going to receive a packet and put it in the output link push method of the filtering app we showed before looks like what it's going to do is going to read the input the incoming link and whether packets receive them and if the packet matches the expression the filtering expression is going to transmit it to the output link and if not it drops it as a summary in SNAP there are two things to keep in mind is that usually there's only one app that is going to introduce packets into the graph I call this app the originator app this will be apps such as driver that reach packets from the nick or for example a pickup reader that reach packets from a pickup file or an application that builds packets on the flight and then all the other apps are going to implement a push method because the push method is what it gives the apps the opportunity to do something with a packet this is how a packet looks like in SNAP it's a very simple data structure it's on a ride of bytes 10k actually with a length field and here's how a link looks like a link is a buffer of packets 1000 and it's actually a ring buffer it has two pointers, a read pointer to read packets from and a write pointer to write packets to anything that has to do with packets you can build it with SNAP but when you start with SNAP you don't have to build everything from scratch because SNAP already gives you a bus catalog of things that are already built into the toolkit for example libraries you have libraries for parsing protocols IPv4, IPv6, TCP calculating checksums, you have apps, drivers filtering apps, load generators, sockets it's mostly aimed for high performance networking but still you can use the interfaces by the operating system unique socket, raw socket, top and it already has full-fledged programs L2VPN, a list per lightweight after network function IPv6, on the next slide I'm going to go through some of these applications that we built and I will show you the things that we did for example we built a lightweight after network function lightweight after is a network function it's part of a standard call lightweight for over 6 this is an architecture for deploying IPv6-only networks and still offering IPv4 connectivity and services it was a project mostly founded by Doge Telecom because they are deploying this architecture in their next generation networks called Terastream a few months ago I wrote down a blog post about how lightweight after, lightweight for over 6 actually works if you are interested, please check it out another application we built was Snapwall it's a L7 firewall and also that's L3 and L4 for the L7 firewalling we use a library called Live NTPI it's an open source library and for the L3 and L4 filtering we developed our own domain-specific language called PFLAMP, it was a project founded by NetFoundation, this is a foundation based in the Netherlands that founds open source projects that helps improving the internet actually it's a chef delivered a talk yesterday a keynote called the future of the internet and all the information on the development process of Snapwall is at this website, snapwall.org it was a project developed by my colleague Asumu and Adrian IPv6 stands for IP flow and flow information export is a format for exporting network flows it's based on Cisco's net flow I think it was also a project developed by my colleague Asumu and he wrote down a blog post explaining how the tool was built and how it works and lastly this is something that I built recently it's a DNS service discovery I developed this as a tutorial to show how you can use Snap for building programs using top interfaces and what it does is it helps you to discover multicast devices in a local network such as Croncast or Amazon's Fire TV etc and I wrote down a blog post about how all the development process from an idea to code so how to get it started all the source code is at the GitHub the community also has around in a Slack channel there's also an official getting started guide but maybe it's a little bit outdated a few months ago I wrote down a blog post called Snap explaining it in less than 10 minutes which is essentially this talk in a blog format that was all from my site I hope you like it if there are any questions I'm glad to answer thank you very much thank you for the talk Diego we do have some time for questions thank you for your presentation I like the idea that Lua is used for programming and projects that also have programming capability open with speech users like low level not state machine but like a Turing machine with a set of registers so have you tried to do some switching L2, L3 switches with Snap okay I think the question is we have tried to build a switch with Snap I think you could build it, I think actually there is a learning bridge that means there is a tool that can learn the mapping between MAC address and port but I think I don't know if it's part of the repository or it was never merged but there was some attempts to build this tool a switch hello I read in your GitHub that you have some support for OpenStack what's the status of this support so you're providing amount to drivers for OpenStack somehow the question that there is support for OpenStack with Snap I'm not very familiar with but what I know is that Luke Gorey some years ago I think it was in 2015 he had to implement Snap, the interfaces of the neutron layer of OpenStack so you could plug Snap into the neutron layer of OpenStack does it make sense because I'm not very familiar with OpenStack so are you using OBS OpenVirtual Switch or something like that or you are just programming this Layer 2 functionalities directly with Snap I don't know I know they're familiar I know that he had to implement that because it was a requirement it was a long process OpenStack is like that my last question, sorry can you implement using Snap higher functions like filtering or load balancers or something like that, thanks load balancers no, what was the other one filtering? ah, firewall ok, the question was if we have tried if we have used Snap for building high level applications such as firewall or load balancer load balancer no on a firewall, the one that I mentioned is a firewall even for L7 for application level and also that's L3 and L4 there's a lot of time now so if you have any more questions you can take it off stage please thanks once again, thank you very much