 So welcome to this session. It's called if up down to we all know what if up down is so we're expecting some enhancements here on building on the existing infrastructure and I'm happy to introduce to you now you're gonna say your name who comes to us from Camelus Camelus Networks and you're working on this yes because your buyer is paying you so please welcome and let's have a good session. Hello everybody I'm Rupa and I work for Camelus Networks so I think I'll just start with the talk. The outline a bit of background of who we are because that sets the context for why we went into these chain why did we upgrade I mean develop if up down to and enhancements to if up down to a little bit of light into network interface configuration on network switches and the challenges we faced a little bit about if up down introduce if up down to a bit into the features of if up down to next steps and questions after that. Background, Camelus Linux is a Debian based distribution for network switches. Our philosophy is manage your network switch as a server. We use the server distribution on it on a network switch we have made some enhancements to some of the packages and our philosophy is you use your existing tools to configure network switches so your network switch ports appear as nick ports on the server and we basically use the Debian server environment to boot up the switch and you know what manage network interfaces etc. So current release of Camelus is based on Debian VZ and we started with the if up down deploying if up down on our switches and we faced a few challenges and hence the if up down to. What kind of configuration we do on our switches usually there are large number of interfaces we have usually about 52 switch ports and we configure a ton of bridges every VLAN is a bridge and we have bonds and tons of VLAN devices and the VLAN devices scale with the number of VLAN so you can imagine we have thousands of VLANs and thousands of bridges and and large number of interface attributes as well we have a bunch of network protocols that run on our box because it's a network switch so and every interface every bridge interface we run STP, MSTP and so we have ton of attributes for STP IGMP attributes that we need to be configured on interfaces mostly static configuration unlike hypervisors or desktops this is yeah so the challenges we faced with existing network interface management tools this is mostly in context of if up down but in general we did a survey of other network interface management tools on Linux and we did find that they didn't suit very well for our needs so mostly most of the network interface managers are optimized for desktop and hypervisor environments mostly to environments where with dynamic interface configurations and we've seen that complexity increases with the interface configuration scale with large number of interfaces the interface configuration becomes very complex large number of files or the files get too large and so on in most cases the dependency of network interface configuration ordering is on the user user has to decide user has to declare as interface configuration in a way that bonds are bonds that are members of bridge ports have to be created before the before the bridge etc and lack of support for incremental changes to network interfaces this it was particularly in context of if up down if up down did not allow you to change an already existing interface or update configuration on an already existing interface you had to bring down the interface and then bring it back up and lack of tools to query and validate so you apply a configuration with if up down and then to query the applied state you had to go back and look at the configuration using native Linux tools for example a bridge here you could create a bridge using half up down but go back and look at it using BRCTL or IP route to etc this slide is basically just says the same things but it also tells it also shows what are the benefits of I have up down it has a pluggable architecture which is good you can drop a script in for any new configuration that you want uses native Linux tools enables faster development and you don't have to duplicate whatever your native Linux tool does into your network manager for example today IP route to is kind of becoming the only tool to manage all network interfaces or create or create network interfaces so we use IP route to a lot and we believe that in the future IP route to is going to have support for almost all types of interfaces and I have done as good user documentation it's a well-known tool and most of our previous customers first release customers were already on I have up down so and the challenges most of them I spoke on the last slide the other thing was development was also a challenge because I have up down was little written in a literate programming language and we did face a few bugs with I have up down I have up down to I have down to is basically re-implementation of I have up down in python which was python because it's easier and faster development and we could use existing modules and add functionality quicker you will see in the next few slides and it's backward compatible with I have up down with interfaces format and also with the names of commands the commands you'll still find the commands I have up I have down I have query but you'll find them with the newer options and you know there are a few missing minor missing functionality like mapping and maybe address other address families it only supports I knit and I knit 6 today but that is that will be trivial to add in the future if needed and it continues to use existing native Linux tools pluggable architecture I have up down used bash scripts this supports Python modules and meet some of the challenges that I described in previous slides packages I initially did it as two packages that can that could change the future but to keep things separate I have them as two packages right now so I have up down to contains the base infrastructure to parse schedule and order interface network network interface configuration I have up down to add-ons contains a bunch of Python modules add-on modules as you can see there is a module to configure addresses a module to configure DHCP and bonds and ETH tool and MSTP bridge etc this picture just shows the I have up down to package that's a infrastructure package sitting at the top I have up down to add-ons which are add-on Python modules for each kind of interface configuration and you can see these modules they spit out commands in native tools language to configure network interface life of an I face object so it's Python and I have up down used environment variables to pass on interface attributes to modules I have up down to uses a Python object so the on the far left you see the first block shows you an ETC network interfaces file section which is an I face object and there you can see the I face object translated which each attributes you can see some metadata I have up down to discovers dependence of that particular I face object and pass the I face object is passed on to the modules and the modules translate that to native Linux commands how does I have up down solve the network interface dependency order I have up down to the main infrastructure package queries individual modules for dependence for example a bridge port the bridge module is queried for bridge port dependence sorry bridge dependence which are bridge ports and it builds a dependency graph of all the interfaces in the ETC network interfaces file it sorts them and executes them in sorted order and it provides options to query and execute interface configuration dependency order there is an example in the next slide so you can see on the right there is a tip that's a typical graph typical configuration that we see not sure it's very clear but at the top there is a bridge and it has switch ports VLAN devices on switch ports as it's as the bridge ports and there is a bond interface and bond has slaves and that gets translated into the sorted interface list at the bottom built-in devices support this was specific not a requirement I would say it was because our files had too many VLAN devices and it was like thousands of VLAN devices and having all the VLAN devices getting listed in the ETC network interfaces file made it very large so built-in devices support in fine print it tells you the details it basically I've up down to can recognize VLAN devices and when they appear as dependents of a bridge and even physical interfaces and it picks up those devices as it brings the bridge out so this kind of reduced the interface size a lot so this is just an example of the previous slide options to IF query to query the dependents you care there is a big depends option there is a print dependency list or you can print the dependency list in dot format and can translate that dot into a visual format using any graphics tool incremental changes I have up down to queries running state of the interface before it applies the Delta config I have queries extended to with options like running and check to check the applied state of the to compare the applied state against the persistent user supplied config there's a new command IF reload which is similar to IF up minus a you can say or similar to service networking restart but it only executes up on interfaces that have changed and the example there actually shows that you can use IF query check option to check if a bridge interface a running state of a bridge interface matches the user supplied configuration in the net interfaces file and yeah the check basically returns with 0 or 1 depending on the running state here is an example of IF query check this on the left side there's a bridge with ports and you can say check tells you that the bridge is all fine and on the right side I remove a port from the bridge it indicates that there was there is a missing port and you can do an IF up an IF up will apply the Delta that is it will add the missing port to the bridge and the next IF query check command actually indicates that the bridge is fine templating this was again necessary to reduce the file size we in our deployments we have seen cookie cutter interface configurations for bridges especially so I've done two integrates an external templating agent called Mako and Mako was a choice because it was used in our other projects at that time but I have done two provides easy options to plug any other template engine so the right side there is an example of how we create hundred bridges by using Mako and Mako is nothing but you can encode Python language into your ETC network interfaces file and you can use IF query to render this template to see your expanded list of interfaces so this reduces the file size to quite an extent API it supports JSON both input and outputs in JSON format you can provide an ETC network interfaces as in JSON object format and the examples actually show IF query yeah there's a format option to print in JSON so all the all the commands by default take native input and output but here there is a format option to input and output in JSON format IF query to persist running config so since I had all the infrastructure to actually query the running state this was an easy addition to the tool what it what the example shows is you create a bridge using native commands and then use IF query running to actually translate that running state into ETC network interfaces format and the last command actually cats that to ETC network interfaces next steps I'm looking at pushing IF up down to into the Debian repo integrate IF up down to with system D and also work with network manager we want to be close to close to Debian native as possible and yeah so then some more bug fixes and some more compatibility options so getting started with IF up down to it's already on github and there is a lot of documentation there is developer documentation as well to add more modules to IF up down and there is a lot of cumulus Linux documentation for IF up down to we have a ton of example files as well and there is my email ID there and that's about it actually questions hi yeah so I'm relaying a question from IRC there's people watching this over the world and the question is if there's a reason not to replace the current IF up down with IF up down to oh there is we would love to actually and now the question is if there's a reason not to do it oh you mean yeah actually my first slide yeah my first slide did contain some challenges it does we could not actually well perhaps if I come if I can comment about the experience we've had with IF up down in Ubuntu there has been a delta for for for a large part for and for a long period of time because of things like network manager the managed true function that that use exists there and one of the issues is that if up down is kind of hard to maintain if you're trying to do stuff and and trying to change it so at least in a language that's more easily approachable by people is a really great thing yeah that was that would have been my exact answer it's in literate sorry the question is if there's a reason not to replace the old one with the new one so if there's something in the new one that you would say okay there's something that's not yet ready or some cases that might not work yeah like I said there are two or three things that are not supported it only supports INIT and INIT 6 today but if desktop environments need other families it could be added very easily but yeah those things if they're fixed I don't see any reason why it can't be maybe it's not exactly what you had in mind when asking the question or they had in mind when asking the question but I've heard a lot of talk that the system you guys plan to have network D you know issuing the no seriously issuing the syscalls directly for configuring interfaces rather than you know shelling out to all of these external commands like IP and that you know maybe it's better if we just burn down the entire IF down thing entirely and just move over to using that I wonder if you've been talking to the system D maintainers about that I have looked at system D right now it still has the same problems like I think you have to order your interfaces file but this can be fixed obviously but I wonder if you've talked to the system D maintainers and Debian about this because I think if your plan is to replace IF up down with this and their plan is to replace IF up down with network D there's going to be a problem coming sure we are talking to so I think I mean we clearly understand that this is a conversation that needs to happen and that's actually why we are here and part of the problem is that the simple use case in some sense of a system D style solution still for our use case is going to not be complete you still need a scriptable templatable engine that will go on top so where the the breakdown happens and how the interfacing happens is something that we obviously are interested in making sure we arrive at a common answer to we have no I think it's fairly safe to say we have no vested interest in making this be the solution but clearly as was noted earlier IF up down was not cutting it for us and this is our answer if there's a better answer that's okay that's great but the configurability and the templating is incredibly important because I mean on the desktop you talk about four interfaces maybe six VLANs it's an interesting answer or even on a server on a switch with 4,000 VLANs 16,000 VX LANs and God knows how many ports coming in the next few generations that becomes un-maintainable and unmanageable and the thing that Rupa keeps talking about the priority and ordering keeping that in in your head in five-page screenshots or five you know while it's scrolling by it's impossible it's it's guaranteed breakage and that's a solution we have to have. So a couple of things one one on the system D network D stuff my understanding that came up a little bit earlier in Debian developers been a little bit of discussion about that my understanding is is that that is at present in development stage that I would call science experiment there's something there they can bring up a network but it's not it certainly doesn't do everything that the existing IF up down does let alone all the things that you're in the enhanced one that you talked about today would do I think that that that the future of the system D network D stuff is probably more along the lines of if you have a system that it only is only going to have one static IP address with one static default route you might want to be just use system D network D because it's about the simplest thing you can possibly run and if you're in that situation why have any complexity but I don't think it has much desire to move higher in the stack I think most of the system D folks are using network manager to do anything that's particularly complicated so I don't know that they were targeting the stuff that you're targeting with it if up down to and the other thing is in terms of why not to replace the existing if up down with if up down to the most common complaint that I have heard on that front is the fact that it's written in Python and if up down is a required package so that would mean introducing at least some amount of Python into required which is something that a lot of people have objected to in the past there are folks who are already unhappy that we have pearl big chunks of pearl and required let alone adding a second interpreted language Ubuntu already did that I think so they've already yet so they crossed that bridge but Debian has not at least to date I have two questions actually I would like to know first of all if the if there's going to be complete compatibility to all the scripts the if up and if down hooks because we use them a lot in Debian packages and my second question is whether if up down to has any plans to support dynamic changes or because if up down doesn't do that but for instance if your DHCP gets a new you get a new address as happens here at PSU or if you go offline and so on whether you're planning to hook into the netlink layer and react to it yeah I think today we already who can do I have plug D and I've plugged he can do most of the dynamic configuration using if up down and your first question on if up down compatibility yes I think we will be able to do full compatibility I think it's very important yes yes will this be packaged in time for the freeze the Jesse freeze well if today it's packaged as dead and it's not in the Debian repo so first we'll have to get it in the Debian repo and yeah I think what he's saying is hurry up if you want it in okay okay thanks I have a question more about how cumulus uses this our users of your switches logging in and running these tools by hand or there or if you design this to allow there to be graphical or web-based front-ends to do things they're usually logging in and running it use the CLI but there's also orchestration tools that we integrate with like chef so there are a bunch of cumulus guys people sitting there so they will definitely be able to yeah I mean they're a bunch of different you know if you have one or two switches you're probably configuring this by hand by ss hing in and editing it if you have you know a thousand switches you're probably using either some homegrown orchestration tool or something like chef or puppet or answer or you know what what have you yeah there's as far as I know there's no web or graphical package I mean maybe Webman has a module but I'm not sure it's something that we I think that reaction says it all right there there's nothing stopping from anyone from writing and that a lot of the tools you know have configurable output formats you know defaults to human readable but you can get JSON formats or whatever so if you're writing a tool like that that would be a natural way to plug in so I think it's important I suspect if we are our current use cases definitely in high volume lots of ports lots of instances if you are in the desktop or you're trying to make it simple for a laptop which is where we wanted to even get to there will need to be a graphical tool I think that that part we understand and it's on the close yeah just don't make it wet men there's just something wrong about a network accessible network configuration tool couple questions what's the contributor license agreement what's the license and what's the community like for if up down to licensing questions to Nolan I've got a follow-up to there's no contributor licensing agreement you own your changes obviously we'll we're more than happy to accept patches under the we wouldn't we went GPL GPL me to and the community right now is essentially you're looking at it but we're hoping to expand that hoping to get you know more people interested involved and are you guys running this on anything but cumulus network switch gear running cumulus Linux OS like you talk about wanting this to replace if up down have you tried it servers we do run it on VMs which runs it yeah I mean we run in VMs and also our internal one of our internal service is actually running this is just a server so you know it's the overwhelming majority of it is currently used on switches but it has an employee yeah I've made sure it works on a boom to 1404 so it it works with no no problem on that with under with four interfaces so I have another question from IRC it's people that are really eager to test this and they are asking if they can like install it while they are running and it will take over from the old one there are instructions on the github documentation page you have to it conflicts with the iphone down package so you have to remove the iphone down package and then install this yeah but can this be done like live or will network be down in the middle I think you can do it live yeah I've not tested that but if you courage if you courage if up down the network stays up so if you pick up from that then that's fine well one thing that I thought might be interesting to know you mentioned that the templating helps reducing the size of the actual interfaces file but to what extent have you tested like let's say to what extent have you stressed this to the tool say could you install a million interfaces and how what kind of time would it take to to actually apply these this kind of configuration wouldn't say a million but maybe thousand it'll bring down if you're talking about several thousand interfaces this will not be your slow point in the system yeah because they end up with sis control and sis config becoming the slow drag while it does its linear walk through all interface names yeah this will be done within like a tenth of the time yeah we've seen a lot of bottlenecks in the other pieces any other questions yeah yeah well I need some help getting this into Debian so I don't know whom to talk to maybe once volunteers sure I'll come and talk to you Matthew you well that's cool but you guys should flock together after the session okay it's not that hard all right any other questions I don't think that's the case so thank you very much for this presentation you've seen that there were a lot of questions there's a lot of interest I can tell you from personal experience if people want something else okay so good luck okay thank you join the community thanks