 So, I'm from Cisco, if you can see. So, who's this guy? So, originally I did physics, but then got sidetracked into IT, and about 10 years ago, a friend of mine said, go to Cisco, they've got some interviews. And I got a job. So, I used to work in the Cisco pack. So, that's the, let's say, help desk. So, if stuff works, I'm surprised. And also, during the first time network for the last 10 years now, more recently, I'm doing data science and machine learning. So, what are you using for connectivity? So, logically, actually, it's a really simple setup. You've got a core switch with a second router, and then a few switches per location. However, traffic is not so simple. When you actually talk to one of the APs, your traffic goes to one of the APs here in the room, then gets encapsulated, sends to the network of the UAB, and then it goes into wireless LAN controller that basically handles all of your traffic. So, if you have a connection to the network and you walk out of this room, the WRC is going to notice that the signal strength is going to go down and that the signal strength for the other AP is going to go up and is going to tell your client to please connect to the other AP. And that will sometimes work. Better like this? Better like this? OK. So, the WRC handles all your traffic. So, what we actually do is we use all the APs of the UAB. So, 10 years ago, we would install access points all over the place, but we don't do that anymore. We basically do it just in Jonson. You've seen the poles with the APs on top. After the WRC gets the traffic, it gives back to the router that then manipulates it and sends it out to the internet at large. So, basically, your traffic goes over our network twice, sometimes three times, depending on it. So, everything goes through the router. So, that's it during build-up. It's about this high. It costs the better part of a house. So, at the bottom, you see the two RPs. We'll come back to that. And above that, the switching fabric. And then, basically, the spas, they call. Not drinking water, but stuff to take the cards. And above it, you actually see the core switch. So, main features. ISSU. Who here knows ISSU? Good. Two guys I expected to know. So, ISSU, basically, you can upgrade this, and it won't go down. You can actually upgrade it while it's forwarding traffic. Nothing happens. Sorry? I mean, I see a lot of problems, and sometimes it even works. It also has non-stop forwarding, which basically is used by ISSU. Basically, you've got two RPs, and one of the two can basically crash, die, and burn, and that one takes over. It has ORI, so you can plug new devices in while it's still running. And it's not using iOS, please. Don't say iOS for everything. This is iOS XE, which is the new version. High level. It's a bit like this. So, can I actually move my mouse up to here? Yeah. So, you have the two RPs, who are the route processors, and they basically talk BGP with the rest of the world. Then you've got the two ESPs, who actually do the forwarding of the traffic. And then you've got the route cards below that basically take care of the card, so they check to have this and stuff. So, the RP, in the past, routers, small CPUs. No, these are two Xeon processors with 16 gigabytes of memory. The memory is there for the BGP table. We see all the networks on the Internet. Actually, at the hard disk. So, you actually log something decent. And that's where you actually log into. On some models, you can actually run service modules, which is called KVM. You might have heard of it. The ESP, that's actually what I always consider SDN. So, the forwarding actually is done not in hardware, but there is called a quantum flow processor, marketing, that basically is like a big piece of silicon. It contains 256 CPUs that basically handle all the traffic. So, that talks to the RP with an API. So, if you're running, for instance, a cloud services router, it's the same architecture, except this is just another process on your Linux box. And then you've got the sparse, tiny computers, because they basically just take care of the cards. Is the card there? Is it healthy? Stuff like that. So, how do you monitor the traffic network actually? I'm going to talk a bit about this. There's a talk of Richard probably tomorrow, and he's going to talk more about how they set up Grafana and stuff. So, we use SNMP. Who knows what SNMP means? Simply not my problem. So, what is it exactly? Well, it's really old. Really old. And you can see it in some of the excellent design choices. So, a bit of language. On the router we have an agent, 80s, and the agent can send traps to the management station, like I just lost an interface. And the management station can get values from the agent, but can also set them, actually. So, that's SDN Avon la lettre. That's actually what some of our products are. Cisco Prime and stuff, they often use SNMP to set values on devices. So, as the piece sounds a bit strange, basically it represents everything as a tree. And the path through the tree is the OID. And you can basically say, get me that OID if you know it, or you can say walk the tree for me. It uses UDP. So, if there's a network problem and you need to send a trap, you use UDP. So, it's sure to get lost along the way. So, version one, we trust everybody. Version two, students are getting nasty. So, let's put clear text authentication in. That didn't work. So, it made version three, which nobody's using in secure mode because it's the customer certificate authorities. Nobody's using that. So, the OID, wonderful. A series of numbers. So, for that, you need to add it into something that a human can use, and that you use a MIP for. The MIP uses ASN1 syntax. So, sorry for the flashbacks and painful memories. So, it looks a bit like this. Just too hard to live. So, MIPs are extremely messy. On my WN box, if I install the SNMP MIP package, and I download them all, and I say I want to have the Cisco ones, and I do a... I get one value from my WRC. That will produce 13,000 lines of errors. It will also take 1.2 seconds, just because it needs to load the MIPs and pass them, so it's incredibly bad. So, basically, nobody does this. They all go to the... Well, people are using Cisco devices. Go to the object navigator where you can give in the name. For instance, the IP address type, and you press Enter, it will tell you the OID. This wonderful set of IPs. And it will also tell you which MIP it is and what device it supports. You can actually walk that tree then, and you can see what other interesting things are in the neighborhood. So, for example, this is how a walk looks like if you ignore all the error messages. So, you get stuff like IF number. But then, look at what the rest is. You get a table of interfaces with an index and then a number. So, interface one is this thing, which is Ethernet, which has an empty U of 1500. Now, there are two problems with this. One, these numbers by standards can change. If you reload the device, it might be something different. Two, it's the wrong order. The device itself doesn't keep track of its interfaces in that way. There is one block per interface with all the statistics. It's not a tree. So, for one, two interfaces, this is no problem. If you've got a fully loaded Nexus 7000 with 16000 interfaces, not nice. Anyway, we collect this from all the switches, the ASR and, thank you, UOB, also from the WRC. So, if you see those nice graphs with the amount of people that came and left and then they went over the drinking of drunk and now also the sales of t-shirts, for example, that also is in the Grafana and it comes from this. So, as I said, several problems. It's UDP. So, it's fire and forget. Most often forget. There is no concept of high availability. So, companies want high availability. So, there's not one management station for your device. There are two. And then it turns out that the server team also wants to pull. So, there are four. And then the database team also wants to see those numbers. So, there are six. And then your device doesn't work anymore because it's only pushing out SNP data. So, it's pull. You don't push. So, you have to basically say, when will this counter change? I don't know. Has it changed yet? Has it changed yet? It's very verbose. As I said, it goes across the wrong way. However, let's move on. I'll come back to this later. The other thing if you use this NetFlow. Now, who knows NetFlow? Still a few people. So, NetFlow, actually, it's called Internet Protocol Flow Information Export, or IP Fix, or FNN, or NetFlow, marketing. So, basically, with NetFlow, you can basically export information about traffic that passes through the router. And you can filter, you can sample because you don't want to know everything about every flow if you're on a really big device. And you subscribe to it. This is not what's called Legio Intercept or a Spun or something. So, we don't copy your data. We just gather information about this. So, what can we gather? Traditional double source IP destination IP ports, flags, header fields, stuff like that. How much you are talking? I mean, packets, bytes. Some quality of service measurements. But if you've got force over IP, we can basically dump jitter and drops and basically, why was the phone call bad? We can tell you. Also, if you have a BGP, we can tell you the autonomous system for the source or destination or the path or what was the neighbor you were using. And then you can use N-bar. Who knows N-bar? There's still quite a lot of people. N-bar basically is deep packet inspection on a router. And you install protocol packs and they will tell you what this traffic was up to some granularity, of course. And then you can use that to either implement quality of service or export it as a net flow. So, what protocol does it know about? Well, quite a lot. For example, if you go to Forchian, that's a protocol. Porn sites are also protocols. Skype is a protocol. And because it changes all the time, you get these protocol packs with more updates. So, per protocol, there are some stuff you can export. So, for instance, for SNMP, SNTP, sorry, you can export the sender and the server. For HTTP, host, referral, URL, we don't care about that. We care about user agents. Because the Forchian people always ask us how many people are here. I don't know. How many people are still using Windows? We don't know. The user agents, we can gather statistics so we can tell you how many Windows users there are, how many Android users, et cetera. It was interesting because when we implemented the IBV6 only, we didn't know if it was causing a problem or not. But by using the user agents, C was fleeing to force them ancient legacy and we could actually report that actually everything seemed to be working. So, how expensive is it all? Actually, not that much. DSP doing it and it's twiddling its tabs. So, let's look at this live. So, I'm not... This is one I prepared earlier. And I'm not using SNMP because that's a bit too voluminous. I'm just using the command outputs. It also teaches you the commands. So, how does CPU load? Well, lots of SNMP. That's a bit expected. As you can see, actually, not much is going on in this device. Because remember, it's the ESPs doing all the work. But so, you can actually say, tell me how the control processor is doing. You see everything. So, you've got the two RPs, the whole processors, the two ESP forwarding engines, and all the cards. And everything's healthy and happy. By the way, does something look familiar in these outputs? It looks like top. Yeah, it looks like... Yeah, that actually is a traditional load average from a Unix machine. Because guess what they're running? You can also see how much memory is in there. As you can see, the route process is a lot. And you can see how much is used and committed. But everything's healthy. That's good to see. And you can see the CPU usage. Again, here, user system, nice idle IRQ, soft IRQ and IO weight should trigger some memories. And now we can actually also look at the data path, so actually the forwarding itself. So, here you can see how many packets per second are getting forwarded and how many bytes per second. This looks impressive. And don't you look at the bottom line, 8%. So, and this is with everything on. So, it's taking all the data, exporting it. And the pipe was pretty full at this moment. Let's keep the T-cam because nobody really cares. If you know what the T-cam is, you have my sympathies. So, you can actually also check what the statistics are for the net flow on the router itself because it needs to gather all these flows. So, for instance, you can say for the IPv4, you can basically see how many flows there are at this moment. There were almost 14,000 flows going on and at most there were 18,000. And you get the packet distribution and how many flows you have per protocol. So, still an amazing amount of telnet. Something must be wrong there. And what the fastest and the slowest protocol was. This all gets exported to a collector. And then basically for a flow, we get the source destination, how many packets, and then the user agent. So, how we analyze this? Well, open source support for this is sparse, especially flexible node flow. Most of the dumpers, they are out there. They don't handle the extra fields. So, brace yourselves. I wrote the Python, sorry, a Perl script. The first line is use modern Perl. So, I'm good, right? It basically takes the data and dumps it. And then because this is Perl, I could extend it to name all the extra fields. However, this is the old stuff. The future is young and telemetry and netconf. That's what you guys know, right? If not, let me tell you what's wrong with SNMP. So, it's for both. If you query it for 10,000 interfaces, it will tell you zero bytes for errors 10,000 times. It doesn't handle age availability, UDP. You have to pull it every few seconds and then it works the tree, which is basically pretty much wrong. So, as a solution, it's telemetry. So, all the buzzwords work. So, basically, you subscribe to the flow. It uses Google protocol buffers, which basically means that instead of having a 16-kilobyte packet, you have a 200-byte packet because it actually compresses all the values. And internally, it will collect all this information once and then basically send it to all of your monitoring stations. So, if you want eight copies of the same data, it's okay. We don't care anymore. And, actually, we get a lot more metrics and we get them faster. A lot more metrics. The metrics we get, they are described by models. And they're actually on GitHub. So, as you can see, for XC16 Polaris, we actually published models this Christmas. I was very happy because I knew they were coming. And, actually, Cisco wants to push this. So, there is a whole landing page and a young development kit to basically get you started with this. And even blog posts that tell you how to do it. So, with this, you could basically fully not only monitor the device but also configure it using whatever language you want. Lots of people seem to like Python more than Pearl. But, okay. There is one more thing. I am intact, so I need to push a bit on our agenda, too. Who knows the wonderful Cisco command line analyzer. These are, yay. So, basically what it is, it's a tool. You download it. You either dump ShowTex into it or you basically download it to connect to a device and it will run some commands. And, basically, the commands it will run actually come from TAC. Because if you sent a 200 megabyte ShowTex to us, we are not going to read it manually. We have some tools for that. And, basically, some of the tools we made external, so you'll have to log in. But the data will be analyzed by the same tool that TAC is using and you'll get a report basically saying these and these and these things are wrong with your device. I even ran it against the FOSDEM infrastructure and, to my surprise, it even found problems still. So, nobody's perfect, least of all me. So, so, questions. So, okay, so the question was what platforms support telemetry? All platforms that are not legacy. So, iOS, iOS, iOS, that's legacy. So, but iOS XE, which is running on the router. The next version actually runs on the router, but also on, for instance, wireless LAN controllers. It's the Polaris. iOS, iOS XR, which is the provider iOS. So, if you've got 16 racks of units and that's one router, you're running iOS XR. It's also supported on there. And it's also supported on NXOS. So, the data center switches and some devices. They're also supporting it now. So, basically, everything that's not base iOS. Can we see the live dashboard for now? The live connection? Well, I can log in. But you can remove the plug. You need to remove the plug actually. Let me first quickly go and mirror the screen then. This IP is not a secret. If you trace out, that's the one you're going to see. So, this is, yeah, the SSH key authorization. It also works for, I have aliases to see this. So, at the moment, I'm going to, yeah, healthy. It's too small. So, yeah, SNMP is still the major cause of problems. And you guys are not trying. It's still only 7% of processing load on the devices. What is the current bandwidth to go to the Internet? To 2,000. Option ever. I think a bit per second. And about also 380 back. And you don't know what was the maximum today? That's what we're using SNMP for. There is history. But it's not... Last question. What are the Wi-Fi issues with the classic one? Or what do you have to use with FOSDEM ancients? Ah, so, yeah. What are the Wi-Fi issues with FOSDEM ancients? What device are you using? When I tried FOSDEM in the network, I have to use FOSDEM ancients. Ah, okay. Yes. So, FOSDEM is IPv6 only. So, it works. But if you go to a site that doesn't have IPv6 addresses, we lie to you. You do because IPv6 is such a used address space. You basically have the entire IPv4 address space tucked away somewhere. So, if you do a DNS request for a site that's only IPv4, we will lie to you and return IPv6 address which we fabricated. If you then go to that IP, the router knows that this was a lie and will convert IPv6 IP back into an IPv4 IP and send it out on the Internet. So, we do it like nuts, but on IPv4, IPv6 to IPv4. That works if you're using DNS requests. If you just say, I want to go to 8888, no route. To be a lot of people lost in the Wi-Fi. So, that might also be a reason why... Yeah, it could also be. The Wi-Fi in these groups, so by ULB, and the priority of the ULB is that the professor at his desk has Wi-Fi. And the students is optional. So, okay, so about the NAT64. So, the DNS64 is run on a server we have, but then the NAT64 is on the ASR1000, yes. What's the level of resources of the ASR1000 that users have done in the ESP? Okay. So, what resources are that used? That basically is fully accelerated in the ESP. So, almost no traffic hits the RP at all. How much traffic is there? Ooh, so how much traffic is the NAT64? We can see how much is in the use in the pool, but no, it's only connections we can see, not the amount of traffic. Do you have flow data on the NAT? No, because... So, we only collect flow data on the inside because the outside is just too much junk. So, are you aware of the open source traffic switch? Do something similar like split DNS 4, 4, and 6? Yes, so if there is an open source NAT64, yes, actually the first implementation ever was on a free BSD. I mean the DNS 2. The DNS 2, but now it's part of bind. It's an option in bind to basically define the prefix and then everything is automatic.