 So the plan is very simple. I will start with network visualization, and I will end with network automation. But before I start, I want to make a short introduction into networking, like a crash course in networking. For those of you who don't know well what it means. So when I say network, what I mean is a telecommunication network. It's a network which nodes are networking devices like routers, switches, and antennas. And a network which links are transmission links, usually Ethernet cables and optical fibers. In the network of an internet service provider, we find three main types of nodes, router, optical switch, and antenna. As you can see, these are very big devices, like with lots of cards and lots of interfaces. So this has nothing to do with the type of device that you would have at home. These are corporate devices that we would find in the network of a big company, like Facebook or Google, or in the network of an internet service provider, like Vodafone, Telefonica, and so on. Same for the transmission links. These are links like those ones on the left, an internet cable with 10G capacity, a 10G connector, and on the right an optical fiber. So internet cables are used for short distance transmission, and optical fibers can be used for both short distance transmission and long distance transmissions. Well, there are a few approximations in this slide because it's like to make it simple for you to understand. But if we imagine that you are in Remini, and you want to call someone in France, like with your mobile phone with a 4G, what would happen is that your mobile phone will send a microwave signal to an antenna that should be located nearby, like close to this Palacongresi. And then the signal will travel through a bunch of antennas. It will be a microwave signal. And this microwave part with antennas, we call it the microwave backhaul. Eventually, it will reach a router. A router is a device that will take the forwarding decision. What we can say about them, it's like the brain of the router. They know how to reach any part of the world, and they will make the forwarding decision. They will know which pass the traffic has to take. In order to do that, the router has what we call a routing table, which is a table that associates an outgoing interface to an IPRS. And finally, since we want to go to France, which is very far, we need the signal to go very fast. So the router will send the signal to what we call an optical switch. And this optical switch will perform an electrical to optical conversion, which means the signal is electrical at first, and it will become a wavelength, which is a stream of light. And on top of that, in a single fiber, we can have usually up to 88 wavelengths, so 88 stream of light. And this is a technology we call Waveland Division Multiplexing, which is used to have very high bandwidth capacity. When we buy network devices from a vendor like Cisco, Nokia, Huawei, they usually provide what we call a network management system. It's like a software or a web application that can be used for network visualization, inventory, provisioning, supervision, and sometimes automation as well. It's a network that has like an overview of the network, of all devices. If we want to visualize our network, the input file that we have is usually a network topology. It could be like a CSV file, Excel file, text file. Basically, it's a file that will contain the list of all nodes and the list of all links and the position of the nodes as well. So if we want to draw this network, as you can see, if we draw nodes at random positions, this doesn't yield a very good display, so we have to find other ways. We have basically two options. The first one is to use, the first one is to draw the nodes at their physical location, where they're physically located by using GPS coordinates. To do that, we use a type of software that we call GIS software, which stands for Geographic Information System. And that allows us to, the idea is to draw maps and draw our network devices on this map. And the other option is to use a graph-during algorithm. I will make a demonstration of both if it keeps working. So what I'm going to show now is how we can create a simple GIS software in Python, so software to draw maps. The first thing that we need is a grid programming framework. We have a lot of such framework in Python. We have PyGDK, PyQT, WX, Python, PySide. But the one I'm going to use is Tecinter. It's Python interface to Tico. And it says the only grid framework that is part of the standard Python distributions, which means you don't have to pip install anything. It's by default. It's a built-in module. Inside that framework, we will need a widget for drawing. All widgets have one. In Tecinter, it's called a canvas. It's basically a widget that allows you to draw circles, rectangles, but most importantly, polygons. And we will use inside that widget a method to draw a polygon. The reason for that is that a map is actually just a set of polygons. If you look at Italy on this slide, Italy is just like a polygon. It can be considered a polygon. So to draw these polygons, we need to have coordinates. And we will use a special type of file, which is called a shape file. A shape file is a file that contains shapes. And the shape file that I'm going to use contains two types of shapes, polygons and multi-polygons. These shapes are described as a list of coordinates, as a list of points on the Earth. And a point on the Earth is defined as a longitude and a latitude. So the first library that we will use is called PySHP. It's simply a library that allows us to extract the shapes from the shape file. It contains a reader object that we can use to read the file. We can draw polygon with the create polygon method in Techinter. But as for multi-polygons, we cannot draw them. We can only draw polygons. So what we will need to do is to convert all the multi-polygon is actually just a set of polygons. And we will need to convert a multi-polygon into multiple polygons in order to be able to draw them. And to do that, we will use another library called when you use shapely, and you convert a shapefile multi-polygon into a shapely multi-polygon. It will actually make the multi-polygon iterable, which means it will have this double underscore iter function so that you can loop over the multi-polygon to yield all the polygons it contains. As I've said, shapefiles contain coordinates defined as a longitude and a latitude. And if you look on this slide, you will see that these are angles. But when we draw polygons in a software, in a canvas, we cannot use angles. We need pixel coordinates. So we will have to make some kind of conversion from angles to pixel. This is actually a conversion from 3D to 2D because we have a point on the Earth. And we need to have a point on a plan. And this is what we call a projection. So there is a mathematical theorem by Gauss that basically says that if you make such a projection, like you have a sphere and you want to project this sphere on a map, it's called the remarkable theorem, then you will lose some information in the process. There will be what we call a distortion. It could be a distortion of distance, of angles, of shapes, any distortion. But we will lose some information. For instance, if we consider the Mercator projection, it preserves angles, but it doesn't preserve areas. In order to make that projection to convert 3D coordinates to 2D coordinates, we will use another library called Pyproj. It's a library where you can choose a type of projection. It could be Mercator, or it could be azimuthal orthographic, like you see on this slide. And we will convert a longitude and a latitude into pixel coordinates. So if I sum up, we need a grid programming framework. We will use TakeInter. Inside this framework, we will need to use a widget to draw polygons, TakeInter scanVas. We need a file that contains the coordinates of the map, a shape file. We'll use PySHP to read the shape file. We'll use ShapeLy to convert multi-polygons into polygons. And we'll use Pyproj to convert geographic coordinates into projected coordinates. So OK. OK. I will open a Jupyter notebook to show you how it looks like. So this first code shows you how to initialize TakeInter main window and how to initialize a canvas. And finally, how to use a createPolygon method in order to create the polygons. If I start this code, I will get this, which is like a graphical interface with a polygon. Finally, I have this snippet to show you how to use Pyproj. We need to initialize a projection. This EPSG code, like 3395, stands for the Mercator projection. Then I have the longitude and latitude of Remini. And I can use the Mercator projection to convert longitude and latitude into projected coordinates. And I can use this inverse equal true to make the reverse operation, which is convert projected coordinates into geographic coordinates. And if I run this code, I print the coordinates. We can see that we have the xy and longitude latitude. Then I show you how to use the shapefile reader and how to import shapely. So I have this local pass to a shapefile. And I will start with the reader object. I will extract all the shapes from the shapefile. And when I loop over the shapes, I will convert all of them into shapely shapes so that polygons are iterable. Like this. You see that I have this azat shape magic iter. And it returns true. So all polygons are iterable. And finally, this is the final code that shows you everything, how to start the kinter, how to give the local pass to the shapefile. I have this two projected coordinates function in order to convert longitude and latitude into projected coordinates. I will read the file, extract all its shapes. I will loop over the polygons, convert them into shapely polygons. And finally, I will use the createPolygon function. If I execute this code, I get this. It's very simple, but it's working. And I can improve it a little bit by I can change the projection here to have an azimutal autographic projection. I will create a circle with which radius is the radius of the Earth. And I will use takeInter file dialog so that when you run the code, takeInter will actually ask you to choose a file. And it will then draw it. So if I run this code, it asks me for a file. And if I draw it, I get this. So this is very easy to do in Python. As you can see, it only takes 30 lines of Python to get this result. So what I want to show now, if it works, it doesn't work. OK, so I will actually use this method in order to import a real network. I will import project. This network file contains a real network. It's actually one of the main French backbone. You will see in a few minutes. OK. So if I import this file, OK. So this is one of the main French network. It is made of about 350 optical switches. In case you're interested, these optical switches are Alcatelucent Photonic Service Switch, or PSS1830. So basically, if you are in France and you're trying to use internet with your mobile phone or with your computer, well, there is a chance that the signal you send will actually be transported over this network. Of course, if I wanted to do just visualization, I could use frameworks like I could use a Google Map API. I could use Geo Django, or I could use D3 also if I wanted to do it in JavaScript. But the reason why I want to use an actual software is that I can select some nodes. I can move them. I can delete them. I can create new nodes with a drag-and-drop system. So I can actually create a network. And I can look at the property of the node. For instance, if I put the IP address here, OK. Well, I can say like this node has this IP address. And once this is done, I can hopefully start an usage connection to the device like this. OK, so the device I'm using, you don't see it from there. But it's actually just here on the ground. It's a Cisco 1841. It's back. OK, so I'm connected to the router. There is a few things that I want to show you. The name of the router, it's called router 2. It's what you see on the left part right here. So this is what we call the host name. If I want to change it, what I need to do is to enter the configuration mode by writing configure terminal. And then type host name and the host name I want, for instance, Remini. And when I do that, you see that the host name, the left part, is changed to Remini. So what I'm going to do now, well, the idea is that if you have like 5,000 devices and you want to change the host name on all 5,000 devices, you cannot possibly log into all the devices like by hand and change the host name yourself. So you need to automate this process. And we can do this by using Python. So I will come back to my slide. Well, OK, so this is a slide. This is an overview of the network automation landscape, which is all the method you can use in order to automate the network. The first one is the one that network engineers have been using for many years. It's simply to connect to the device with a library such as Paramico. Paramico is a default library in Python to start SSH connections. But Paramico is very complicated to use. So network engineers have built other libraries on top of Paramico in order to make it easier for network engineers to automate. The two most famous libraries are NetMico and XCrypt. Finally, you have also the NetConf and Yang solution. NetConf is a network configuration protocol. And Yang is a modeling language that we use to shape the data that are sent over NetConf. This is standardized by the IETF. I don't want to spend too much time on this one, but just so you know, there was a talk at the last Python US in Portland about how to use NetConf and Yang. Finally, you can use OSS protocols like TL1 and SNMP with Python library to SNMP, which is called PySNMP. But SNMP is usually very complicated to use. You can also use configuration management tools. These tools originate from the DevOps world, but they can actually be used for network devices as well. So you have Puppet, Chef, Saltstack, HPNA. But you have also Ansible, which is the one that is gaining momentum at the time. And it's an open source Python project. And well, if it works, I will make a demonstration of how to use NetMico, XCrypt, and Ansible in order to change the hostname. So I will come back to the code. I will just explain what I would have done. What I wanted to do was to use XCrypt, NetMico, and Ansible in order to show you how to do basic automation. I'm not sure I can still do it. OK, I'm still connected to the Cisco device. You can see the hostname is Remini. And if I use XCrypt, so the idea is that I must create an Acunt object in order to give the credentials to connect to the device. I must create a host object to give him the IP address of the device. And finally, I will send all the commands required to make that automation. So if I say hostname France, and I run this code in a new shell, you see the hostname was changed to France without having to log into the device. So if we had a lot of devices, what we would use is to multistread all the SSH connection in order to configure all the router at the same time. XCrypt allows you to do this by, if you loop over the IP address and you create host for each IP address, then you append them to a host list that will contain all the device that you want to run the script on. Then XCrypt will actually use multiple threads so that all SSH connections are done in parallel. And you have this max thread parameter here when you can say, I want at most five threads. You can do exactly the same thing with NetMico. You have this connection parameters dictionary that contains the credentials like username password, as well as the IP address of the device. You would use this connect handler to log into the device. And you will send all the command you need to send. So for instance, what I'm doing here is that I'm adding a description to the interface Fastet.Net zero slash zero and I'm also storing the result of show running config which is the configuration of the router inside a config variable. So if I run this script, it will take a few seconds. Okay, if I print config, I see that config contains the configuration of the router. And if I look at the interface, the configuration of the interface, I see that it has this description OSPF interface. That configuration was pushed with NetMico. I can also do the same thing with Ansible. By, I will start Siegwin. So the idea with Ansible is that you have a file called inventory. If I look at this file, you see it contains the IP address of the device you want to send the script to. And you have what we call playbooks. These are this file dot yml, yml files. You can look at one example here in order to do the same thing which is changing the hostname. So I will first give the credential to Ansible and I will then use the iOS config module. And this lines hostname Paris will change the hostname to Paris. So I can do that using Siegwin by writing Ansible playbook. I tell him that the inventory file is inventory and I change the hostname. Okay, as you can see it worked. The Ansible script was sent to the device and the hostname was changed. What I can also do is use the iOS command Ansible module in order to send the show running config command and then I will use copy to store this configuration in a file output dot text. I will also use Siegwin to send this one. So change hostname. Save running config, okay. And if I look at the folder, you can see that I have this new output dot text file and if I look at what's inside, it contains the configuration of my device. So, well, I had more things to show but I think we can stop here. Thank you. Thanks Antoine for the talk, questions. When you want to use commands that will require confirmation, like, I don't know, reload the router probably will... Yes, actually with X-Crypt, you should look at the documentation. I don't know if I have internet but if you look at X-Crypt documentation, you can tell when X-Crypt should expect a prompt that it does not recognize. For instance, if you connect to an Alcatel devices, sometimes it will ask you for yes or no, say yes to continue the login part and you can tell X-Crypt that if it says yes, then it should understand that the device is expecting something and that works perfectly. Maybe you have told us where have you got the data for the geographic representation from? Well, I took the shapefiles. I mean, the raw data, where did you get it from? The shapefiles, you can find them online. I just Googled shapefiles countries, our shapefile continents, about the data of the network. That's because, well, I work at the company that runs this network, so I did an XML query to the SAM 5620, which is a software that supervises the network in order to retrieve the topology. Hi. Hi. What are the best practices to map a lot of devices? Yeah, well, actually the tool I made, I wanted to show it, but because of the technical issues I couldn't, can do that, that was my next demonstration. If you have several devices like this, what you could do is, what I did is providing a graphical interface so that you don't have to cut anything, it's so graphical. I can go to script creation, and I can make a script change hostname. And then if I write configure terminal, and then hostname, you would like this part to be a variable, right? Is that your question? Yeah, let's say you have like a device. Yeah, so using this software, what you could do is like doing hostname, which means replace this part with the hostname value of the property of the device, and you can save this script. Then if you look at the properties, it will look for the hostname properties, and it will replace the value. And actually the hostname properties does not exist yet. So what I did is a way for network engineers to create properties on the fly, even if it does not exist. So I have a file called change hostname, and it contains the value of the hostname. And if I import this file like this, import hostname, you can see that the property, well, it doesn't work here, but you can see that you have new hostname properties, and it actually contains the value that is in the Excel file. And then what you would do is select all three devices like this, and then you click on send a script, you will choose a script, and for each devices, the hostname part would be replaced with the actual value of hostname for the node, you know? This is using X script. X script on the back. X script, but you could do the same using NetMico, and you could do the same using Ansible as well. And if you wanted to do this like in pure Python, you would just need to have a dictionary that maps, well, the IP address of the device to the value of hostname or whatever you want, and that would work too. The question was if you use some bolt or some secret storage, something like that, because actually those credentials in customer environment are very important. Ah, yeah, no, I don't use any secret storage, because it was just for a simple demonstration, and well, it's not like, but yeah, you would need to encrypt the data in a real network. Okay, thanks. More questions. So is there an Ansible interface for PyNMS? Is it what? Can you run Ansible scripts, whereas like selecting a few hosts and right click run script? Actually, I want to add that as a new feature, but right now I'm not like, I'm just working on Ansible, trying to make a useful Ansible script, and then when I get something that's working, I want to make PyNMS as a graphical interface for Ansible, so that you don't need to actually write the YAML file, you can do it graphically as well. More questions. Okay, no questions. So let's thank Antoine again. Thanks.