 Let me quickly tell you who I am, my name is Wim Goden, I'm from Belgium, I think most of you know that, well, probably because of beer and chocolate and other tasty, yummy things and odd-looking buildings, maybe. I started the company about 18 years ago now called Kube Solutions. We do mostly PHP development, we also do quite a bit of infrastructure, which explains why I'm doing a talk about networking, of course. And we also started a training service called TechPath. And I've been doing open source for the last 20 or so years. And I wrote things like PHP compatibility, which is a tool you might want to check out if you need to migrate your code to PHP 7.x. And I've been doing talks like these for quite a while. Okay, let's see who you are, who is a developer, most people, any system engineers, anyone doing network, anyone network engineer? Oh, you know one, okay, that's good. Who knows how the internet works completely? Oh, a few hands and then they went down again, okay. So, yeah, obviously we're web developers, we might be DevOps, maybe, or SysOps. We're not network engineers in general, except the few people here. But we know enough to build stuff, obviously. We know enough to maintain existing configurations, existing servers and so on. But let's imagine the customer calls your boss and says, or to the support desk and says, the website is slow, please fix it. Well, that's all good and well. And they might call you and say, the website is slow, fix it, please. And you go on the website and you have no problem. Because maybe the problem is not your code, maybe the problem is not database related, maybe the problem is network related. So what do you do? How are you going to fix it as a developer? And so that involves knowing things like DNS, yes, TCP, BGP, whatever else. You know, a lot of different terminology that most developers have never encountered or yeah, they know sort of what it means but don't know the basics of it. So that's what this session is kind of about. Getting you, getting those of you who don't know a lot about networking up to speed to a little bit more of the basic understanding of how a network works and how the bigger internet, the bigger network called the internet works. Okay, so networking, there's a model that describes different layers that we have on the network and basically it starts with layer one, which is a physical network which is basically your wires, your network cards, your wireless interface and so on. And we have layer two on top of that, which is the data link layer, which is the data protocol, usually it's ethernet, we use ethernet the most. There's a few others, but we're not going to discuss that. Then there's layer three, the network layer, which we all know, I think, which is about IP addressing and yeah, that's basically what we do all the time. And then there's the transport layer on top of it, which is TCP, which we all use every day, UDP, things with ports, everything lives in that layer. Then it gets a bit blurry because the session layer is, well, there's TLS in there, there's L2TP socks and a few other things. But some people say that that lives in layer six and other people say, no, that belongs to layer five. And it gets even more blurry when we go to layer six, which is about serialization and data translation. And then finally we end up with layer seven, which is where we build our stuff, which is, it's about HTTP, DNS and stuff like that. So we build on top of that. I can go on, these are the seven official original layers. Some people describe layers up to layer 12 and it gets even blurrier. So yeah, but networking basically revolves around bits of data that are being sent over a physical cable or wireless and are being translated on the other end into bits of data. And these bits of data are usually grouped together in packets. And a packet always consists of a header and actual content. Now packets can contain and usually do contain other packets. So you have a packet which has contents and then that contents has another packet which has a header and that's contents and so on. So it basically looks a bit like this. This is an Ethernet packet which has a destination MAC address, which I'll get to in a minute, and a source address and a type, which in our case is usually IP, the type. And then it has a payload and that payload is basically another packet, in this case an IP packet, which has a lot of headers as you can see. And then at the bottom of it, it says contents of the packet, which means there's another packet in it, in this case a TCP packet. And so you can have multiple layers deep and so on. So if you send a packet from one machine to the next, then it's going to have to get each packet out of the previous packet and basically sort of, yeah, I would say unpack it, yeah. And then if it needs to transmit that to the next host, it's going to take all those bits and pieces and group them back together in the right order and send it along again. So that's how networking actually uses packets. So if we look at an example, a typical example is an Ethernet packet because we use Ethernet all the time and that contains an IPv4 packet and that then contains a TCP packet and so on. Now, okay. So we have here a destination MAC address and a source MAC address. Now these are sort of the basic concepts without those networking on the Ethernet would not work. What is a MAC address? A MAC address is basically a unique identifier for every single device on a network. It's supposed to be unique worldwide, but as you can tell, six bytes is not a whole lot. So theoretically, there could be conflicts. In reality, there shouldn't be yet. But if we keep adding more and more devices, we will run into trouble, of course, at some point. Now, let's first talk about how do we send data across a local network? So imagine that we have a couple of devices here all wired up on a local network. How do we send that? Well, in the old days, we used to use a device called a hub. Now, a hub looked a little bit like this. You will recognize standard RG45 ports, but you'll also recognize a coax port there, which is not really in use a lot anymore today. The problem with a hub was all devices could be sending at the same time. So you would get one device sending a packet and another device sending at the same time, and it would collide. It would be a collision, as they called it, and then it would have to retransmit it. A hub worked on layer one, so purely on the data on the physical layer. Not very efficient, not very fast. Speeds usually limited to, well, 10 megabits per second. So not very efficient. Luckily, in the next phase, somewhere in the mid-90s, we got a better device for that, and we used a switch, which looks a little bit like this, which we all recognize, I think. And what happens here is, as I said, every device on a network has a MAC address, which is supposed to be unique. It's a signed by new manufacturer. Now, you can override it if, for example, you're setting up VMware, then you're actually creating virtual MAC addresses. You can define them yourself. You can modify them if you want, which means you can mess with things as well. Now, if you are located on the same physical network, like if you plug into the switch with two devices, you are on the same physical network, then the way you communicate is by sending an internet packet with the destination MAC address. And the switch actually knows which device has which MAC address, so which port on the switch has which MAC address, so it knows where to send that traffic. So that's layer two. If you want to send IP traffic, well, an IP packet is part of an internet packet, so you're still sending the same kind of packets. It's just you're adding a packet within the packet. And then you require IP addresses. Now, where do you send them? Well, on a local network, all we can do is send things to a MAC address, because that's internet. So we use the thing called ARP to find out the MAC address of the device we want to communicate to you. How does that work? Well, this is actually what such a packet translated in human-readable form looks like. This is an ARP request. Basically, 192.168012 is requesting the MAC address of 192.168015. And so that device is saying, hey, I'm looking for the MAC address. Can someone tell me? And then the one with dot 15 at the end is going to reply and say, dot 15 is at this MAC address. And now the two devices can actually communicate because they know each other's MAC address. This is cached. This is stored in a so-called ARP table so that you don't have to do this lookup for every single packet that you want to send. Now, what is local traffic? Basically, local for IP means you're on the same IP subnet. I'm sure most of you have heard of what a subnet is. But what is it really? What does it define? So if we look at IP version 4, which is what most servers are still running on, we'll get to IP version 6 later, it uses CIDR notation, classless interdomain routing notation, not important. It looks like this, we all know this. So four numbers separated by a dot, where each number is between and including 0 and 255. So it goes from 0, 0, 0, and so on to 255. In reality, what that means is it's 4 times 8 bits. So the address could look a little bit like this, for example. 192, 432, 1 in reality is sent across the wire as a number of bits. Now, this means if you have 4 times 8 bits, it means 256 times 256 times 256 times 256 addresses, or 2 to the power of 32, or 4.3 billion, sort of. That's the total number of IP addresses that are available to the world. We'll get to why that's a problem. I mean, you can already tell if there's 8 billion of us, and we have only 4.3 billion addresses. That's kind of a problem, but I'll get back to that. Now, IP networking requires an IP address, and as I previously said, a subnet mask. Now, what is that subnet mask? All it defines, basically, is, OK, you have an IP address, but to which range of IP addresses does this IP belong? So it kind of puts sort of a limiter on the range to which it belongs. It basically says, yes, I know this IP address is part of the 4.3 billion, but it's also part of a limited network. And the reason to do that is that any address that's part of the same range can talk to each other locally without having to go through some kind of router, without having to go through other devices. So any IP within that range can talk to each other. For example, this range, 194, 50, 97, 0, 255, has a certain subnet mask. And any IP addresses in that same range are considered to be locally on the same network, so they can talk to each other directly. Now, there's two notations for that. Well, I'm going to go back. So a subnet mask can be defined in the first notation, the 255 dot, and so on. Or it can be defined as slash 24. You often see that when you go on Linux command line and you type IP adder or IF config or so on, then you often see a notation like that, slash 24, slash 25, and so on. The reason it is expressed that way is basically IPv4 gives you 2 to the power of 32. And then a slash 24 will give you 2 to the power of 32 minus 24, so 2 to the power of 8. So it's going to give you 256 addresses. Quickly, I want to mention there's three exceptions. I'm pretty sure you recognize the last one, 192.168.0.0. That's usually the IP you get from your local router at home from your ISP. Those are ranges that do not work on the internet. They only work on local networks. That IP is not routable anywhere on the internet. Now, let's look at a quick example. Let's say we have a range 194.710 slash 24. That means it's a range from 1.0 to 1.255. It contains 256 addresses. And let's imagine you want eight addresses to install your server zone. Then you could just use this range, or you could say, well, I only need eight. So I'm going to make a smaller range instead of 256 addresses. And I'm going to reserve that range specifically for my servers. And they're going to be on this specific switch or router and so on. So you could say, I want a slash 28. Now, what is a slash 28? Well, it's 2 to the power of 32 minus 28. So it's going to give you 16 addresses. Now, you might think, wait, a slash 29 will give me eight addresses. And I need eight addresses. That's perfect. The problem is every single subnet that you make, the first address is called the network address. And the highest address is called the broadcast address. And you cannot use those. On top of that, each subnet usually needs a default gateway. So you're losing three addresses. So out of 16, you have 13 usable. This also means the more subnets you make, the more addresses you will lose. So the 4.3 billion, given how many networks there are in the world, networks subnets there are in the world, we lose a lot of those 4.3 billion anyway. So the subnet in this case, 194, 710 slash 28, also written in a different way, which I'm not going to get into right now. Now, to be clear, the subnet doesn't have to start at the dot 0. It can also start at dot 16, or dot 32, or dot 48. It can only start, though, at a multiplication of the number of addresses. So a slash 28, which has 16 addresses, can only start on dot 1, dot 16, dot 32, and so on. But you can make combinations. So if you have a big range of 256 addresses, you can make it start at dot 0 and go to 127 with a slash 25. And then the next one can start at 128 and go to 159, and so on. So any combination makes sense. You can just not overlap, of course. That wouldn't work. Devices that overlap on the same range, they would not be able to talk to each other in a decent way. Just something I want to throw in between. If you ever want to check if an IP is inside a range, we're at a PHP conference, so I have to show something of PHP, right? So this is a very convenient piece of code, which basically avoids that you have to do all sorts of parsing and checking with dots and splitting and then doing all sorts of weird calculations. Basically what it's doing, it's using IP too long to turn an IP address into a long, and then printing the bits, 32 bits, and then running substring compared to just to check, is this IP within this range? In what, six lines of code? Very convenient. Anyone ever heard this phrase? I think there's a world market for maybe five computers. Someone said it somewhere in the 50s. I think it was someone working for IBM. Or 640K is more memory than anyone will ever need, something that Bill Gates said at some point. And that's another one. 4.3 billion IP addresses is more than enough. That's what they thought in the 70s when they built IPv4. It is not enough. We are running out, and we have been running out for a while. 4.3 billion doesn't cover it anymore, especially with mobile devices. So IPv6 was created, published in 1998, but it's still not being deployed everywhere. We're slowly getting there. Most major sites have it. More and more ISPs at home have it, but a lot of small sites are still behind. And now an IPv4 address, well, you already saw that. An IPv6 address looks a little different, a little bit longer. Now, luckily, anywhere where you see four zeros, you can actually remove that and say double colon, double colon, but you can only do that once. So you cannot have four zeros and then something else and then four zeros and remove both of those. The problem is IPv4 and IPv6 cannot talk to each other. They live side by side. They can run on the same devices, but they cannot communicate with each other. So basically, if you want to use IPv6 and you want to fetch something from a website, you can only communicate with that website if they are also running IPv6. Now, to give you an idea, the address space of IPv6 is 2 to the power of 128, which is that many addresses. We are not going to run it out anytime soon. Please don't quote me on that. And to give you an idea of the deployment rates, globally, 22% of people at home have IPv6. In the US, it's a little more. Belgium is number one here. We crossed the 50% barrier last year, which is good, but still a long way to go, of course. Should you use it? Yes. Should you just tell your network engineer or your sysops guy, enable IPv6? No. Be careful. On any Linux environment, and I think on Windows, it's the same thing, firewall rules are set per protocol. So if you have a firewall rules set for IPv4, it's not activated for IPv6. So if you just enable IPv6, you're going to run into trouble because basically, you have no firewall rules enabled at all. So yes, you should use it, but you should look at your firewalling first. OK, so we talked a lot about IP addresses. Now let's see if we can send some IP traffic on a local network. So we have a client here, and we have a server, and they're on the same IP range. And the client wants to make a connection and fetch information and so on. The first thing he has to do is, well, they're on the local network. The only way they can exchange information is over internet. So they need an internet packet. And in that internet packet, they need a MAC address. So he's going to send an ARP request and say, does anyone know the MAC address for that IP? And that server is going to reply and say, oh, this is my MAC address. And now they can talk because now they can exchange internet packets. And within those internet packets, there are IP packets and so on. Now, how do IP packets find their way when they want to go beyond that, when they want to go anywhere, really? How do they know whether they're on the local network or not? Well, they use routing for that. Each layer three network, so each IP network, has a routing table. And you can easily find that on Linux. You just type route. On Windows, you type route print. And then it gives you output that looks a little like this. And what you can see here is that, yeah, OK. So you can see at the back, there's iFace interface, which is basically the network interface that you're plugged into. So we have two of them here, ETH1, ETH0. And you can see here there's a difference between this route and that route, just with a flag. So the one that has no G in there is basically one that's defined by your network interface. The one that does have a G has a gateway address. This one, all the IP addresses on this interface are directly available on the ETH0. This one is available on the ETH0 as well, but only through a gateway. That means any packet that is being sent to this range is actually being transmitted to that gateway, so using the MAC address of that gateway. Now, the top one is a special case. It has destination 0, 0, 0, 0, which basically means anything else. In most cases, this means the internet. So anything else is being sent to 10, 0, 0, 0, 1. Now, where does 10, 0, 0, 1 live? Oh, it lives on the second range, so it lives on ETH1. So this is a way to see, actually, where is my traffic going to go. So if you want to send IP traffic to a remote device, where do you send it? Well, if that remote device is not on your local network, then you cannot send it, because you need a MAC address of the device. And MAC addresses are not shared beyond the local network. So then you use that routing table, and you send it to the default gateway. The default gateway is something that's provided usually statically on a server. So you just set it up. Your network engineer will say, this is the default gateway. Or if you have a router at home, for example, it's provided by that router using DHCP. And so what happens then is if we want to communicate from our client to our server somewhere on the internet, then we're going to say, OK, I want to communicate to the internet, which means I have to use my default gateway, which is that router over there. I'm going to ask for the MAC address, because that router is on my local network within the same IP range. I'm going to get that MAC address back. Now I'm going to send an ethernet packets to my router, not to my server, to my router. And within there is a TCP packets for 194.714, the server. The router is going to get that packet, say, oh, I've got an ethernet packet. What's inside this ethernet packet? Oh, there's an IP packet. I have to send it to the server. OK, I'm going to make a new ethernet packets for the next router. I'm going to put that IP packet in there. I'm going to send it along, and the next router is going to do exactly the same thing. Until in the end, it's going to end up on the server on the other side. So you can imagine if you're sending a packet to the other side of the world, and it's crossing 20 routers, this happens every single time. And then the return packet, same thing. Yeah, you can see the full ARP table on any operating system with ARP-A, and I already said route-n or route print. So let's have a look at TCP packets, because we use those all the time. HTTP is sent across TCP. So this is what a TCP packet looks like. The content is here. Above that is a lot of headers, and they're all important. But I want to focus on three now, which is sequence number, acknowledgment number, and flag. So what happens when you try to establish a TCP connection between a client and a server? The first thing that happens is a client sends a packet, and in that packet is a flag syn. The flag syn is set, and it sends a sequence number, which is generated in a certain way. It's sort of random. Let's imagine right now that it's a thousand, and it always increments. So it sends it, and if the server receives it, the server is going to reply. And if it wants to accept that connection, it's going to reply with a packet with two flags set, syn and act. And the sequence number is now one hire, and there's an acknowledgment number saying, yes, I agree. I acknowledge your message, also with a number. And then the client is going to reply again with a flag act. I acknowledge that you acknowledged with the sequence number one hire and the acknowledgment number one hire. And now we can communicate. Now the client can also say, I'm sending a get request. So this is called a three-way handshake, going back and forth. Now let's imagine we want to communicate between Brussels and Montreal. And let's imagine that the minimum time for that is 45 milliseconds. That means after 45 milliseconds, my sync packet has arrived. 90 milliseconds here, I send a get request. After 135 milliseconds, the server has finally received what I want the server to execute, a get request. I want to retrieve some piece of information. That takes a long time. Imagine we do it between Brussels and London and it's 10 milliseconds. Then obviously that only takes 30 milliseconds in total. So you can see why sometimes it's a good idea to have more than one location for your data so that clients from the other side of the world can actually access it a lot faster. So that's one way of improving speed. There's another way. It's called TCP window size. For anyone running on older version of Linux and older Linux kernel, this is the reason why you should upgrade. So you have a sync, you have a sync act, and the server can actually mention a size of packets, the maximum amount of data that it can receive. So it can say eight kilobytes. And then the client can say, yeah, I agree. I can also receive eight kilobytes. And then the server is going to reply back and say, I can also receive 16 kilobytes if that's fine. And the client is going to reply back with 16 kilobytes and so on. And it can go with IPv4, it can go up to 64 kilobytes. But the bigger that size, the larger the packets are gonna be. So if you have a lot of data to send, you're gonna be able to send it in less round trips. Because every single time, let's imagine the server has to reply with 500 megabytes of data. It's gonna be one packet at a time in this case. So it's gonna be eight kilobytes, eight kilobytes, eight kilobytes, eight kilobytes. If you have 45 millisecond delay between the two, it's gonna take a long time. So if you can increase the packet size, the number of times, it's gonna reduce the number of times it has to go back and forth. Recent Linux kernels have a higher standard value for that. There's also something called TCP Slow Start. So what that does is it allows the server to say, oh, I'm sending two packets back already. And I'm waiting for you to acknowledge both. And if you do that, I'm gonna send you four packets. And if you acknowledge all four, I'm gonna send you eight. And so it keeps increasing, keeps multiplying by two until at some point, the client doesn't reply anymore or doesn't reply to all of them and then is gonna go back and go to the previous value. So in this case, it's not a matter of sending one packet, one reply, one packet, one reply. It can actually do four or eight or 16 or more. This is also by default on newer Linux versions. So a good reason to upgrade to the latest versions. So there's a big difference between, and this is very important, this is something web developers can change on the server actually. There's a big difference between establishing a new connection or reusing an existing one. So if you have a new connection, well, it's obvious. It's SIN, SINAC, ACK, and then at the same time we send our GET request and then the server after 135 milliseconds, the server says, okay, now I can start processing. Let's imagine that takes 100 milliseconds, means after 235 milliseconds, the server can reply. Let's imagine that we have TCP slow start enabled and we can immediately send four and we acknowledge with four packets and then the server sends eight packets and we acknowledge four with eight. And let's imagine the whole bit of data was 12 packets. So we're done. Still it took 415 milliseconds, almost half a second for a single request. And normally you would cut the connection here. You would cut the TCP connection and that would be it. But let's imagine that you still want to retrieve CSS and JavaScript and all those things. If you cut the connection, you need to establish a new one. You need to have a new three-way handshake. However, if you don't cut the connection and you can do that in Apache and Nginx and any other web server nowadays, if you don't cut the connection, it means the client doesn't have to do a three-way handshake. You can just say, give me the data. It still requires 100 milliseconds of processing. That's your PHP doing its work. But now the server can reply immediately with 12 packets because he knew from the previous request, eight was not a problem. So we're gonna try to send 16 now, but we had only 12 packets of data. So it's gonna send 12 packets and the client will acknowledge. So we just saved almost 50% in time. So don't close your connections, obviously. So there's a couple of ways to upgrade your performance. Upgrade to the latest version. Check your window size, whether it's properly set. Reduce latency while move your servers closer to your client. Well, if you know, for example, you're building a website and it's gonna be used in the UK, don't put your servers on AWS in California. It makes no sense. Put them on the European instance of AWS. If you have clients, both in the UK and Japan, it might make sense, a lot of sense, to have two instances, one here and one in Asia somewhere. And reuse established connections already. That's an easy fix that's going into your web server configuration and just changing that and restarting it. So this is for standard TCP connections. It also is the same thing for anything that has to do with HTTPS connection because you have another three-way handshake for anything that's SSL or TLS. Now, again, if you close the connection at the end, the next time you do a request, you will have to do a three-way handshake for TCP and for the SSL encryption. However, if you keep the connection open, you can use something called session resumption which means you can actually resume in one less back and forth. So you would gain here 90 milliseconds per request. And I'm guessing most of us are now using HTTPS on most of the websites, I'm hoping at least, given the fact that certificates are now free, basically. There is actually a good way of making sure that your clients are always going on HTTPS and it's a good way to make sure that they are always secure and they're not leaking any information. There's a thing called HSTS, HTTP Strict Transport Security. It's basically all you have to do is send a header on the first request. And what happens is it tells your browser, it tells the client's browser, hey, this website is HTTPS only, switch now and remember it. And even then, if you try to type HTTP, I don't know, HTTP Google.com, the browser will actually add an S automatically. It will not allow you to go to HTTP anymore for those websites. So for example, it prevents that you leak session cookies over unsecured Wi-Fi because you forgot to type the S. And it's just one header that you need to add. Okay, so we talked about TCP a lot because, well, that's what we usually use on HTTP. But there's also another thing called UDP. Officially, it's called User Datagram Protocol. It's also called Unreliable Datagram Protocol. And the reason it's called unreliable is it's because it is connectionless. It doesn't establish a connection. It doesn't do three-way handshakes. It doesn't care about that. It just sends packets. So it has a very simple packet structure. It has a source port destination port, which we'll get to in a minute. It has a length, a checksum, it still has that. And then it has contents. There's nothing else in here. There's no flags, there's no sequence numbers or anything like that. It just sends data. This means that the packet might not arrive. We don't know because we don't wait for acknowledgement. It also means that the second packet might arrive before the first packet. So how do you deal with data that you get then? Well, that's up to you. That's basically the protocol you implement within UDP. You handle some kind of sequence number in there or you decide that if it doesn't arrive within five seconds, you drop it. So that's the reason it's used a lot for streaming or gaming, things like that. Now both TCP and UDP have source port and destination port. Now we all know destination port, I think. That's TCP port 80, TCP port 443 or UDP port 53 for DNS. The source port, however, is equally important because if you connect from a client to a server three times simultaneously to port 80 and the server wants to reply, well, if he just sends data back, then for which request was that data? So what's going to happen is the client is actually, when opening a TCP connection, is going to use a source port. It's going to say, I'm sending this request to port 80. I want this URL. And I'm sending it from port 5000. So if you send an answer back, please send it to port 5000 so that I know that it's the answer for this specific request. If you want to see the active connections, so if you want to see which ports are being used for that, you can do netstat-n, and it will give you a huge list. Don't do it on a super busy server because you'll get a massive list. That's why the dash-n is there so it doesn't do DNS lookups. All right, let's say we want to fetch a website because we're all web developers, so we need to fetch our website, for example. Now, TCP, of course, doesn't know what cu.be is. It doesn't deal with letters. It only deals with IP addresses. So we need an IP address. We need to look it up. Now we need to open a socket, which we haven't discussed yet, which we'll do in a minute. So we'll connect to this IP address on port 443. We'll send an HTTPS request. We'll get our data back. We'll get images, CSS, JavaScript, preferably over the same connection. We'll close the connection and we'll show the web page. Now some of these can happen simultaneously. But I want to talk quickly about DNS and I want to talk quickly about opening a socket because I'm sure most of you know how DNS works, but I want to mention it anyway, a few minor things in there. So there's two types of DNS servers. There's the authoritative server, which is basically the server that's responsible for the domain name. And then there's recursive server. The recursive server doesn't know anything. It always needs to ask an authoritative server and then it keeps the answer of that server for a while. It caches it, defined by the time to live. Usually you will use a recursive server provided by your ISP. So what happens is our client wants to go to our website, it's gonna do a DNS lookup, ask the recursive DNS server for the IP for cu.be. And the recursive DNS server says, I haven't got a clue. It even thinks, hmm, .be, I have no idea what that is. If it doesn't even know .be, it's gonna ask, it's gonna say, okay, I'm going to ask .be where that is, I'm gonna ask it to the root DNS server. Now there's 13 of them worldwide. There's, I mean, there's 13 IP addresses for them. There's a few more servers for them, of course. And so whenever you have no clue what even .be means or .co.uk means, it's gonna go to a root DNS server. And it's gonna ask, can you give me the IP for cu.be? And they're gonna reply, nope, I don't know, but I can tell you the IP for the .be DNS server. You may be, maybe you can ask them. And so, okay, we go and ask them, we ask the .be DNS server, can you give me the IP for cu.be? And they reply, nope, we don't know, but I can tell you the DNS server for cu.be, you can ask them. And so then we ask them and then finally we get the answer. So there's a lot going on here, just for one single DNS request, which explains why sometimes a DNS request takes a little time. So there's a couple of different ones also, it's quite important. There's an A record, which is what we usually have, which basically points to an IP. There's a quad A record, which points to an IPv6 address. So nowadays, if you're in your browser and you have an IPv6 connection, you type Google.com, it's first gonna look up IPv6 address. And if it doesn't get one back, then it's gonna look up an IPv4 address. So another reason why sometimes it takes a bit longer. There's a CNM record, which is just an alias. There's MX records, which define mail servers. There's the DNS server records. And then there's the TXT records, which are usually used for anti-spam. And there's a whole lot more, I'm not gonna get into those now. Two useful tools there are dig and NSlookup, where you can use to get complete listings of what is present on the DNS. Now each DNS, each domain should have at least two DNS servers. The order is not important. It's gonna pick one randomly. And DNS is UDP based. Now that's kind of a good thing, because it means we don't need to establish a three-way handshake. So that's faster. But it also means we don't know if the packet arrived. So we get no acknowledgement. So the only way to fix that is by having a timeout after X number of seconds, usually two seconds. And then we try another DNS server. It can work over TCP, but it's not used a lot anymore. Most people use UDP nowadays. Okay, so that's a bit more about DNS. And then I said I wanna talk about sockets quickly. I'm sure everyone's seen sockets in the sense of an error on the MySQL socket or something like that. This is basically the same thing. With MySQL it's usually a file socket if you're connecting locally. This is a network socket. Basically it's a layer between your application and a protocol so that you don't have to write the protocol yourself anymore. So it abstracts the syntax. You can switch from UDP to TCP more easily and so on. And it basically converts the data you wanna send to packets and vice versa. That's all it does. Again, if you wanna see what is open, netstat-n will show you all open connections, but also open socket connections also locally. So let's imagine I showed this diagram before so we're sending information to the internet. We know now how we can go from our client to our router and then our router will do something. It will forward it somehow. But what happens on the internet? I mean, we all work with it every day. What actually happens? How is that data being transmitted? So there's a protocol there called BGP. Bordegate to a protocol. And it decides how packets are being sent from one place to the other. And it uses a system called autonomous systems. Each network on the internet has an autonomous system number. It's just a number from one to, well, it used to be 64,000. Now it's a bit higher even because there are too many networks on the internet. And BGP will actually use so-called announcements. So each autonomous system will basically tell other autonomous systems, hey, you can reach this IP subnet through me. And they will receive that and they will communicate that further upwards. So we are announcing to those two networks and they will continue our announcement to the next one, to the next one, to the next one. So at some point, every router on the internet knows where every single IP range can be found. That means there's over 200,000 IP ranges in a big table in every single router on the internet just to be able to find out where to send packets to. So what that means is if our client wants to go to our server, our client is connected to AS number one, for example. That router is going to say, oh, you want to go to this specific IP? Oh, I know where that is. I need to send you there. And then that one is gonna do the same thing and gonna send you there. And then finally that one's gonna say, oh, I know where it is. It's on my local network. So I'm gonna send you there. And then the reply comes back and now it's perfectly possible that that router is gonna say, yeah, I can send you to AS five, but that link is too expensive. We pay too much money for it. So I'm gonna send you there. So the returning packet follows a completely different path, perfectly possible. And actually these two routers can be in one network. And so you can have multiple connections with the same network. That's fine. Works just fine. So BGP basically looks up, based on the IP address, it looks up the autonomous system number, then picks the shortest path. If there are two shortest paths, then it's gonna calculate it based on preference, meaning usually money. And then it's gonna send the packet along to the gateway. Okay. Quickly wanna talk about something about mobile devices. So mobile devices switch between networks, switch between towers. If you have a good mobile network, that's not really a problem. If you have a poor mobile network, it can be a problem. Your IP address might change. You might have connections that drop and so on. This means that three-way handshakes also on slower connections, they're an issue. So with people using more and more mobile devices, you wanna avoid having those three-way handshakes as much as possible. The best way right now is to use HTTP too, for a number of reasons, which I'll get to on the next slide. The other way, of course, is keep your connections active, as I mentioned before. So make sure you say keep alive on Apache and Nginx and put that time out in like 15 or 60 seconds or whatever, so that you don't drop connections right away. Another problem, of course, is latency. Connections being slow is an issue with mobile devices. Now HTTP is what we use every day. There's a relatively new version. It's been around for a few years now, but most web servers are not ready. I mean, haven't been upgraded yet or haven't been enabled yet. Originally, this was done made by Google. What it allows you to do is it allows you to make simultaneous requests, HTTP requests over one connection, and get simultaneous responses as well. So you can do five post requests and two get requests simultaneously. It's not an issue. It's binary, which means it's a lot more efficient. It also means that you cannot just debug it as easily because it's all in binary and it's not very visible. It's not easy to read. And encryption is kind of standard. You can disable it, but nobody does, and why would you? You can also even prioritize. You can say, okay, do this post request, but first do this get request because that's more important. And it supports server push, which is useful for notifications and stuff like that, so it's a good alternative in that way for WebSocket-type applications. Try it out, deploy it. Getting it running, very easy. You add a module, which you probably already have. You add one line of code, one line of configuration, and you're done. With Nginx, you just add HTTP2. And then, yeah, you have to, of course, add an SSL certificate as well and make sure it's set up properly. But that's all you have to do. And all of a sudden, you can have multiple connection, multiple requests over a single connection, and they are a lot faster. Yeah, some tools you might want to know on Linux. These are just, I'm going to put the slides online afterwards, I'm joined in, definitely. So these are some of the tools you might want to get to know if you don't know them already. So on Linux, IP, Adger is one of them, shows you a whole lot of information. IF config shows you pretty much the same thing, little bit less details, but then again, it also shows you how many packets, how many bytes have crossed the interface, how many errors you have on the interface. Route-n shows you the routing table. Netstat is really useful for debugging not just what active connections you have, but also if you do dash P, sorry, dash L, dash P, it shows you which process is listening on which port, which can also help sometimes to see what is going on, why am I getting errors, why is it saying that the port is already in use, and so on. TCP dump allows you to inspect packets, and there's a very nice tool called Wireshark, which allows you to actually look at every Ethernet packet, see what is inside that packet, and then you see an IP packet and TCP packet, and even it can even decode HTTP packets and tell you, okay, I got this get request and this response, and so on. So it really helps to see what's going on in the network. Okay, I wanna illustrate why I'm doing this talk is basically to help developers in realize what is going on in the network, and I wanna illustrate that with a little example that we had with a customer of ours. I'm not gonna name the customer. They had a pretty big website at the time, it was 150,000 visits at the time, that was a lot, and they had like a ticker on there. That ticker was actually an XML feed that came from a website that they also owned, and it was cached for 15 minutes locally so that they wouldn't hit it all the time. Now there's a piece of code here. So we create a cache file, we check if it's older than 15 minutes, then they deleted it, then they did a file get contents of the original feed, put it back in the cache file, parse the feed, and so on. Anyone tell me what's wrong with the code? What happens if you get two requests at the same time? Yeah, it's gonna go in there twice, and you might get some weird behavior, very good point. Yeah, what else is wrong? Exactly, you delete it before you're gonna get the file so you don't know yet if that get is gonna be successful or not. There's a lot of things that are wrong with the code, and I'll walk through all of them, but you are getting to a good point there. So the feed was located on that machine, and then the actual website was located here, and it was retrieving it every 15 minutes, and then that data center lost power, power failure, and first couple of minutes, everything was fine because the feed was still cached, and then the feed expires, the cache expires, and now it's trying to fetch that URL. Everybody's going into that if structure, and well, you time out after 60 seconds because of course that's the default in PHP. Really, every visitor on that busy website waiting 60 seconds, no, refresh, refresh. My Chrome is stuck, I'm opening a new tab, I'm opening a new browser, so more load, more load, more load, until at some point they were using Apache, they hit the maximum connection limit. And now it's not just that single page that's down, it's their entire website that was down. And so we get called in and we look at the website and there's 400 Apache processes, zero load, because they're all just sitting there waiting for some file to be returned. So what's the issue? Well, oops, yeah, this is the fix they came up with, by the way. They made a time out of five seconds and then, yeah, that doesn't really fix the problem, does it, but in the meantime, what else is wrong here? Well, of course, we shouldn't unlink because it's possible that 15 people go into the F structure and then if you unlink, while someone else just did the file put contents, everyone else is going back into the F structure. So that's a bad idea. Now, the other thing is, if we have a time out, what does file get contents return? Any ideas? It returns false. So we would actually take false and put that in the cache file. That's not gonna parse very well, of course. So we wanna somehow assign that to a variable and check for it and so on. Then another thing is, we are doing parse XML every single time. Now, you might wanna do that once and then put that content into the cache file. Different thing, of course. There's one more thing here. File get contents and file put contents in PHP are not atomic operations. So you can actually do file put contents with two people at the same time and that's gonna cause the file to be corrupted unless it's a very small file. But if the file is huge and it cannot do it in one right, it's actually gonna corrupt the file. Plus, somebody could be doing file put contents, someone else could be reading the file. It's also gonna be a corrupted file. So that's kind of an issue as well. Basically here, you're relying on the user. It's the user that's going to initiate this piece of code and that's a bad idea, especially with concurrent requests. So it would be better in this case to run that cron job every 15 minutes or so. Now, if we're talking about network resources in your code, use timeouts for everything. Whether that's an F open or a curl request or a soap request or whatever it is, use a timeout. Don't rely on the standard timeout of 60 seconds. In this case, if you own that other data source, just do the opposite direction. So get the feed instead of pushing it the other way. Now, if you wanna ever simulate that, like how does my website behave when we have a network issue? There's a couple of tools that might help. There's one tool that's called Wanim. It's written by an Indian company. It's basically a visual tool and you can just say, oh, I want to introduce 5% packet loss on the link and I wanna see what happens or I want to introduce a two second delay everywhere or two second delay specifically on that XML feed. This is a very interesting tool to use and it makes you actually prepare for what could possibly go wrong. On Linux, there's a couple of tools for it. You can just use standard IP tables and just say, for example, drop 0.1 and that basically means 10% of all packets are now being dropped. Or you can use TC, which is traffic control and say, okay, I wanna delay of 50 milliseconds with a variance of 20 milliseconds. And so then you can see how it behaves if your network connection is erratic or slow and so on. There's a very nice tool actually called Comcast which is described as simulating shitty network connections so you can build better systems. A really nice tool that allows you to actually simulate what could go wrong if anything that you connect to or that your users need to connect to doesn't behave as expected. Yeah, I just wanna show a few more funny pictures, of course. So if it looks like that, your data room, it doesn't show very well, right? But that's before and after, it looks a little bit better now. Finally, last slide, and I'm just in time. Failover or disaster recovery are great if they work. But when was the last time you tested it, of course? Yeah, if it doesn't work, it should be top priority. Like if you have a master slave setup and you wanted to, on MySQL, you wanted to failover automatically or you have a network setup that needs to failover automatically. Fix it right away. I mean, that includes network failover. Anything really, like even a system restore from backup, it should be fixed. It should be top priority in any case.