 And I'm here to talk to you today about how you can tell stories with Traceroute. So let's start with what Traceroute even is in the first place. So it's a networking tool. You can find it in both Unix and in Windows, though in Windows it has a different name. And it's a way of showing the path that your request takes through a network and how long it took to get to each stage of that network. You can think of it kind of like breadcrumbs, right? If you're going on an adventure to your destination, you will leave this trail of breadcrumbs that you'll come back and look at later. And it will show you which path you took to get there. It has some pretty verbose and kind of intimidating looking output. You don't need to look at it in too much detail. But we're going to break it down so you understand what's actually happening here. Essentially, this is a Traceroute to github.com, mostly because it has a very informative set of host names. But if you break it down line by line, there's four pieces of information for each step. The first piece of information is the hop number. So this is which router you're at. The second is the host name of the router. The third is the IP address. And then you get three different latency measurements, which is how long it took to get to the router three different times. So you can see whether there's high variability. And if you want to understand the path that your request took through a network, what you really want to look at is probably the host names and IP addresses. So if we take a look at just those in this trace, you can see the path that we took, in this case, from Sydney Airport to get to github. So we started off going through a local router. It's probably a local router because it doesn't have a host name defined. Then we go through Vocus's network, which connects from Sydney over to San Jose. You can see that in the different host names returned. Then from Vocus, we go to Telio, which is an interconnect network provider that goes up the West Coast. And finally, we make it to github. And this is kind of really cool because I can see that my connection from Sydney goes through San Jose to get to Seattle, which isn't necessarily immediately obvious, and is super useful if you live in Australia. Because if my internet goes down, I don't necessarily know which part of the connection between me and a server in the US is actually broken. Looking at over here, we can actually see with those latency measurements where we make the hop across the Pacific. So the first two steps are very fast, only about two milliseconds to get to them. But by the time we're at the third, it's taking 200 milliseconds to get there. That's because it turns out that crossing the Pacific takes a while, even at the speed of light. And I use this a lot because I don't always know if it's my Wi-Fi that's broken or my internet that's broken or the link between Australia and the US that's broken or just the server on the other side. There's a lot of things that could go wrong. So that's how Tracerate is meant to be used. It's meant to be used as this debugging tool, like why did my packets go away? But back in 2013, I read a super cool story about a network engineer called Ryan Wubber, and he actually lives in Boston. And back in 2013, there was some pretty bad snowstorms, and he was bored. And he really liked Star Wars. In case anyone's not familiar with Star Wars, it's very famous for having this crawl text at the beginning of each film that gives you a narrative background, and it's kind of slow. You slowly get to read the story of what's happened in the background context for the movie. So he decided that he would build this, but with Tracerate. And this is pretty cool, because if you look at it, you're getting, like, 270 milliseconds, and it takes a little while for each line to appear, so it does actually kind of seem to crawl up the screen. And I thought this was the coolest thing ever. So I wanted to build my own. But the problem I had was, when I went and looked up how he built it, he's a network engineer, so he has access to, like, commercial grade routing equipment, and he had two different routers bouncing these packets between them so that he could print this out. I wanted my own, but I do not, in fact, have access to commercial grade routers for my side projects. So I wanted to see if I could build this in the cloud with a single server, but that meant that I needed to understand how routing works on the Internet. So at a high level, this is the Tracerate we saw before. You know, we see my laptop connection going to a router, then through focus interior, it's a little bit zoomed out, and then on to GitHub. But this is kind of more like what the Internet looks like, right? It's very interconnected. It's very unclear which path you should take. How do we know which path we should go down in order to get to GitHub? Well, it's kind of like being lost in a foreign country. But without your phone, either it doesn't have data, or it's flat, or it's the 1990s. And so you're trying to get to a friend's house. You don't know how to get there. So maybe you ask a friendly stranger, hi, here's my friend's address. How do I get to this location? And they say, ah, I don't really know exactly where that is. But that suburb, to get there, you need to take the train to Town Hall. So you do that, take the train to Town Hall, and you get off the train. Ask the next person, hi, trying to get to this address, do you know how to get there? They say, oh, I'm not sure exactly where that is, but that's straight. You want to walk north to get there. You keep doing this until you actually get to your friend's house. That's what's happening with Internet routing. My laptop doesn't know how to get to GitHub. But it does know that GitHub's on the Internet, and the best way to get to the Internet is through the router. The router also doesn't know where GitHub is, but it knows that it's in the US, so it takes the focus path. Focus doesn't know where it is, but it knows it's in Seattle, and the best way to get to Seattle is through Tillia. And then Tillia does know how to get to GitHub. So that's what it chooses to do. And this all works fantastically, assuming there's no loops. But you can see that we could run into some troubles if focus and Tillia disagree on where they think GitHub is. If focus thinks it's on the east coast, say because they think GitHub's west coast data center is down, and Tillia thinks it's on the west coast, we can end up with this loop. And then our packets and requests are just going to kind of loop around forever and get stuck. So the way we solve this is by adding what's called a time to live counter. And this is literally just a number on every request you send that gets decremented with each hop. So it might start at, say, 64, or it makes it to the local router, and we'll decrement it by one, because that's one more step. And we'll keep doing this. And if we ever get to the point where the counter reaches zero and we haven't gotten to where we want to go, we'll send a special message called a TTL-expired message back to the person who sent the original request. And this basically says, look, you said to try for like five minutes, and it's been five minutes, and it hasn't got there. So I'm going to give up now. So let's go back to those breadcrumbs. Let's say you want to have breadcrumbs of where your request has been, but you don't want to have to implement anything special in the protocol to get support for this. How could you do this? Well, you can imagine that you can send a packet to GitHub, but set a time to live of zero. It's going to make it one step, and immediately the router's going to be like, I'm all out of time. I'll send you back an expired message. Cool. So then you can print that the router who sent you that expired message was the first step. Then you'll send a counter with one. This time you'll make it to focus one step further, print the message, and then print that you made it to focus. We'll do that for Tealier as well. And then when we get to GitHub, we'll get a normal response back. So we'll print that we made it there, how long it took, and then we'll stop incrementing that counter. And that's actually how Traceroot works. But since Traceroot itself is a hack, I can abuse it with my own hacks. So rather than sending TTL-expired messages just if I get a packet with a TTL of zero, what happens if I also send it for any packet that has a TTL less than a certain amount? Say the amount we want to pretend that we're farther away by. So concretely, if I got a request like this, time to live is zero when it gets to me, but the packet was destined for me in the first place. So I should really be sending a normal response. But instead, I'll send a TTL-expired response. Then when I get a response with a TTL of one, this time, or a request for the TTL of one, this time I'll actually send a normal response. To Traceroot, it looks like this happened. There was this one extra hop along the way, and so it's gonna print out one extra line. Looks a little bit like this if you implement it in like pseudo code Python. So we see two duplicate lines in there. And we can obviously add multiple steps and we can smooth who we send that TTL-expired message from to control the IP addresses that we see in this output. And that's all very well and good, but we said we were gonna tell stories. And stories aren't told with numbers, they're told with words. Which means we need human readable names on these, not just IP addresses. The way Traceroot is defining the host names on those, each of those steps is using reverse DNS, which is like everything else in this talk, a hack on top of DNS. So DNS is very simple, it just lets you map a host name to an IP address. That's how we resolve names on the internet. But let's say that you wanted to know, given I've got this IP address, which host name does it go back to? That's not immediately obvious because it's a one-way lookup. What we could do though is, DNS has this cool property that for a host name, you can strip off any prefix and you'll get to like a broader source of truth basically. So if I wanna know who owns color.io, I can strip off the color and just ask who owns .io. They should know who owns color.io. So if we wanna do this for IP addresses, we could, hypothetically, reverse the IP address, stick a special thing on the end and then do exactly the same mapping, which is what reverse DNS does. It's a total hack on top of DNS. But because it's a total hack, that means there's no actual protocol requirement that those two things match. I can set the reverse DNS record to something totally arbitrary. And not only can it be totally arbitrary, it doesn't have to match the forward pointing record, it can also just not be a valid host name at all, which is what we've done here. So if we put it all together, basically what you gotta do is buy some IP addresses that you control, set reverse DNS on them to whatever you want. I recommend doing this with providers that are less strict. Some providers like Amazon say that you have to have a forward pointing record, but do you find a really sketchy hosting company? Don't have to worry about that. Even with the really sketchy hosting company, I managed to find who would also let me buy IP addresses and not stick them on, like, stick machines on them. Still costs about $3 for each one. So that means that essentially each line of your story is gonna cost three bucks a month. Kind of expensive. This is kind of what it ends up looking like. And now since Julia's thing is beefy, let's do a demo. So live, this is from a server actually in San Francisco because my other server is in San Francisco and it kind of just speeds things up. We can see that we make it to the Palo Alto internet exchange there, and then there's this huge long wait because Amazon refuses to return any information. So we have a bunch of steps that just go missing. Can wait a little bit longer. And there you go. So that's how you can use a single server and a bunch of IPs to tell your story.