 Welcome. I'm Tonstier. Thanks for coming to this talk. Give me a lever long enough and a fulcrum on which to place it and I shall move the world. Humans are the cleverest animals in the universe but our brains forged over millions of years in the unforgiving furnace of natural selection are still primitive. They evolve to be good at hunting animals and making tools and recognizing facial expressions and maintaining social bonds but they're just not very good at working with several difficult ideas simultaneously and they're particularly bad at reasoning about the counter-intuitive emergent behavior of complex non-human systems. That's a problem because complex non-human systems now dominate our world. Abstraction is our solution to this problem. Abstraction works as an adapter between the fearsome complexity of the universe and our simple primate minds. When the real world is too detailed or too confusing or too counter-intuitive for us to work with directly, abstraction gives us big friendly levers to pull on instead. This one powerful idea underlies computers, design, culture, language and everything else we rely on in our daily work. So I'm going to talk a bit about abstraction, where it comes from, what it's for and how we can use it to make our programs better. Let's begin by talking about numbers. Numbers seem obvious and natural to us but they have no independent existence. They're a human invention, a human technology designed by us to solve problems we have. There was a point in our cultural history where we had no concept of numbers but then we created them and gave ourselves a new and powerful tool for categorizing, understanding and predicting the real world. Imagine you're living in a prehistoric time and the concept of numbers hasn't been invented yet. You've grown some apples, here they are, which is great but you've been eating apples every winter for years and you're kind of tired of them now. Fortunately a visitor arrives from the next village and they have some bananas. I accept that the botany of this story might not be historically accurate. Now you feel that swapping an apple for a banana is a fair trade and you would like as many bananas as possible so are you prepared to swap your entire collection of apples for the visitor's collection of bananas? Is that fair to both of you? Well yeah it is fair because this collection of apples has something in common with this collection of bananas. There's a special property that they both share even though they're made of different fruit and that's a property that your apple collection doesn't share with this collection of bananas or this one or this collection of apples even though they're the same kind of fruit or this one. The special property shared by these apples and these bananas means that they can be paired up. Each apple goes with a banana with no apples or bananas left over. For example this apple can be paired with this banana, this one with this one and then this one with this one. It doesn't matter how they're paired up. We can come up with other collections that have this property too. This collection of cherries has the same property. We can pair them up with the apples like this. Every apple has a cherry and vice versa nothing gets left over. In fact it's fairly obvious that the kind of fruit is irrelevant. Being a fruit at all is irrelevant. This collection of dots has the property as well. So if it's irrelevant whether the things in the collection are fruit or dots or whatever then what else is irrelevant? Well the size and arrangement of the things in the collection doesn't matter either. This collection of dots has the same property and so does this one and so does this one. What do we call this property? How do we identify the property that those apples and those bananas and those cherries and all those different collections of dots have? Our usual name for it is three. Now technically speaking what you're seeing up here isn't actually the number three. This is just a pattern of light made of pixels. That pattern of light represents a specific glyph from a typeface and that glyph represents a character in this case a numeral also called a numerical digit and that numerical digit denotes the number three in the base ten positional numeral system which is the usual system used in western culture. The number three itself is this property, this relationship, this abstract structure. If you've got a collection with three things in it you can put it on one side of this diagram and connect each thing in the collection to a dot on the other side. Three is our name for all the things that look like this. That's what it means to be three. Now unlike our imaginary prehistoric person you learned about three when you were very young so you already have a picture in your head of what three looks like and it's probably like one of these arrangements of dots. You can recognize other collections of three things by pairing them up with the dots in your mental image. Once we have the idea of a number we can build other concepts from it. For example we can identify the particular relationship between these collections which is that when we pair them up the right hand collection has an apple left over. When this happens we say that the number represented by the right hand collection is the successor of the number represented by the left hand collection in other words four is the successor of three. That gives us a way to generate new numbers. The successor of three is four and the successor of four is five and the successor of five is six. And that suggests another relationship between numbers when two numbers are connected by an unbroken sequence of successes like this we say that the first number is less than the second one. Once we have the idea of successor we can also use it to build a new operation on two collections. For each item in the first collection we take the successor of the second collection. We start with two apples on the right, the successor of two is three, the successor of three is four and the successor of four is five. And that operation is called addition. We've added three to two to get five. The point of all this is that numbers and operations on numbers allow us to predict the outcome of events in the real world. You don't need to physically pair up your fruit with a visitor's fruit to decide if it's a fair trade. You recognize that they have three bananas and you have three apples. You don't even need to see their bananas. They can just tell you that they have three bananas and you'll be able to decide. If you want to know whether any of your sheep have escaped when you go to round them up on the hillside you don't actually have to search the whole mountain just count them and you'll know. Likewise for addition. If you want to know what the combined contents of two buckets of apples will look like you could physically combine them. Or instead of working in the real world you could step into the abstract world, count each bucket to get two numbers and then add them to get a result. That result is valuable because it's the same as what you get if you actually do the work in the real world and then turn the result into a number. The abstract operations in the world of numbers accurately predict what happens in reality by focusing only on those aspects of reality that affect the outcome. This technology is so groundbreaking useful that it just becomes second nature and we get used to working in this abstract world. Numbers feel very simple and obvious to us. But they only exist because people observe the real world, spotted patterns in it and built abstractions to represent those patterns. Of course once we have numbers we need to develop other layers of abstract ideas to make them easier to work with. Here's an example taking the sum of lots of numbers. What's the sum of all the numbers between 1 and 10? That's not too hard to work out by hand. 1 plus 2 is 3, plus 3 is 6, plus 4 is 10, plus 5 is 15, plus 6 is 21, plus 7 is 28, plus 8 is 36, plus 9 is 45, plus 10 is 55. So the answer is that 1 plus 2 plus 3 all the way up to 10 is 55. But here's a harder version. What's the sum of all the numbers between 1 and 100? It would take us at least 10 times as long to work that out by hand. There's a story about a child prodigy called Carl Friedrich Gauss born in the 18th century. The story goes that his primary school teacher told him to do this sum as a punishment but he came up with the answer almost immediately. Now that story might not be true but it could be true because there's an easy way of working it out. Let's go back to the 1 to 10 example. One property of adding is that it doesn't matter what order we list the numbers in. We still get the same result. The fancy way of saying that is that addition is commutative. So we can add the numbers from 1 to 10 in a different order and it won't affect the answer. Let's rearrange them. Now they're listed slightly differently. It goes 1, 2, 3, 4, 5 down the left side and then 6, 7, 8, 9, 10 up the right side. But all the same numbers are still there so the answer will be the same. Another property of addition is that if we're adding many numbers it doesn't matter what order we do the individual additions in and the fancy word for that property is associativity. So we can choose to add together the first two numbers to get 11. Then we can pick any other numbers say the two and the nine and decide to add them next. Then choose the three and the eight, then the four and the seven, then the five and the six. So once we rearrange the numbers into those pairs we found that each pair added up to 11 and now it's easy to see that the answer is 55. Here's how we got there. We rearrange the numbers into five pairs. There are five because five is half of 10 and each pair adds up to 11 which is one more than 10. So here's the secret. When you're adding up the numbers from 1 to 10 there are five pairs of them and each pair adds up to 11 and this is what Gauss ostensibly realized. Or in particular he realized that we can do the same thing for the numbers up to 100. We can rewrite this sum as 1 plus 100 plus 2 plus 99 all the way up to 54 plus 57 plus 55 plus 56. And again by choosing to add each pair first we can see that 1 plus 100 is 101 and 2 plus 99 is 101 and so on up to 54 plus 57 is 101 and 55 plus 56 is 101. There are 100 numbers to start with so there must have been 50 pairs here and although this is slightly harder 50 times 101 is anyone? 5050 thank you. So here's our secret. When you add up the numbers from 1 to 100 that's the same as adding 50 pairs of numbers and each pair adds up to 101. And this pattern generalizes 10 and 100 aren't special. The sum of all the numbers up to any whole number n is half of n times the number one greater than n. If n is odd then you get something in a half on the left but then you'll be multiplying by an even number n plus 1 so the half goes away again. Another way of thinking about this is by looking at it visually. When we represent numbers with dots instead of numerals 1 plus 2 plus 3 all the way up to 10 looks like this 1 plus 2 plus 3 plus 4 plus 5 plus 6 plus 7 plus 8 plus 9 plus 10. That gives us a triangle. But by left justifying the rows of dots we can rearrange that triangle to look like this. And now we can see that it's half of a rectangle. If we fill in the other half we can see that the resulting rectangle is 10 dots tall and 11 dots wide. So there are 110 dots overall and our original dots make up half of that. Half of 110 is 55 so our original triangle has 55 dots in it which is the same answer as we got by adding up the half pairs. Here's what we did. The sum of all the numbers up to any whole number n is the number that's 1 greater than n times n divided by 2. And that's the same as what we got before just with the n plus 1 and n divided by 2 the other way around which doesn't change anything because multiplication is commutative too. Now can you remember all the details of that argument and why it must be true? If not then it doesn't matter because you don't need to remember the details you just need to know this formula. It's a pattern, an abstract tool that you can use even if you don't know why it works. So quick what's 1 plus 2 plus 3 all the way up to a thousand? Well we know that it's half of a thousand times one more than a thousand which is 500 times 1001 which is 500,000 500. Pretty impressive. Well done everyone. Let's have a look at an example of an abstraction that isn't quite so concerned with numbers. This is an old map of the city of Königsberg in Prussia which is now Kaliningrad in Russia. There's a river running through the city with two large islands in it and seven bridges connecting the islands to the north and south banks and to each other. Here's a question about Königsberg. Is it possible to walk around the city in a way that crosses each bridge exactly once? Let's try it. Here's one possible walking route. We can start on the south bank, cross all the way up to the north bank then all the way back down to the south bank then all the way north again. That crosses six out of the seven bridges but it misses out the bridge connecting the two islands. If we try to detour across that bridge at the end we get stuck on the small island and we can't reach the last bridge. Here's a different approach spiraling inwards but again we miss out on one of the northern bridges. If we detour across that bridge we get stuck on the north bank before we're finished. This way doesn't work either nor does this way where we start on the small island and nor does this way where we start on the large one. If we tried this for long enough we become pretty convinced that it can't be done but why can't it be done? What is it about Königsberg that makes it impossible? That question was analysed by a guy called Leonhard Euler in the 18th century. He proved it was impossible and he explained exactly why. To see why let's concentrate on this small island. When you arrive on this island by a bridge you must leave by another bridge unless this is the end of your journey. The next time you arrive by a bridge you must leave by another bridge. The number of times you arrive on this island equals the number of times you leave unless you start or finish your walk there. So unless you want to start or finish your walk on this island it must have an even number of bridges connecting it to the other land masses half to arrive by and the other half to leave by. And that's the case for all the other land masses too. We can start or finish on a land mass with an odd number of bridges but all the other ones must have an even number. But all of the land masses in Königsberg have an odd number of bridges. If we highlight the end of every bridge and then group them by land mass we can see that the small island has five bridges connected to it and all the other land masses have three. They're all suitable places for starting or finishing a walk but a walk can only have two end points so some of them have to be visited partway through the walk. Oiler noticed some other important things about this problem. The first thing was that the route you take on the land doesn't matter just the order in which you cross the bridges so we can disregard everything except the existence of each land mass and the bridges that connect them. Oiler noticed some other important things like the other thing he noticed was that the relative positions of the land masses and bridges don't matter. All we care about is how they're connected. So for our purposes this arrangement is the same as this one which is the same as this one. This gives us a structure called a graph. A graph is made of two things a set of nodes which are the dots here which have no structure at all they just have identity and a set of edges the lines connecting the dots. Edges have no direction and edge is just an assertion that two particular nodes are connected. So a graph is incredibly simple but it's actually a very rich structure and we can identify lots of interesting properties of graphs by thinking about them more. The first property we're interested in here is called degree which counts how many edges are connected to a node. This node for example has two edges connected to it so its degree is two. This node is connected to three edges so its degree is three. The next idea is a walk which is a sequence of connected nodes and edges. As an arbitrary example if we start our walk with this node we can walk along this edge to here then up this edge to here then back down the same edge to get back here then across this edge to here then finally up this edge to end up here. That sequence of nodes and edges that we visited is called a walk. Walks give us a way to decide when a graph is connected. A connected graph is one where you can find a walk between any two nodes that you pick. If we pick these two nodes we can start a walk here down this edge to here then across to here then up to here. So there's at least one walk that connects those two nodes. If we pick two different nodes there's at least one walk between them and if we pick these nodes there's a walk between them two and we can do that for any two nodes on this graph so this graph is connected. If we took away a couple of edges the graph isn't connected anymore because now there's a way to pick two nodes that have no walk between them. The next idea is a trail which is just a walk where all the edges are distinct. You can't use an edge more than once. If we start here we can make a trail by going down this edge to this node but we can't go back up again because now we've used that edge but we can go across to here then up to here then down to here. That's a trail just a particular kind of walk. And finally there's an idea called an Eulerian trail which is a trail that visits every edge in a graph. That may or may not be possible on a given graph in this case it is. If we start here we can go around these three nodes and back again and then go across and up and down and back and we're done. That's an Eulerian trail because every edge got visited exactly once. Now the point of naming all those ideas about graphs is that they give us a precise way of stating the answer to the Königsberg bridges problem. Crossing all the bridges means finding an Eulerian trail in the graph of land masses and bridges. And Euler showed that it's only possible when the graph is connected and has at most two nodes of odd degree because your Eulerian trail can start and finish at nodes with odd degree but the trail only has two ends. Now in a week's time will you remember all the details of that argument and why it must be true. If not then it doesn't matter because you don't need to remember the details. You just need to know this. You can use this result even if you don't know why it works. For example is it possible to drive around the United States covering each stretch of interstate exactly once even if we ignore the interstates that are only connected on one end or that lead out of the US. To decide this by trial and error would take much more work than the Königsberg problem but because of Euler's result we can just look for nodes of odd degree and look I already see one in Portland, one in Denver, one in Buffalo and that's it we can stop. There are more than two nodes of odd degree so there can be no Eulerian trail so that incredibly boring road trip can't happen. Here's one last example, triangles. When you draw a right-angled triangle you can control the lengths of two of its sides and then you have to measure the third side to find out how long it is. It's pretty clear that these three lengths are related somehow. If you make one side longer another side gets longer. If you make one side longer again and another side gets longer again make one of the sides shorter and another side gets shorter. So what exactly is the relationship between the lengths of the sides? It's been known for thousands of years if we call the lengths of the sides A, B and C then the sum of the squares of A and B is always equal to the square of C. This is known as the Pythagorean theorem although the relationship was known well before Pythagoras' time he just came up with a particularly simple way of proving it. Pythagoras realized that when you arrange four copies of the same triangle into a large square that leaves space for a smaller rotated square in the middle. The length of each side of this square is C so its area must be C squared. But by rearranging the triangles inside the large square you can make space for two smaller squares. One of them has sides of length A so its area is A squared and the other one has sides of length B so its area is B squared. These two arrangements have the same total area and so if we take away the four triangles we're left with some we're left with two squares that must be the same as the area of the larger square. You can draw this diagram for any right angle to triangle so even if you make the triangle fatter or thinner that just corresponds to the right hand square being rotated at a different angle. So that's what Pythagoras proved. Now do you remember the diagram that proves that this is correct? Would you be able to draw it from memory? No. Well it doesn't matter you just need to know this formula now. I've been talking about abstraction but there might be another word that comes to mind when I show you all these things and that word is mathematics. That's not an accident because mathematics is the study of abstraction. I've shown you a few general problems that we can solve by first solving individual examples of the problem and then looking for patterns in our solutions. When we spot a pattern we can use it to design an abstraction that lets us solve any instance of that problem without having to think about or even understand why it works. That's essentially what mathematics is. Spotting patterns and building reusable abstractions. It's pretty popular at the moment to say that you don't need mathematics or computer science to do programming and although I basically agree with what people mean when they say that, I do think that the words they're actually saying are wrong. Most of us have had terrible mathematics education where we've been taught that mathematics is about memorizing meaningless formulas or symbols or mnemonics so that we can pass exams but mathematics isn't about that. It's about building abstractions that magnify the force we can apply with our minds. So it's like saying that you don't need to know music to be able to play the violin. Well if by music you mean knowledge of music theory or the ability to read sheet music notation or other incidental things then of course you don't need any of that to play the violin. You can just pick one up and learn to play it. But that's not what music is. Music isn't blobs on a stave or theories about harmonic intervals. Music is what happens when you play the violin and in the same way mathematics isn't Greek symbols and strange words and remembering how to integrate every trigonometric function. Mathematics is what happens when you simplify the world by building abstractions. Mathematics is what happens when you write a program. Let me show you a more practical example of this. Here's an exercise I often do with Ruby programmers. Write some code that can compare two poker hands and decide which one beats the other. In case you don't know a hand in poker is five playing cards and depending on what those cards are the hand falls into a particular category. The simplest category is a high card hand which is just five cards with no particular pattern. To compare two high card hands you just compare their highest card and if those are the same then you fall back to the second highest card and so on. Then there's a one pair hand which contains two cards of the same rank. This one's got two twos. A one pair hand always beats a high card hand and it beats another one pair hand if the pair card is higher. Even better than that is a hand with two pairs and then there's hands with three cards of the same rank then hands with five consecutively ranked cards then hands where all the cards are the same suit and it keeps going after that there's full house then four of a kind and finally straight flush. So how can we implement all these rules? Well the naive way is to pick an easy way of representing cards like a struct with rank and suit attributes and then use an ordinary number to represent the rank of a card and a symbol or a string for its suit. For the face cards Jack Queen King we can just use the not use the numbers 11 12 and 13 and we can use 14 for an ace because an ace is mostly the highest card. Then we can create a hand class that wraps up a collection of cards and then mix incomparable and start implementing the spaceship operator so that two hands can be compared. To support high card hands the common case is that we just compare the rank of the best card in each hand I'll leave out the necessary helper methods here. If those highest cards are ranked the same we can compare the hands again with the highest card removed so the recursive call will be check in the second highest cards and so on and we need a base case for that recursion if both hands are empty we'll say neither one wins which is what zero means here. To support one pair hands we need to mostly duplicate the code we've already written. First we check if we're comparing a one pair hand with another one pair hand and compare the pairs if so otherwise we fall through and do the previous thing. Then we add two pair hands and three of a kind then straights and so on. The code very quickly gets unmanageable. We're essentially just typing in an extremely large flow chart about how to compare every possible combination of hands. It's possible to do a certain amount of refactoring here to reduce the repetition but that just moves the mess around and tends to produce overly clever, overly metaprogrammed templaty code where it's hard to see where the actual work is getting done. Let's start again and think a bit harder. First off we were using numbers as ranks but ranks aren't numbers not really you can't add them or multiply them. Queen isn't a number they're mainly just unique identifiers. So let's make an empty card rank class and create inside it 13 constants one for each rank. Now a rank is just an instance of this vanilla class so it doesn't have any special behavior or structure. We can do the same thing with suits. We don't need them to actually contain the name of the suit or support any special methods they just need separate identities. And now creating cards look like this. If it's the sort of thing you like you could put a cute of method on the rank class so you could create a card by writing queen.of hearts. We do need to be able to compare the ranks so let's add the structure to the rank class so that we can do that. The ranks are already declared in ascending order so we can just remember that order and then write a spaceship operator that compares them by their position in that list. Now briefly we can introduce another class to represent the different categories of hand. We do a similar trick with this one a constant to represent each category and a spaceship operator to compare them. Then we can add singleton methods to each category. The matches method decides whether a hand meets the requirements for that category and compares two hands in that category. We don't have time to go into the details of how that works but there are some other interesting abstractions to discover in there. Those methods are different for the one pair category and so on for two pair, three of a kind, straight and so on. Finally we can make a hand rank class. This is a decorator for hand objects. It adds the ability to find the best category for a hand which lets us compare two ranks according to the order of categories and the special per category rules. So when you have two hand objects you can compare them by getting their respective rank objects and comparing those. It's good to keep this separate because two hands having equal rank doesn't necessarily mean that the hands themselves are equal. Those are different ideas. For example they might be royal flushes of different suits. As before we can enrich these simple abstractions by building operations on top of them. For example we can give card ranks a new operation called SUC for successor that just returns the next highest rank. So SUC of ten is Jack and SUC of ace is nil because it's the highest rank. Notice that Fixnum already has its own SUC method that we can use here. That SUC operation gives us an elegant way of identifying a straight which is a hand containing five cards of consecutive ranks. This one has five, six, seven, eight, nine. First we take the successor of each card after a six comes a seven, after a nine comes a ten and so on. Then we try to pair up cards of the same rank between these two hands. This six goes with this six, this nine goes with this nine, this seven with this seven and this eight goes with this eight. If and only if the original hand was a straight will end up with one unpaired card in the original hand and one unpaired card in the hand of successes. Even better than that the unpaired card from the original hand must be the lowest card in the straight and if we look at which card produced the other unpaired card as its successor that must be the highest card in the straight which is what we'll use to rank it against other straights. So I'm sorry that was very brief I don't have time to explain it in detail but you can have fun later thinking about why it must work. This technique is beautiful because it distills the idea of a straight down to its abstract essence which is that the cards are of consecutive ranks. As a bonus we also get to identify the highest card without relying on the overall order of ranks. We don't have to sort them or take the max or anything it just falls out. That makes it easy to implement straights containing low aces. We just make suck wrap around so that the successor of an ace is a two and then we can have straights that go ace two three four five and the five will come out as the highest card even though ace is larger than five and then we need to filter out invalid straights where the ace appears halfway through but that's another story. Maybe we would just call this object oriented design but it works by identifying minimal abstractions and plugging them together to get more complex emergent behavior. The final solution has a structure that more closely mirrors the actual rules of poker and you never have to do anything clever. I watched a documentary recently where Richard Feynman said offhand when a thing looks complicated it's possible that we're looking at it wrong and missing some of the pieces of the puzzle. He was talking about particle physics which is a classic example of this. Real experiments produce lots of complex looking results and the challenge is to find the elegant abstract model that predicts them. To a lesser degree we have the same problem when making software or cooking a meal or writing a novel or understanding someone else's feelings. We need abstractions to take apart the complex things from the real world and reconstruct them in our heads so that we can get some purchase on them. All right it's time to wrap up. How do you make use of all the stuff I just said? Firstly when building software you should make time to find the honest abstraction. It should go without saying that the wrong abstraction is worse than no abstraction at all but the right abstraction in the right place can make the difference between an incomprehensible solution and an effortless one. So pay attention to your domain take a step back from the detail look at the overall shape of the problem and separate its essential features from its incidental ones. Once you recognize the true structure of your problem you can allow the answer to naturally assume the same shape. If you've ever seen two developers write almost identical codes to solve the same problem this is probably why. They both saw through the fog and found the problem's underlying structure and that structure guided them towards the suddenly obvious solution. That takes time but it's worth it because it feels so good when you build an honest model of your problem and the solution just falls out when the abstraction works like a lever like a crowbar magnifying the weak effort of your mind at the surface of the problem and producing a more powerful force deep down below. It's also important to find the minimal abstraction. Even a very simple abstraction like a number or a graph or a triangle can actually be very rich and support a lot of complex ideas. So when you're making abstractions start from nothing and only add structure and behavior that are absolutely essential. This is especially important in a dynamically typed language like Ruby where you have to think of the public API of an object as a sort of attack surface that's exposed to every other object in the program. So keep the number of ideas and operations as small and as honest as possible and your codes more likely to be good. Trapped inside every boring problem is an interesting problem waiting to be discovered. A lot of problems appear superficially boring but your code will be better and you will be happier if you take a little time to discover that nugget of interestingness. First you can enjoy it and then you can exploit the hell out of it. Be excited. Mathematics is exciting and beautiful and it belongs to all of us. The most powerful thing you can do with your mind is to strap it into the hulking metal exoskeleton of mathematics and throw complexity straight out of an airlock. Don't waste it because you had a bad experience in high school. But finally remember to keep your perspective. All of this advice comes with a caveat. Don't lose sight of why we write software which most of the time is to deliver value to users not to satisfy our own curiosity and appetite for beauty. The measure of greatness for most software is does it make people's lives better? Not does it contain beautiful abstractions. But those goals don't have to be in opposition either. A bit of both is healthy. Better abstractions can lead to more satisfaction for you and for your software's users. I'd like to leave you with this thought. If you can write a program you're good at mathematics. You might not choose to call it that. You might not even like it but that's the way it is. Mathematics is the study of abstraction and abstraction is what we talk about when we talk about programming. I'm sorry to drop that on you like this. I'm not really sorry. Thanks for listening.