 byddwch y cysylltu ym mwyaf, y dweud y ddechrau i unedig ac yna'r ddweud yma. Rwy'n meddwl am ddweud yma a'r ddweud yma'r ddweud o'r sempas. Ond mae'r ddweud yn y ddweud yma, mae'r ddweud yn ddefnyddio'r ddweud, mae'r ddweud yn ddweud yn y ddweud, ac mae'n ddweud i'r ddweud yn ddweud ar y ddweud yma. Mae'r ddweud yn y ddweud yma. A rwy'n ddweud yma, before finally going to the professor and saying we are really sorry. We went to visit some friends last evening and we would have made it back in time except for the fact that we had a flat tyre. Can we please do a make-up test? The professor thought about it eventually agreed and wrote out another test for them. When the time came for them to take it, she set them to separate rooms. They sat down and turned the page to the first question. Easy, they thought! Then they turned the page to the second question. So what would you write in this situation? It was stories like this that got me into game theory. I'm a software engineer at Pivotal labs in London ac rwy'n meddwl o'r cyfnod o'r platform cyllid yn cael dechrau gyda'i cyfnod, a rydyn ni'n cyfnod o'r ddod yn lleasol iawn i'r gwasb. A'n wych i ddweud ddweud yng Nghymru yn ddweud yn swydd ddweud, mae'n fydig i'n dod o'r ddod yn bwysig i'ch gweithio eich ddweud yn y lleisio ar yr unrhyw gŷnol. A'n ddweud i'n cyfnod i'r mynd. Rwy'n amgylch i'r Uneddiadau Ieconomix i'r Ynysgol, a gwirthwyrdd i'w gwneud hynny'n ymwneud i'w ffaintwyd. O'r yw'r aww, dyfodd sydd y gwirthwyrdd yw yn fwyaf symud i gweithio gwirthwyrdd yw'r cyffredin a cyffredin yn y gwirthwyr wrthwyr. Ac mae'n edrych ar gyfer hyn ar gyfer yw'r pwylltyn o'r problemu. So yw'r aww, mae'n gofo'r cysylltu i gwirthwyrdd ar y gwirthwyrdd. Felly mae hi'n gweld wedi gweld y gweithwyr. The first one looks at bargaining and the second one an auction scenario. Then we're going to look at how these simple examples have led to interesting real-world case studies, before it will be time to wrap up. By the end, I hope to have shown you that not only is game theory a useful tool for working with distributed systems, but by using game theory, you can discover how you can efficiently allocate resources in systems that you go on to build or work with. So let's begin with more about game theory. Game theory is the study of strategic interactions between rational agents, and applications include economics, politics, biology and computer science. It was developed by John von Neumann and Oscar Morganston in the 1940s, with many other scholars developing it extensively in the 1950s onwards. One such notable scholar was John Nash, who was sadly killed with his wife Alicia last month in a car accident in New Jersey. So game theory applies to a range of games and interactions, and we typically see it applied to behavioural scenarios. But despite its breadth, all games, no matter how complex, have some basic components. So what is a game? There are at least two players. Each player has a set of strategies based on the information they have and the actions they can take. And there are payoffs to each player for every possible outcome. So we're going to look at games that help us solve a load balancing problem. So we've got a distributed system of two computers, A and B. And we have a scenario where jobs are being sent into a task distributor, which then has to decide how many jobs to send to each computer. So let's look at our first game, Barnley. One afternoon, you're walking along the street with your friend Gemma, and you come across a bag of sweets on the floor. You both want the sweets, and so now you have to negotiate as to how you're going to split them up. Barnley's theory attempts to give a precise answer as to how you will eventually divide up those sweets between you. So another way of saying that is that bargaining theory asks, how will surplus be split between agents? So in this case with sweets, there's surplus to be gained from acquiring some sweets, and the maximum surplus that you can get is equivalent to you getting all of the sweets and Gemma getting none. So one way of answering this question is to look at something called a Nash bargaining solution. So you can plot a chart showing all the different possible payoffs depending on how the sweets are divided up. And the outer edge of the payoffset represents all the possible combinations where the full surplus is extracted. So this means that all of the sweets are divided between you and Gemma. And anywhere in the purple area means that after you split up the sweets between you, you've left some behind on the floor. On the axis representing Gemma's payoffs, we have the case where Gemma takes the whole bag of sweets. And on the axis representing your payoff, we've also got the case where you take all of the sweets. We find something called the Nash bargaining solution point on the frontier of the payoffset, where both you and Gemma have a higher utility than if you'd not have any sweets. But there's nowhere else in this payoffset where you can increase the payoff of one of you without decreasing the payoff of the other. So in this example, I'm assuming that yours and Gemma's payoffs are linear and the same with respect to the sweets. So this means that an additional sweet or two or three in your pocket represent identical payoffs or identical happiness. But often in game theory, examples are more complicated. And the frontier tends to be curved, not straight. And this represents the fact that agents typically have non-linear and differing responses to marginal increases or decreases to whatever is being bargained for. But in this case, we'll assume linear payoffs, and that means that the bag of sweets will be split in half. So this is all well and good, but you're wondering what has distributed systems got to do with this. So we can use bargaining theory to think about how resources should be allocated within a distributed system. So in a system where some computers are heavily loaded and others have a lighter load, we can have cases of poor system performance and we want to mitigate that. So let's return to our distributed system of two computers that need to work together to complete some tasks. So how do we model this? The players are the computers. Each computer has a different processing rate, and we know this information from the beginning of the game. Each computer's strategy is to accept a pre-agreed number of incoming jobs. And the payoff to each computer is related to their final load, and we assume that computers with lighter loads are better off. So we've got two computers, one that can handle a maximum of five incoming jobs every second before it crashes, and another one that can handle two. And together they need to deal with four jobs every second. So here's what's happening. Four jobs are fed in from a main computer. Alongside this, a certain number of jobs are sent to computer A and a certain number of jobs to computer B. And we want to know how many jobs should each computer agree to accept in this bargaining game. We want to distribute the jobs in such a way that optimally minimizes execution time. And what do I mean by optimal? I mean that once we've come up with the job allocation, we cannot reallocate the jobs in such a way that the performance of at least one computer is not worsened and the overall execution time increases. So how do we break this down? We assume that each computer has a payoff function or we can view it as happiness. And in this example, I'm going to say that that is equal to log of x minus y. Where x is the computer's processing rate and y is the rate of jobs arriving at the computer. Now don't worry too much about the log function. It's just more of a way of saying that as more and more jobs arrive at a computer, the response time to complete those jobs increases exponentially. So here's what that looks like for the computer that can handle five jobs every second. If I'm the computer, from my point of view, I'm happiest when I can sit around and not do any work. I know I have to do some, but I want that to be as little as possible. And as the amount of work that you send me increases, I get unhappier and unhappier. But the fewer the number of jobs I have to complete, the shorter the time it takes for me to complete them, therefore, the happier I am. So we want to find the maximum of a function that looks like this. And this is what you get when you take one of those happiness functions for each of the computers, substitute in their processing rates and add it together. And we have the constraint that the total number of jobs arriving at each computer has to equal four, as stated in the problem. So if you have to plot that on a graph, you get something that looks like this. And we want to find the point up here where happiness is greatest in the system. So when we solve for that point, we get that the number of jobs to send to computer A is three and a half, meaning half a job should be sent to computer B. So here's what that looks like. So what might that look like on a bargaining diagram? Here you can see the joint pay offset for the system with a curved outer frontier. And this is the line where all four jobs are being completed. And it's curved because the processing rates of the computer, computers mean that marginal increases and decreases in the number of jobs that they have to handle results in different performance levels depending on which computer we're looking at and its current capacity. So with the allocation of three and a half jobs to computer A, you get that the response time based on the numbers given is 1.33 seconds. But what happens if we were to give a bit more work to computer A and look at this point over here? Well, then the optimal response time becomes 1.39 seconds. And what if we were to give a bit more work to computer B to this point over here? Well, then the optimal response time becomes 1.36 seconds. And what I'm trying to show is that when you calculate the job allocation based on the Nash bargaining solution, then there's no way that you can reallocate those jobs without increasing the response time. So by using game theoretic principles, we can find efficient distributions of resources in our computer systems. And we've looked at an example with only two computers, but we can extrapolate the same calculation to any number of computers. And this enables us to start thinking about algorithms that can dynamically and efficiently distribute jobs as they queue up in a system. And there'll be a bit more on this later on in a talk. But for now, let's look at another game, an auction. So in the end, you and Gemma decide that instead of splitting up the suites between you, you'd be better off taking them to the school playground and auctioning them off amongst your friends. So how does this auction work? We're going to use something called a second price auction setup. So participants in this auction for your suites simultaneously submit their bids to you in an envelope. The person who makes the highest bid wins the suites, but they'll only play the value of the second highest bid. So this may seem like a strange setup, but there are auctions today that use this concept, including eBay and Google and Yahoo's online advertising programmes. So let's look at an example with three participants. Lucy bids $25. Mark bids $5. Helen bids $10. So given the second price auction setup, Lucy wins for making the highest bid, but she only pays $10, the second highest bid. So auction theory asks, how should an agent bid, given varying intrinsic values that the agent has for the object being bid for? So I want to interject now to introduce the concept of a dominant strategy. So a strategy is dominant if that strategy gives the player their highest payoff, regardless of what the other players do. So to clarify, a dominant strategy is not the same as a winning strategy. It does not guarantee that you win and nor does every game have one. And a dominant strategy doesn't always exist. But if it does, then it will guarantee your highest payoff, so you're always better off playing it. And the reason I introduce that now is because I want to show you that in a second price auction game, it's a dominant strategy to bid truthfully. So this means that in the case of the suits, it's in the best interests of the participants to bid the amount that the bag of suits is worth for them. So let's assume that all three of the participants had bid the true value that they held for the suits and that these are their respective payoffs. Zero if you don't win the suits and the true value that you hold for the suits minus the second highest bid. So for Lucy, that's 15 because she values the suits at $25 and only pays $10 for them. So the way a second price auction is set up, there is no way that Lucy, Helen or Mark could have rebid such that their payoff improves. So let's focus on Helen. She's come second in this auction. There's no point in her bidding any lower than $10 because she'll still be losing. And any bid up to $25 again makes no difference. But what if she had bid $26 to move her into a winning position? Well then her payoff becomes minus $16 because Helen only values the suits at $10 in reality but she has to pay $26 for them. So there is no other bid but her true value of $10 that makes sense for Helen. And this result is true for any participant in a second price auction. So we can use auction theory to reveal the resource capabilities of machines within a distributed system. Sometimes there are cases where the computers are owned by self-interested agents and any load allocation algorithms are vulnerable to manipulation. So how can we model this? Again, the players are the computers. Each computer has different capabilities which this time are only known to the computers themselves. And each computer's strategy is to announce a bid. So in this case a bid is going to be a statement about what capability that machine has. So for example a computer can have a memory of 10 gigabytes and so an honest bid would be represented by the number 10 in this example. And the payoff to each computer is related to the result of the auction. So again we've got two computers. Computer A with 15 gigs of memory and computer B with 10. And they're going to enter an auction to win the prize of running a set of jobs in their machine. Computer A will submit its bid in the form of bid A and computer B will submit its bid in the form of bid B. And these bids are going to take the form of a statement of how much memory the computer has. And ideally we want the jobs to run on the machine that has the most memory but the computers may have other motivations. So let's return to our diagram which now has a new component of an auctioneer. First the auctioneer will announce an auction to run the jobs. Each computer will then submit their bids to the auctioneer and after receiving the bids the auctioneer selects the winner and communicates the results to the task distributor who then sends the incoming jobs to the winning machine. Now there's also something else going on behind the scenes. The computers are programmed with payoff functions that mimic the second price auction set up. So let's look at what this means. So this is a rather abstract example and it's independent to the one we looked up earlier. And for now we're going to ignore the burden on the machine of any jobs that it eventually runs. So let's have a look at computer A's potential payoff the one with 15 gigabytes of memory. If computer A does not participate then the payoff is zero. If computer A loses so it submits a bit lower than computer B again it's payoff is zero. But if computer A wins the auction the payoff becomes 15 minus bid B so that it receives a payoff that's representative of its memory but subtracted from that is whatever computer B did. So similar to the earlier example I'm going to show that with this set up is the dominant strategy for computers to truthfully report their memory. So if for computer A it's always in its best interests to bid 15. So first I'm going to demonstrate where computer A will never want to bid higher than 15. So in this first graph computer A has bid 15 and computer B has bid 8. So computer A wins and has a payoff of 7 15 minus 8. If computer A would have bid higher than 15 so let's say 20 then there would be no increase in payoff due to the way that they're structured so there's no incentive to bid higher. But what about the case where computer A is losing at a bid of 15 as shown in this chart where computer B has bid 17? What if computer A had put in that bid of 20 then? Well it would have won the auction but given the payoff structure the computer ends up with a negative payoff. So similar to the sweets example whenever a computer has to bid a higher value than its true capabilities in order to win an auction the ultimate payoff would always be negative meaning that the computer would prefer not to deviate from its true value and now we'll look at why computer A will never want to bid lower than 15. So in this case computer A is losing with its bid of 15 and moving its bid to anything lower say 11 for example will make no difference the payoff remains at zero there's no incentive to move away from that bid of 15 but what about where computer A is winning with a bid of 15? Again it can bid anything lower up until 8 but that will make no difference the payoff will remain at 7 but if computer A had tried to underreport capabilities to say 6 then the payoff goes from 7 to 0 when it loses and so once again there is no better alternative to an honest bid of 15. So with this example each computer will submit an honest bid when asked and since it has more memory computer A will win the opportunity to run all of the jobs so what does this result show us? Using concepts like second price auctions we can design systems that incentivise machines to truthfully report their capabilities meaning that distribution of jobs remains efficient even as resource availability changes so so far we've looked at a couple of examples and made quite a few assumptions for example we've assumed that all jobs and tasks considered are the same but how can we distinguish between one of tasks or long running processes and what about things such as deadlines or wanting certain jobs to run on the same machine and in the auction example we ignored the burden upon a machine of actually running the jobs and just focused on the abstract second price auction payoffs and there were many more assumptions implicit but the simple starting points are key however because not only do they help us to build up models to think around these problems these starting points have already led to interesting real world applications so let's go back to that first bargaining example in a paper called load balancing a game theoretic approach Daniel Gross who shows how to develop an algorithm based off of the Nash bargaining solution calculation say we had a distributed system of three computers A, B and C with the following processing rates and they need to handle seven jobs every second after doing the initial calculation to work out how many jobs to send to each computer computer C returns a negative value in this framework that is equivalent to the computer being too slow to be effective and so we remove it from the system before redoing the optimisation and we repeat the calculation until all remaining computers in the system has a positive value for the number of jobs they accept so the final outcome represents the unique bargaining point as dictated by the Nash bargaining solution so this means that we've got that optimal outcome that we're looking for that is there are no gains in performance to be made by redistributing the jobs but this also leads to another interesting result so experiments were set up using this algorithm and it was pitted against other allocation mechanisms that optimised for different things and during the experiments not only was this algorithm simpler to grasp and simpler to compute it gave each job the same expected response time independent of which computer that job ran on the slowest computers were not utilised by the algorithm whereas other allocation frameworks tended to overload slower computers leading to performance inefficiencies and what about auctions? so Cloud Foundry, the platform that I work on a pivotal incorporates some of these auction elements when it comes to orchestrating where applications are run so at a very basic level this is how it works a user asks the platform for a certain number of application instances an auctioneer requests bids from all the virtual machines in the system and the machines report back as to their capabilities so in a blog post by a Cloud Foundry engineer on app placement he writes how these bids are constructed out of various pieces of information such as available memory and available disks and given certain constraints such as the available memory on a machine has to be at least as great as the required memory for the application instance the auctioneer can then determine where best to place an app instance so on the surface this is a rather straightforward setup and this structure came out of a rewrite of how Cloud Foundry operated where a previously complicated setup led to fragility so game theoretic concepts have helped these engineers to find simplicity and a more effective solution earlier I talked through an example that demonstrates how agents can be incentivised to truthfully report so in some distributed systems resources belong to self-interested agents and as I said this can lead to poor performance if there's any manipulation of the load allocation algorithm and so we have incomplete information and to help uncover this information frameworks have been built around ideas that stem from the second price auction and one such framework is something called a Vickery Clarke-Groves mechanism where computers are programmed with a profit function equal to a payment minus a cost the payment is paid by the auctioneer to each machine after it receives the bids and the cost is related to how much work the computer eventually has to do so the structure of payments given out in this system is such that each machine only maximises their profit when they truthfully report their capabilities and again you can find more on this in Groesu's paper but what do game theoretic approaches mean more broadly for the types of systems that we'll build Nassim Nicholas Taleb in his latest book develops the concept of anti-fragility where in the face of failures systems learn from feedback and get stronger systems built focused in individual machines that optimise with game theoretic algorithms are well placed to be able to adapt to strenuous circumstances if a machine goes down or there are network partitions the algorithms in the system can dynamically update and reallocate jobs where necessary so they may be able to demonstrate some of these anti-fragile characteristics so now it's time to wrap up what are the specific results that we have looked at today when we know the capabilities of the machines in a distributed system we can use things such as the Nash bargaining solution to help us to determine how we should allocate those resources and when we don't know the capabilities of the machines and we need to know them then frameworks built around second price auctions can be really helpful in terms of revealing what those capabilities are so in some ways computers are similar to humans in that they can often behave in ways you don't expect and when you're dealing with uncertainty you need ways to clear away the noise and this is where game theory becomes very handy you don't need maths to use game theory as a tool to understand complex issues as a starting point just think about how you would model parts of your system as a game and what contracts or competition need to happen between the components and for the mathematical out there it opens up a world of interesting simple yet powerful algorithms to help us produce better performing systems so what can you go and do now you can start by reading this book it doesn't really look like it but it's essentially an introduction to game theory filled with stories from a range of areas spanning from football to war strategies and when reading think about how the examples may relate to different computer setups there were many other types of games that I didn't touch on so perhaps if you're interested in cloud security you can find some interesting parallels to draw with non-cooperative game theory maybe you're interested in clustering you're perhaps working with a database like Cassandra well evolutionary game theory will have some interesting results for you and maybe you're interested in the platform as a service landscape well then coalition theory can help with that if you want to dig deeper these are some of the sources that I use to help me write this talk and one final question remains would you like to play again thank you