 Great. Thanks. Yeah, it was really cool. I only learned much later in life that it's not normal for a six-year-old to beg his dad to take him to work on the weekends, because my now seven-year-old never has, even though I work for a video game company, right? Cool. So what I want to talk about is the virtuous cycles of velocity, what I learned about going fast at eBay and Google. So I don't usually start with a background slide, it's relevant to the title of the talk. So right now, and for the last several months, I've been the CTO at Gaming Company Kixi. And our mission, my particular mission, is making awesome games that much awesomer and scalabler and reliable. Before that, I was director of engineering at Google for Google App Engine. And if you're not familiar with it, it's the world's largest platform as a service. Then before that, for about six and a half years, I was the chief engineer at eBay. And one of the main things that I worked on there was multiple generations of eBay's real-time search infrastructure. And some of the stories that I'm going to tell here come from those experiences. So I like to break it down, because it's a big topic. So break it down in sort of lessons that I learned around people, lessons I've learned around technology, and lessons I've learned around culture. So in terms of people, this isn't easy to say it hard to do, but hire and retain the very best. So if you hire eight players, you will notice that the difference between the top performers and bottom performers is not sort of one and a half X, it's not two X, it's 10 X. And study after study, despite a bunch of blog stuff last month, has continued to prove that. The best example I've seen in my career is the Google hiring process. And it's famously challenging, and it really is. And they're very happy, Google is very happy to have sort of false negatives, by which I mean, they're going to put everybody through a really difficult interview process. And it's more important that they get excellent people out the other end than that they get all excellent people out the other end. Yeah, so that's false negatives. So people might be rejected that otherwise should be at Google, in Google's eyes. And what's wonderful about that is that Google knows that. So there's actually a concept in the hiring committees as when they evaluate potential candidates of when, not if, when is this person going to be sort of, when will we accept applications from this person again, right? So somebody goes through, okay, they didn't make it this cut. That's cool. It's totally understandable. And yeah, I mean, the conversation is literally, yeah, that was only her first time. So that's cool. Give her another shot in six months or a year if she's still interested and we'll go from there. There is a very obvious and straightforward and well understood virtuous cycle right here, which is A players tend to hire A players because they're not afraid of them. B players sadly tend to hire C players. So one of the things that we do for all the values of we that I said, particularly at kick size, that we're constantly raising the bar on the level of talent that we're, that we're looking for and that we're bringing into the organization. And that's super important to do always no matter what your, no matter what your organization is. Because that, that 10X thing is not, is not going away. And if you think of it from a purely crassly economic perspective, you don't typically pay the highest performer 10X the lowest. So you're actually winning by, by hiring excellent people. Once they're there, respect them. The people are your most, your people are your most valuable and irreplaceable asset. People are not cogs and they're not fungible. So the, the anti story I like to tell in this area is actually Adrian mentioned it as an aside in the keynote. eBay, the old school eBay, this isn't true of the, of the new eBay. The old school eBay had a concept of train seats where a train seat was a unit, an engineer two week unit, right? One engineers work for two weeks. And boy was it awful. There was a concept of, okay, we have this idea of something that we want to do and it was scoped out, you know, was sort of specked out, right? And then it was scoped by someone, not the person that's going to do the work, by the way, but someone said, yeah, I think that's going to take, you know, eight train seats, right? Eight times two weeks worth of engineers. Then really some random set of engineers is going to be assigned to do, to implement that idea. Wow, is that bad? That destroyed everybody's incentives all along that chain, right? The people that were doing the work didn't get to estimate it. They didn't get to design it, right? They just had to do it. And I don't know about you, but that, that's extremely unmotivating for talented engineers like we hope we got on the last slide. It destroyed people's personal pride because they didn't have a ownership in this particular task that they were doing. But even more importantly, it was run as a job shop, so they didn't have long term ownership of even the code base that they were working in. So that was a real, that was a real challenge and clearly needed to be fixed. It really came out of, you know, thinking of humans as, as manufacturing, you know, like literally the widgets, not even the, not even the assembly line. Pretty awful. So fix now. Look at eBay stock price. They are not unconnected. People are an asset, not a cost center. The better the more investment that we make in people, the more directly investment we're going to make in great products and great technology and great user experiences. And the environment should treat people, should make people make our employees know that they're valued. Famously, if you look at at public photos, actually, if you watch the internship, the Google Workplace is famous for making people feel, have a really fun time making people feel extremely, extremely valued. And you feel that, right? You absolutely feel that. And it's extremely motivating, well beyond any cost that goes into building a great cafe, hiring a great chef, having slides all over the place. I mean, it's fun. And in all honesty, I mean, in back of my head, I go, yeah, that's a little much. But the overall effect of that is, wow, Google really cares about me. And that makes people, makes people want to work there, and makes people work hard when they're there. So the virtuous cycle of people, hire a players, treat them well. You're going to keep them and retain them, and raise the bar. Yeah. Next, I want to talk about technology. So the first thing I want to say about technology is how one organizes, I'm a server side guy. So I think of it this way, but you can, this is applicable to, you know, to any discipline in technology. Think about things as individual services. And well, what do I mean by that? I mean, somebody that people that are going to produce some particular set of work are going to own it for a while, we hope, and have them be a small team, right, not a team of 50, not a team of 100, but a team of like five, right, to do some particular thing. Amazon famously has this idea, they practice this extremely well. Amazon has this famous idea of the two pizza team, right, it's a team that can be fed by two large pizzas. And the joke I remember hearing from some of my friends there is like, yeah, you know, actually, the young guys are really hungry, so we can only have as many, you know, fewer younger guys than the older folks. It's actually really, it's actually really, really funny. But it totally works, right, because everybody knows each other super well. Any kind of methodology that you want to practice, whether it's any flavor of agile or anything, makes, is a lot easier when you're only coordinating among some very small number of people, right? Well, how does that work with the rest of the organization? Because we have a thousand, or in Google's case, 54,000 people's worth of stuff to do. So that team should have a well-defined interface to the rest of the company. And I could, you can think of that as in a purely technology sense, right, a service should have a well-defined interface, it does these things and other things, super simple. But that's the same mistrue of sort of any team that interacts with any other team. It's a bit surprising, but maybe shouldn't be that everything we all know is stuff that the hardware guys knew 20 or 30 years ago. Like, this is exactly how chips are designed. And my dad knew this after. They should be completely independent, right? So a particular service team, you know, these five or X people, these, you know, these two pizza feeding teams, should be able to be independent of one another. And I mean, I don't mean no relationships, no collaboration, but what I do mean is they get to control their own destiny and they have autonomy in what they do, which leads right into autonomy and responsibility, right? So as that team, they should be able to decide how they work, decide what they do, even what technologies they leverage. Because if you're forcing people to use shared stuff, why do you have to force them, right? Should be good enough that they'd beg you to use it. Autonomy and responsibility, which puts that ownership on us in exactly the right place and aligns the incentives for the output and, you know, how the stuff's actually done. Next thing I want to talk about is quality discipline. And I'm sure this is preaching to the choir in a sort of agile, continuous XYZ crowd. But tests help you go faster, right? The best, the best investment you can make in your own code, let alone the health of the entire organization or the entire piece of technology, the best investment you can make in your own life is testing tests that you put around your own code. Why? Because tests have your back. They give you the confidence to do crazy stuff like break things and refactor things mercilessly. And it allows you to catch bugs a lot earlier when it's easier and cheaper to fix them and fail faster, right? So there's kind of no downsides here. One often hears today and also 10 years ago and probably 20, 30, 50, 100 years ago, we don't have time to do it right. And I actually say wrong. It's not that we don't have time to do it right. We don't have time to do it twice, right? Exactly when you are resource constrained. That's exactly the time when you should be doing things properly, right? Because you don't have time to go back around and do it properly the second time. Always trade-offs to be made, of course, you know, Christmas isn't going to move, so maybe we need to get this thing out. But in the large, doing things right is going to pay off in the long term and it's actually going to pay off in the short term. So you can get into this vicious cycle of technical debt, right? You do something quick and dirty. You accrue some technical debt. There's no time to do the next thing right. So we're going to do it quick and dirty. So we're going to accrue more debt, round and round and round and round. If this were a credit card company, they'd kind of like this, right? But not if you actually want, not, but that's not a good thing for the actual person who's experiencing it. Automating quality. I feel super, super strongly, okay, if quality is a good thing to do, automation is also a good thing to do. Let's put, you know, two great tastes at two great tastes that taste great together. So investment in tooling around quality and I'm not a religious about particular tools, whatever is going to work for your programming environment and your language and so on. But the principles here are make it easy to do the right thing, right? Super easy to test your code, super easy to automate that. And examples there are sort of mocking and testing frameworks. We all know the star units. Monitoring for operations. You know, so we shouldn't be releasing stuff where we can't monitor the quality and the efficacy of the stuff that we, that we have going in our, in our site. And also canarying. And what canary means, Adrian also mentioned this in his keynote, but it's a sort of becoming a bit of a term of art of where, you know, I have a thousand machines that are implementing my service. I'm going to start with one or some small number as a canary, right? The analogy here is to canary in a coal mine. So I give, I tested out on one or some small subset of those machines or of those individual instances and see how it goes and then, then and only then can I decide to go forward or backward, right? That's just a good discipline to have, no matter what. Quality is never, well, never. Quality shouldn't be an afterthought, right? By which I mean quality, reliability, scaling. These are priority zero features, right? We can talk about, yeah, we'd like to have this, you know, interface be prettier and yes, in a world, in a perfect world, it should be prettier. But before we do anything, before anybody plays any of our games at Tixi or uses any of our development environments at App Engine or buy stuff at eBay, you know, quality, reliability, scaling, that needs to be in from the beginning, because that stuff is the most difficult to retrofit when you haven't done it the first time. So the example I want to draw is there's an excellent, excellent automated testing and quality discipline at Google, which is why Google can release, one of the many reasons why Google can have extremely small teams releasing amazingly scalable and powerful features. Every piece of infrastructure that the Google things are built on is all instrumented with automated tests, which has other nice properties that I'll maybe talk about before, where it's really easy to accept code from somebody you've never seen before, like sort of an internal open source model if you like. Hey, instead of just saying I found a bug, here's a bug report, like I found a bug, here's where it is in your code, I fixed it and here's the test. Wow, really cool. And again, I want to stress this is the old eBay that I'm talking about here. There wasn't really a lot of thought given to this and the site suffered, right, and the individuals, those poor folks, poor cogs, you know, slogging away on the stuff they didn't estimate. Their job was made that much harder by not having tests to back themselves up and when they're refactoring code they've never seen before. So now there's a virtuous cycle of quality, right? So if we do testing, we're going to be on a solid foundation, we're going to have confidence, we're going to be able to be faster and better. Now we've got time, right, in the road maps, now we do more testing, more solid foundation, et cetera. And the other analogy I like to use here is we want to run fast. Well, do we run faster on solid ground or do we run faster on quicksand? Like, nobody answers, but I'm going to give you the answer. This was an easy one, solid ground. And actually you can run even faster and I'm stretching the metaphor a little bit but like that really springy stuff that is now showing up on San Francisco playgrounds and on track surfaces, like you run really, really fast on those things and that is sort of I'll analogize that to the really well tested, really well automated stuff. So it can get better and better. You can get better than solid, right? On the culture front, having a culture of accountability and ownership and these all things go together, right? Small teams, you know, if you give people in teams autonomy, they're going to surprise you with what great work they can do, right? People like to have autonomy and in fact there's the trifecta of motivation at work, right? Autonomy, mastery purpose. Autonomy is right up there, it's right in the top three. Hold people accountable for their success, right? Autonomy without accountability is meaningless. So I say with my little five person team, I'm going to do X or Y or Z, yeah, I'm going to do it, right? That's what it means to have integrity, but that's what it means to have accountability and you guys, if we're team members, should expect that of me, right? So hold me accountable to my team's success, hold me accountable for keeping the commitments that I make and then for me, say what I'm going to do and then actually do it. Very closely related to this is collaboration. So the best functioning teams, meta teams if you like, that I've ever seen are the ones that collaborate seamlessly across otherwise randomly drawn organizational boundaries. So in the areas of work that I do that's typically engineering product operations, but you can fill in and actually for the work that other people in Kixi are doing, that includes art, that includes audio, that includes all the fun stuff that goes into making a really fun to play game. So if you don't act as one team, the alternative is basically playing little organizational strategy games instead of solving the real problem and really delivering in our case a great game or a great user experience or a great e-commerce site or something like that. Otherwise you get, you know, CUIA behavior, hiding the ball, all that sort of stuff. And the little anecdote I like to tell here is when I was at Google and App Engine, we were in the San Francisco office, which is wonderful and beautiful. And our people were all on one floor. And when I say our people, I mean all these disciplines, all reporting up through entirely different VPs, et cetera, et cetera, none of whom we've ever met in most cases. We all were co-located on a single floor. So I had the wonderful experience, and it really was literally wonderful, of the site services people who are, you know, trying to plan where the building, what they're going to do with the building, you know, going forward, coming to us and saying, hey, can we talk about your growth plans for the team? What, you know, how, what size are you expecting? One year, two years out, et cetera. And we want to figure out, you know, going forward where we might put you in the building. And so, all right, we talked about what we thought that would work, how that would work, what our growth plans were going to be. And then they said, okay, well, we can probably put you all together in this other floor and what other teams should be with you, right? So, you know, we went through, okay, support ought to be with us and developer relations. We don't need to have them right next to us. We talked to them, you know, we just went through a bunch of this sort of ancillary related teams that are just part of developing, and in this case, producing a developer-oriented service like App Engine. And then the woman says, oh, yeah, that's great. Okay. So, here's your team. What about this other team that's on your floor? Oh, yeah, that's a product. They're us. Like, we didn't even consider that they weren't. Yeah, what about this other team? Oh, yeah, that's SRE. That's Site Reliability Engineering, which is Google Code for Ops. Yeah, that's us, right? Like, it didn't even, it did not even occur to us that they weren't our team. And if you look at Google's organizational structure, like, literally, we only came to get, in some cases, we only came together at Larry. So, there was no, like, there wasn't an organizational thing, right? That was just simply, we were all working together, doing our part to make App Engine a success. And it just simply never occurred to us that we weren't all one team. I really like that story. It kind of warms my heart, even when I tell it the tenth time. Solve problems instead of pointing fingers, right? If you sit together and no people, trust them, this is just going to happen naturally. And when it is not happening, that's a red flag and something should change. Quality over quantity. This is a cultural thing, right? And it's hard. It's really hard, especially with hard driving A players. Like, no, actually, guys, less is more. We need more wood behind fewer arrows. And the, oh, yeah, but we can do all these different things. Look at all the people that we have. Okay, great. Let's do them instead of in parallel in series, right? Let's solve 100% of one problem rather than 50% of two problems. Or equivalently, let's release one great feature for our users instead of two sort of half done iffy one, right? And I'd much rather, if everything is equivalent, hardly ever is, you know, doing 100% of, you know, feature A and then then 100% of feature B as distinct from parallel, you know, has some really nice properties if they were competing for the same resources and blah, blah, blah, right? Obviously parallel is parallel. But if there were, if you had choices between doing some great, again, some great thing now-ish versus two medium great things now-ish, like I choose the one. And actually your users would too. Really, right? Think of the stuff that you like to use is do you go, wow, you know, the new version of XYZ thing just has 12 new features. That's awesome. Well, probably you're not going to use all 12 of them. Somebody as I hope. But it's more important that the one or the two features that you actually care about are in there and are really solid rather than sort of peanut butter spreading your resources over lots of different things in parallel. And again, very sort of related to this is thinking about this stuff as sort of the whole user experience, the whole player experience. And we, we, I love the fact that we have a discipline called user experience and meaning no disrespect to anybody. The user experience is more than the user experience. You see what I mean there? The experience that the user gets out of your product or in my case today game is more than what we say the UX discipline is, right? It's super important. Don't hear me say any otherwise. But the whole experience that I have playing one of the one of kick sized games is the user experience as traditionally, you know, denoted in, in that group. The functionality of it, the performance of it, the latency, particularly for you know, games like we have bugs, right? All those things are part of the experience that I have as a player. And thinking about it holistically, from the player's perspective, right, is really, I think frankly is the only way to really do the most important stuff. A culture of experimentation, right? We've talked a lot about this at this conference, which is great. It's wonderful to hear these messages being reiterated over and over again. I'll give the Kicksai example here. So Kicksai's a gaming company. We've had a bunch of games for hardcore gamers in the market for a while. Our two most successful games are extremely old, by which I mean they're two and a half years old. I'm new to the gaming industry, but people who aren't are like, yeah, that's really old. They're still going strong. Why? Because we are iterating in some cases, certainly every week and in some cases every day on the gameplay, the feature set, the ecosystem, because they're all a little ecosystem, the balance between offensive and defensive. There's all these things that go into the gameplay, and we're just iterating on them constantly, every single week. And what we like to say internally is that launching is only the first step. Traditional shrinkwrap software and shrinkwrap games, for that matter, launches done, right? Because once it's out there on all the consoles and burned into a DVD, you know, we can't update that thing. Yikes, right? This is 2013. Rather, we like to think that we engineer successes. And one of the, one of these, one game that we have that's called Battle Pirates, that was more or less an overnight success. But the second one we did after that, called War Commander wasn't, it wasn't after the first month, it wasn't after the second month, or the third, or the fourth, or the fifth. It took six months of this game being out in the market, until we finally hit on the right combination of gameplay, ecosystem, balance, etc., etc. And it took off and we were off to the races. But think about the kind of discipline that that requires. And if you know anything about the gaming industry, like that's crazy what we did, right? Because lots of, lots of, lots of gaming companies will give up on titles if they're not an outrageous overnight success after, you know, a month or two. Anyway, but this has relevance for, you know, for everybody. But it's a discipline that's important to learn and, and it can be very gut wrenching, right? When you're in that five month and 29 day situation, that's really hard, right? To keep plugging away. But we need to, but you have a little, if you've done it before, you have a little bit of confidence that you can, that you can make it all work. Many small experiments sum to big wins. Happy to go into this in great detail, if people would like individually. But eBay, one of the things that really turned around eBay was a serious focus on experimentation and a serious focus on data science. And we, by which I mean a team that was next to mine, but not mine, worked very hard on improving the sight speed of the eBay search experience, this was a couple years ago, and then iterating on and then finally releasing after quite a bit of work, our first machine learned ranking function. And the combination of these two things was basically 2% of revenue plus another 2% of revenue. Whoa, right? At that year, this is a couple years ago, 2010, 2011, check me on this, but eBay's total gross merchandise volume, by which I mean the total amount of goods and services transacted through eBay was $60 billion. So 4% of that is $2.4 billion again, check me on this, but eBay takes an order 10% cut-ish, you know, across, you know, if I peanut butter aggregated over everything. So that's $240 million to the bottom line that we didn't otherwise have. And every one of those was not swinging for the fences type of thing. Every one of those was grind, grind, grind, sum it all together. Wow. Failure tolerance. We really ought to learn from our mistakes and improve. And that's very much an agile thing. Even the whole meta idea of XP, fix XP, right? That whole thing. So rather than asking something went wrong because it always does. Rather than asking what did you do, ask what did you learn? And particularly for engineers like me, if you take the emotion and personalization out of it, I'm so happy to leap in and go, oh yeah, these five things were totally broken and we didn't do backups and blah, blah, blah, all this stuff. And a great example is from the Google side as this concept of blame-free post-mortem. So we have an outage or something and they happen. We have a post-mortem where everybody involved is in the room and we talk about exactly what happened and there's no blame. We're not finger pointing at Randy or some other person. We just simply say, here's what's happened. Here's what happened. What can we learn from that? What can we do better? And you get this almost perverse competition after you start doing that where literally I've seen this where the different SRE teams for the different dependent services that are all involved are like, oh yeah, you think you screwed up? We weren't even doing X or Y and Z. It was that made my heart sing in this really perverse like, yeah, wonderful. That encourages iteration and velocity, right? I love this Teddy Roosevelt quote where failure is not falling down, it's refusing to get back up. So this leads to a virtuous cycle of culture, right? We start with being honest with ourselves and with each other, that engender's trust allows us to take risks. We get faster and better, makes us more likely to be honest, et cetera, et cetera, et cetera. Round and round. And that's all I had to say. There's the obligatory, of course. Join us. Kick-size hiring. This is an image of one of the backyard monsters. Our very first title as a company was a flash game that I'm told, seen as a classic, called Backyard Monsters. We just released two days ago our very first mobile title, which is this, called Backyard Monsters Unleashed. Look for it on the iOS App Store. It's racing up the charts. Very, very excited about this game. And it was built with a super small team that iterated fast, looked at the data, and went with it. So very, very proud to be part of this. First of many that are going to be coming out real soon now. So thank you.