 Good morning. Please try to contain your enthusiasm. Let's see. Are we ready to go? Let's wait for the video to switch over. It's just so mind-blowing. We have to watch it over and over. Well, while that's going, I'll just start. So imagine you come home from a long day of sysadmitting or operations engineering or devopsing or site reliability engineering or ninjaing or rock-starring or super-heroing. And you decide to sit down and make your entertainment decision. See if I can help out. Because really, the graphic makes it. There we are. Yay! Answer the AV, guys. That's a hard job. So you come home long day after you're rock-starring and ninjaing, and you sit down on your couch, which, of course, my vision looks something like this. And you have a little bit of spare time at the end of the day, and you decide what you're going to do with that spare time. You're having a moment. And in that moment, you have to decide, what am I going to do? What am I going to watch? Maybe something with superheroes? How about that new lawyer show? How about that show with superhero lawyers? Maybe something a little romantic? Perhaps a comedy? Number one, Bojack fan? Maybe a romantic comedy? How about something grand and epic? Or something epically spooky? Maybe a political documentary? I'm just kidding. I know what a documentary is. How about a prison documentary? I was funding twice. How about an actual documentary? Maybe something about retirees? Powerful women? Powerful women retirees in their natural habitat. So you make your decision, whatever that might be. You're presented with the play button. You've made your choice. You sit down. You're ready. You hit play. And this is what you see. Never. This is fictitious. Nobody's ever actually seen this before. We're debuting it at the conference today. Now, if you have had this unfortunate experience, I've long said we should actually change this out for this. That way, if we're having a bad day, we're at least helping you make healthy life choices. So anyway, you're presented with this. And you might say to yourself, huh? Is it just me? So maybe a side, I'll check Twitter. Maybe Facebook or a couple of those news sites. So what's going on at Netflix? So I call this next bit a bite about Netflix. I used to call it a bit about Netflix, but this is at least eight times better. I love tech conferences because everybody gets the nerd jokes, and I have to explain that. So Netflix is a large data and telemetry company that has the byproduct of streaming video over the internet. That was another joke. There's going to be a couple more. Just ask that you keep up with me. In 2016, Netflix went global. Netflix is now available in over 190 countries in the world, minus those few little gray areas that just aren't quite ready for our brand of entertainment just yet. When you talk to Netflix, you talk primarily to three different places. Netflix has a large infrastructure in Amazon web services. It's where we do all of our compute and storage. A little bit with Akamai, UI assets and small assets to help build out that UI that you look at. And then Netflix Open Connect is our purpose built video CDN. Any video bits that you see while viewing Netflix come from our CDN. A few years ago, when we first started out, we would stream from one of the big three, the limelights, the Akamais, or the level threes. But over time, we found some problems. First of all, CDNs can be a little expensive. Especially if you have video files and a lot of them, they're kind of big. It gets to be even a bit more expensive. Secondly, they have requirements around their business processes and needs and goals that Netflix doesn't have. Any time they are to place a cache somewhere, whether it's in the data center or in an IACS or embedded in a network, they have to have some kind of profit model associated with the placement of that cache. So unless you're on a very large network, if you're on a medium-sized network or a smaller one, they can't make as much investment in the caching infrastructure. So that means we can't put as much Netflix out there. So we started building our own. Because our goals are to help win those moments of truth that we were talking about earlier. And one of the best ways that I can do that, beyond amazing content, amazing content, there we are. Thank you. Thank you. You're following along very well. I'm very impressed. Is that you have to have a good experience. So with the Netflix OpenConnect CDN, we can take our caching machines and approach any ISP and give them a stack of caching equipment for free and ask them to put it inside of their network. This provides some really nice benefits, really, for everybody involved. You, as someone who is consuming Netflix, those video bits are now much more close to you. And you don't have to go through the drain point at your ISP's network out to the internet. You're not worried about some of those choke points that happen in those areas. And typically, your communication is simpler and cleaner. Your ISP oftentimes has to pay transit costs to talk to other networks. Wow, this place is sticky. I feel like I'm doing another kind of show. They have to pay those transit costs on the internet. And you may have seen a few of those stories that talk about how, at least in the US, Netflix peaks out at about 37% of US internet bandwidth at peak time. That means that 37% of the transit costs your ISP's are paying for so that you can watch any of those shows. If we put those caches inside of their network, they get all that transit cost back. So it reduces cost for ISP's. It makes the experience better for our customers. And we win. That's the story behind the Netflix OpenConnect CDN. It's another one of our interesting open source projects. Everything about the hardware and the software and how we put it together is available at openconnect.netflix.com if you're curious about how we chose to run a CDN. Netflix is a large microservices infrastructure. It kind of looks like this. I'll give those of you taking notes a minute. It's a little bit of a complex, ever-changing beast. I think it's pretty safe to say, at this point, Netflix is complex enough that there's not one person inside the organization that really understands all of it. Even if we break it down to just one of the other services, and we look, for instance, at call paths coming off of that service, it still kind of looks like this. So Netflix is made up of hundreds of microservices. There are thousands of daily production changes. And I don't mean we updated a record in a database. These are code pushes, feature flag changes, things that actually change our production environment, thousands of those. We run tens of thousands of virtual instances inside of Amazon. We have hundreds of thousands of customer interactions. We're in actually every second now. I actually need to update my slide. Hundreds of thousands of customer interactions every second. We have millions of customers. As of Tuesday when we published those numbers, that's now at 81.5 million global Netflix customers. Billions of time series metrics. Currently we run about 2.5 billion time series metrics every minute that are delivered, processed, and stored. We stream tens of billions of hours of entertainment to our customers every quarter. We do this with tens of operations engineers and no knocks. We also don't have anything that is a knock and has cleverly been renamed to something else so nobody calls it a knock. We don't have data centers anymore. Netflix made that transition. We started, make sure I'm not getting out of myself. There we are, preview. We started that transition in 2010. And six years later, we finally finished our transition out of the data center and now we're completely cloud based. So you may say, it's really great, Dave, but I have a question. Well, I'm glad you asked. How does Netflix think about DevOps? Well, the truth is we don't. You might say, Dave, this is a DevOps conference? The word DevOps is literally in the name of the conference. So I told you we don't think about DevOps. Tell you a few things we do think about. We don't build systems that say no to our developers and engineers. There is no push schedule. There are no push windows. There is no crucible to production people have to go through in order to have their code blessed so that it can go out into the production world. Engineers at Netflix would never see that second half of the screen. Every engineer and really everyone at Netflix has full access to our production environment. There's nothing there to tell them no. We don't take the time to build systems or have policies or procedures that prevent people from accessing the production environment. So what do we think about? We think about freedom and responsibility. One of the goals at Netflix is that we want to hire smart people and get out of their way. If I hire someone who's good at what they do and they're intelligent, they need to have the freedom to make the decisions to solve the problems in the way that they see best. And if we've created a lot of artificial constraints and guardrails trying to predict what it is we need to do, it's done me no good to hire smart people. We also look for people that are the kind of people that take responsibility. They don't wait for it to be given to them. Freedom and responsibility is a balance. So we look for people that enjoy that freedom and exercise that freedom and understand the responsibility that comes along with making those decisions and taking that kind of responsibility. We don't think about uptime at all costs. Now if you've ever looked at Twitter when Netflix is down, some people think they're going to die. So I wanted to let you know we've checked nobody ever has. Now there are some companies and organizations for which uptime at all costs is very important. Things in healthcare, IT, finance, those kinds of areas. Downtime has a different kind of repercussion. Not so at Netflix. We don't look for uptime at all costs. So what do we do? We prize the velocity of innovation. I want those smart engineers that I've hired to use their freedom to develop new things and new features and new ways of exercising the system and new ways of delighting our customers. And Netflix as an organization knows that we are going to trade some amount of uptime to keep that velocity of innovation. I am proud to say our uptime, frankly, is rather good. But it is not our first priority. Our first priority is keeping our engineers doing fun and interesting and exciting things for those millions of customers worldwide. We don't do a lot of processes and procedures. As I mentioned earlier, it's difficult to have a fast moving organization full of people solving new and interesting problems and assuming that someone can build the guardrails that are appropriate to what they're going to be doing. Can think of all the processes necessary that will keep them safe in the decisions that they're making. It's really a very bureaucratic way of thinking of things. Bureaucracies have a certain function. And one of the primary functions of a bureaucracy is to be able to take that bureaucratic machine, plug in virtually any cog, and get the output that the bureaucracy wants. The cogs are only allowed to do certain things. It's very prescriptive. Lots of guardrails, a lot of processes, a lot of procedures so that any cog can fit in and get the job done. That's not what we're doing. We work very hard to look for the right kind of people that aren't a bureaucratic machine member. We want them to challenge the things that we currently do. We want them to have new ideas. We want them to try things. And if we try to contain them too much, that's not going to happen. So what do we do? We trust the people that we hire. That's why we don't have concerns about allowing anyone into production. Now, when I mention that to some people, there's always this wealth. If anybody get into production, they'll just shut everything down. That's happened precisely zero times to us in all the years that production has been open to literally everybody at Netflix. We don't do control. You may have kind of gotten that idea so far. What we do talk about is context. I have the privilege of being able to hire or being on the hiring crew for some managers that we bring into Netflix, people looking to help engineers do their jobs. And one of the things I talk about is that managing at Netflix is very different than it is a lot of other organizations. In many organizations, the job of the manager is to determine what needs to be done, figured out, laid out, and put their cogs to work. It's not true at Netflix. The job, the primary job of a manager at Netflix is to make sure the people they work with have a quality and constant flow of context about the business, pardon me, decisions being made in other departments where other people are going and what they're doing so that the people they work with can make well-informed decisions, exercise their freedom well, and keep moving and doing things. So we prefer context over control. We don't do a lot of required standards. We don't have a thou shalt write in this language and use these libraries and this framework and this IDE. Netflix has, we have a lot of JVM languages running around. We have Java running around. We have Scala running around. We have some Clojure running around. We also have Ruby and Python running around. We have Rust and Go running around. We have a lot of Node running around. So we don't have these required standards. What we focus on is enablement. So as a for instance, the teams that are responsible for your interaction in the UI, if you access Netflix from a laptop or a desktop, something through a browser, decided a couple of years ago that they were making Herculean efforts to write their back-end in Java and the front-end in JavaScript. I said, you know, we think we'd really get, we'd really get some time back and we'd be able to exercise some new advantages and do some new things. If we ran our infrastructure on Node instead of Java. They did not have to go through a research process. They did not go through an approval process. They did not have to run a crucible to make that decision. They decided as a team that this was the best thing to do for the company and they rewrote the portion of the system that they're responsible for and launched it in Node. It's the same way we think about tooling. I mentioned earlier at Netflix there are tens of operations engineers that run this whole thing. Currently, the operations engineering organization at Netflix is about 70 people. Interestingly enough, the vast majority of those people are software engineers. What they do is they focus on writing tools and enabling other software developers to do their job and focus on the things that they're good at focusing on. If we hire someone to work in our billing group because they're really good at writing billing and processing code for billing, should they have to learn a whole lot about time series metrics databases? Should they have to learn about the interesting and twisted path that it takes to get things running in the Amazon web services environment? No, we want them writing billing code. I think it's literally what we hired them for. So we work and focus on enabling our developers to be able to do exactly that, spend the time on the things that we've hired them to do. We don't do silos, we don't do wells, we don't do fences. I was rather impressed in my first few months at Netflix, I spent time just running around. You can tell I'm a tiny bit gregarious and just walking over to other teams and go, so what do you guys do? And I was impressed that they'd take the time to tell me what their team did, how they fit into the ecosystem and even more so, how they worked with the other teams, their dependents and their dependencies in the effects that they had on them and how they worked together. I thought I was gonna have to go to each one and try to piece it together myself and here were these people that are used to talking to each other across teams doing it for me. We also don't have some of those traditional operational fences. The operational engineering group does not sit behind the fence over which code is thrown in hopes that it will show up in production. So we don't have any fences to throw things over. What we do is we focus on making ownership easy. I think probably everyone by now has heard the, you build it, you run it. We focus on you build it, you run it, but with that enablement idea. Again, it's one thing to say, you're gonna run your code, you're gonna deploy your code, you're gonna figure out the operating system, you're gonna figure out your instances, you're gonna figure out all of your ASG and cluster settings and all of your ELBs. And then you get that thing out in production and it'll be fine because that's be fine so that you're getting paid best of luck see you later. So we do you build it, you run it with a focus on enablement, making those kinds of things easy. So for instance, we have a fun tool called Spinnaker. Oh, by the way, I mentioned some of our tools, vastly all of these we've open sourced. You can find them at netflix.github.io. Spinnaker is one of those. And Spinnaker makes it easy for a developer to get their, I term their job putting the appy thing in the cloudy thing. For some reason they don't find that nearly as humorous as I would. But we wanna make that easy. So Spinnaker allows them to describe a little pipeline that says I want my code from here to live in these places and I'd like to have these kinds of things run like the smoke test has to pass. I'd like it to run through the automated canary analysis system and look for any kind of performance regression. So they just do that quick description and then all they have to do is publish their code to a repository and the system handles it for them from there. If the code smells funny, if it doesn't pass tests if there's some issue with the way that it's being built out that'll never make it into production. That passes all of its tests. It shows up exactly where it's supposed to. Traffic is managed and now our 81 and a half million global customers are now talking to that new code base. So we focus on making ownership easy. We don't do a whole lot of guesses and good instinct and we try not to fall victim to the traditional thinking of well that's the way we've always done it. We do focus on data. Netflix is an enormous data company. I mentioned earlier those two and a half billion time series metrics. That's only our operational data that's not accounting for all the data we have to run all those algorithms that we talk about. There's no financial data. That's another awareness data set. A lot of the decisions really the vast majority that Netflix makes are based on data. So a few examples. When Netflix started streaming in 2007, was anybody around streaming Netflix in 2007? Horrible catalog of the time. I'm sorry about that. So we started streaming from our data centers in 2007. In 2008, there was a fire in one of the data centers. Now for those of you that aren't familiar, fires and data centers vary incompatible. So we had a decision to make. You know, we were releasing space from someone who decided, well do we, do we start doing our own data centers? We have some really bright people. I'm sure we could find a way to do data centers and data center management really, really, really well. But what we concluded is that if our job is to win those moments of truth, being a really, really, really good data center operator didn't really help us do that. So that's when we made the decision to move to the cloud. We looked for partners that can do what we call undifferentiated heavy lifting. Work that needs to be done, but for which having it done doesn't bring our customers any direct benefit. There's some people out there on the vendor floor today that Netflix uses their services because we'd much rather have them do it and build it and let us use it. So we decided to move to the cloud. In 2010, we'd done our first bit of streaming to devices out there in the world from Amazon's cloud. And by some time in late 2015, we'd finally finished our transition from the data center to the cloud. For those of you bad at math, that's a long time. Making that transition was hard, but we finally made it. Other times in which data is helpful, we do a lot of looking and digging at what people watch and what they enjoy and what they come back and watch again and the related kinds of things they watch. So we were able to produce, there was House of Cards up there earlier. That was a very data-driven decision. We had a really good idea about the kind of stories people like, the kind of actors they would enjoy, the kind of director that would bring that story to life. And we took a gamble and we produced it. And honestly, it's been one of our biggest success stories ever. So everything Netflix does is driven on data. Now, one thing that we don't do. Mentioned earlier, we don't have a knock or anything named like a knock. This is a picture of the operations group at our headquarters in Los Gatos. You may notice one thing here that's different in a lot of other organizations and their operations group. We aren't surrounded by televisions showing us graphs. We're a data-driven company and a decision-driven company. We have two and a half billion time series metrics. If I took a 15-inch MacBook Pro with a retina screen and I stacked up 90 of them, I could almost get one pixel for every time series that we have. Who's going to stare at that? Interesting experiment, I suppose. We also believe that human beings are entirely too intelligent and frankly, entirely too expensive to spend time staring at a screen hoping they'll pick out a problem out of a complex system. So we invest in data and we invest in algorithms and we invest in systems that can comb over all of that data very, very quickly and let us know when there's a problem. And we lead that charge in the organization by this example. There aren't TVs with screens and graphs. And so I was like, I can tell you the person taking the picture, it's not like they're behind him. We weren't trying to be sneaky. So we are data-driven operationally. And I work on the team that is responsible for you press play and it works. Or if it doesn't work. That's our work environment. We're data-driven as well. So as I said, we don't do DevOps. What we do is focus on our culture. I've mentioned a few words and phrases that are important in the Netflix culture and that we bandy about and we talk about. Things like freedom and responsibility, context over control. Even in a hiring process, if somebody's gonna come in and see if Netflix is a good fit for them. The first thing we always tell them to do is to go through our culture deck and make sure that the way you do things and the way you think about things or that we think about things is compatible with the way you think about things. There've been some people I've had the privilege to interview that were brilliant. But they would not have been a fit in our culture at all. And we value our culture and its benefits so highly that we will pass on people that would have a negative impact on our culture. So the result of the Netflix culture really looks a lot like DevOps. But the important thing is the focus. DevOps is a wonderful result of a healthy culture and healthy thinking. If you have problems, you can't just hope to take the DevOps cream and rub them all over things and all of your problems will go away. Surprisingly, it doesn't work that way. Now, other people have said this to me, not you, of course. Sounds very nice, Dave, but it won't work or I work. And I'm always curious. So I have to ask some questions that are really, why doesn't this thinking or this methodology work? And invariably, I end up making some blunt statement like this. You don't have a DevOps problem, you have a culture problem. If you want the benefit of DevOps or whatever the new phrase will be for this change in mentality for the way that we did IT and engineering over the years, you have to address your culture. Just giving people DevOps titles won't fix the problem. If you're curious about that slide deck that talks about some more of these things is about 127 slides right now, I think. You can find that at jobs.netflix.com. Now, I'll go through all the points of the culture and all the things that we value. Well, I'm glad you asked. There's a lot of information about people we'd like to have come play with us at jobs.netflix.com. I have a couple of cohorts with me. Brian and Blake are here with me from Netflix. We're hanging out at our table out there. Now, we're a little different than some other people you may traditionally see at conferences. We don't have anything to sell you. The first month of the service is free to you and literally billions of other people. All the software we make, we give away at GitHub, but we do have some really nice swag that we like to share with people. And we love answering questions and talking about what we do, how we do things, and why we enjoy it so much. So please combine chat with us. I'm Dave Hahn. I'm a senior SRE on the core team at Netflix. That's where you can find me out on the internet. So our call centers exist in two pieces. The question was how much of this culture exists in our call centers. And I'll let Brian holler at me since he knows this information a bit better than I do. We currently have, I believe, 21 call centers around the world. Close enough that he's not gonna correct me. We have 21 call centers around the world. 19 of those are actually outsourced to business partners. So the culture there is dictated by what the business partner believes is most advantageous. We have recently opened two Netflix branded and owned customer service centers, one in Utah and one in Yokohama, Japan, that will start doing more and more of our customer service work. And our culture is just as important there as it is in Los Gatos. Sure. So the question was how does this whole freedom and responsibility set of verbs deal with things like PCI audits? We do have a small segmented area inside of Amazon where we do all of our payment processing. And if you want access to it, you just have to go ask. Still rather compatible within reason so that audits are reasonable. But not still under the same flag. We run all the same tools, all of those kinds of things. Any other questions? Is there one over here? Yes. How does our culture affect our interview process? Did I hear that correctly? I like this question. So part of our interview panel, we have people in our talent team that actually specialize in feeling out whether or not someone is a culture fit. So that's always tested because we really don't like that idea that we bring somebody in and we hire them and we change their lives just to go, sorry. So it's a very important part. We also have a very free flowing feedback environment. You are expected if you work at Netflix to give people feedback all the time to the point where people come pull it out of you if you're not providing it. We do the same thing in our interview processes. And I've said to people during interviews, you know, I like you a lot, you're a great engineer, but you don't want this job. So very much so even during the interview process. Very important because if we're gonna bring them in they have to be able to work with people like me. So I'll answer the second one first, because that's easier. I mentioned that we're kind of free flowing on languages and the questions was how are things like templating standards enforced? They're handled team by team. That team is responsible for that and that you build it, you run it idea. There are teams that have developed everything from coding standards to template standards to things that work well for that team, but they're not pushed throughout the rest of the organization. So it's a team by team question. Then your other one was the two pizza, three pizza, Amazon kind of thing. I'm not familiar with that one at all. So if you can tell me what you mean, I'd be happy to answer. Otherwise, pizza sounds great. Yes, two please. So the average team size at Netflix last I counted was 11. Now we do have those teams grouped together. For instance, the operations engineering team I mentioned is 70 people. That's broken up amongst, I think, right now, six teams. So maybe not intentionally the same, but very much the same kind of model. We don't have large teams. Let's see, I think Jason's gonna push me off the stage. So I will be back at the Netflix table. Happy to answer more questions. If you'd like intelligent answers, Brian and Blake will be there as well. Thank you all so much.