 All right, well thank you all very much for sticking around. I just learned actually that apparently Puppy Palooza is up right now, so in like five minutes. So this is about to turn into a lightning talk because I'm sure everybody wants to go do that. Well actually great, Jim already said who I was. We're all good on that. What I kind of wanted to talk about today was operations and open source. And what we as open source contributors and leaders really need to be thinking about and doing in the context of certainly the CNCF, generally cloud native, and really microservices in general. So in the event you were living underneath the rock, we'll do the quick, what are microservices? We can all be on the same page. They're distributed system. They were predated by enterprise service buses, and SOA, which was predated by Corba, which was predated by RPC and like on and on. Effectively they're in kind of my words, I think many people's words, the UNIX philosophy apply to distributed systems and scaling. So the UNIX philosophy says do one thing, do it well. There's a bunch of other stuff. We do that with network services. We basically say cool, we're gonna take a thing and encapsulate up data, business logic, and really put that on a network behind an endpoint that is only accessible on the network. And that thing is sort of independently manageable. Importantly it gives us an independent domain for logic, for data, and for sort of fault tolerance and scaling. And then of course we have a giant application that sort of stitches all these things together. This is all kind of bread and butter. I do think something that has been overlooked until the last probably year or two, which I'm really glad this is becoming more of a trend is people talking about how when we wanna use microservices to scale, it applies to both technology, like we were like oh, I'm gonna build a thing that does 10 million TPS, but it's also the way we scale our engineering works and we scale in terms of people. And I think that intersection is really what I wanna talk about for the rest of this and just sort of like what's hard about it and what we wanna do kind of going forward and how we just think about it. And I do think the kind of the last point to kind of make on that is to guess two points is when you're building microservices, you're doing it to scale. It's the only game in town. You're gonna sacrifice everything in terms of complexity, cost, everything in order to get that scale. And the reality of once you go and implement this is to implement them, you have to be good at operations as an organization and most shops aren't. And I know everybody's out there like oh cool, the Oracle guy's up here telling us about how enterprises aren't good at operations or the banks can't do it, et cetera. It's actually everybody. It doesn't matter, I've been doing this for a long time. It doesn't matter if you're talking about startups or you're talking about unicorns or the banks or whatever, most places like Frankie, cool, I'm gonna get some Redis and Node.js and string them together and call it a day. And it's actually much harder than that if you wanna succeed at scale, right? Obviously it's all the projects you've heard about for the last day I've been talking about or the last three days. So we'll start with organizations and come back to technology at the end. Yes, I do know what irony means in case you're wondering. This is a comic that I've already seen for a long time since like 2011 that kind of just describes some org charts. And if you looked at this from 2011 really there are a few and I literally mean a few companies on that chart that were good at doing operations and good at building microservices. And I think we sort of started pattern matching ourselves to that going forward. So I'll give you my very biased input on what I've seen work and kind of like how we're thinking about it as we're building all this new stuff at Oracle. I've spent a lot of time at Amazon, a lot of people have done this. Their two pizza mantra really is effective if you're not familiar with it. Basically it's a very clever marketing term from their HR department that describes basically building five to 10 person teams. It works really well, what it describes as well, cool. Unless you have a bunch of hungry people like me you can feed the team with two pizzas. That's kind of the gist. The real point to this though is you have these small teams that are autonomous and accountable and you have a lot of them, right? And then they all go independently. They're not blocked in each other. They're decoupled, they own their own destiny. So when you're one of these teams you're like, okay I own everything about product requirements, just inception, my technology stack, operations, support, really everything. And then Amazon sort of has this classic fitness function model they apply to it which is just like a single thumbs up, thumbs down kind of goal that you can hold the team accountable to and ultimately change over time if they meet it or don't meet it. So it works really well. It goes through scale really fast as an org basically having a bunch of chaos. What it doesn't do is have like an actual product autonomy very well. So you end up with a bunch of warring factions and tribes fight with each other and there's all sorts of other bad sides that come with it. So there's a lot of other things like from Apple, Google, Oracle, other places in the industry that kind of blend this more sort of a centralized model. Like Google's got a very famous SRE culture. If you've not read up on the air budget stuff I think it's actually one of the best concepts and operations in a long time, it's awesome. You know Apple's got a very centric product model obviously like as the chart kind of showed. So I think you kind of want to take a blend of these things but you know fundamentally you're gonna have a bunch of small autonomous teams going really fast. What actually happens to as well as you build up this organization you have lots of people building lots of independent services. You always have kind of this like emergence of these platform teams that solve a bunch of the common crap nobody wants to build. So suppose we're gonna be in the emoji business because that's clearly hot shit these days. And the focus of the company is almost always towards the user facing, customer facing side of the world and so the team that actually serves up emojis gets all the money or all the attention. And then you have these teams that their customers are really internal customers that emerge to solve stuff like auth. Nobody ever wants to deal with storage unless you're a masochist. Nobody ever wants to deal with like kind of setting up telemetry services, et cetera. So like those kind of teams emerge to basically to serve the other teams. And now importantly remember all these people can go very fast, can choose their own destiny. In the very early days you generally find okay cool the emoji team is like all aligned with the platform teams and like everybody's happy because we're all building things at the same time and that's all fine but of course tech moves fast. 18 months in the new and emojis come out right. And like clearly we're gonna start all over because that's the new thing. And there's always kind of this tension when the new teams come out or you end up with a rewrite or whatever whether to sort of bank on stability and bet on what was there before or bet on innovation or fashion, right? So you're always gonna have this kind of tension which is frankly good in terms of what the technology stack is gonna look like in any one of these teams. Maybe, there we go. So regardless of which kind of team you are like you're one of these teams you're still gonna have to solve your problems. You're still gonna have to pick a stack it ends up looking the same for everybody. It's 2017 the reality of what everybody's gonna do is they're gonna use a mix of cloud and open source, right? So what I've been saying for a long time or told people for a long time like cloud or this route is outsourced operations. Like at the end of the day that's what you're paying for like because of course if you could run everything yourself like of course you would, you want to but the reality is it's expensive and you probably can't. So typically people are gonna use as much cloud as they can because what you're getting is effectively like a financial contract of how to achieve an SLA from somebody else and you can hold them over a barrel whenever they break that. But a lot of times it doesn't work for you, right? Like the interfaces are wrong, the SLA's don't meet your needs. Like you're gonna have to like not, you're gonna always end up in this kind of build versus buy versus blend mode. So you're gonna use as much as cloud as you can because it's easy. And then when you're not gonna be able to use cloud you're gonna use open source, right? That's the reality and you're gonna use open source very specifically because it gives you transparency. So if you're the team that's accountable for your destiny like top to bottom, like what other choice do you have besides to bet on software that you can crack under open and get into the hood and actually introspect everything and understand it. Like it's super important. So this is why like this is the state of the world. This is all obvious I think at this point or it's sort of self evident by the industry but that's what you're gonna do. You know kind of the interesting thing is there's this ton of stuff out there, right? So you know I can poke a little fun at this given Oracle just announced we're part of the CNCF now. This is the CNCF landscape page, I'm sure you've all seen it. There's a ton of shit up there, right? Like there's a lot of stuff. And there's awesome, right? Like it's you know we're exponentially better off this year than we were last year and like orders of magnitude than a decade ago. I can guarantee you. But the reality is none of that stuff actually works together, right? Like it doesn't. You have all these things that kind of go independently and some things are good and some things are crap and somebody has to put it all together and this is always much harder than everybody thinks. So you end up like as a team that you have to build this you know you're gonna build the an emoji service and we're gonna go kill it on the iPhone whatever it is now X10, I don't even know. You know you're gonna have to spend a lot more time than you think putting all this software together. Like this is the fundamental problem most places end up to and you end up with just a ton of reinventing the wheel. It's always very difficult for people to come up with an executable plan that gets them to meet their SLA goals. Frankie as I said at the very beginning a bunch of people don't do that. They kind of start with like well I'll put some stuff for Redison, no JS together and call it a day. That works for a while until you actually get some success and then of course it burns to the ground and you have Twitter fail whales or whatever the name is these days. So it's hard is the real point to this and I guess the point to now wrap this up or you know bring it home so everybody can go pet puppies. You know open source has always been about transparency and empowering end users. Right like back in the day I don't Chris raised the hands of how many people were not here before. You know Microsoft was the devil. Right like oh we're all getting locked into windows on our laptops and we hate them. We have to have Linux on the desktop that's coming in a year now. Awesome by the way Microsoft we love you now we know you're awesome. But that's what it existed for way back then. Right and now the world is a little bit changed like today the cloud industry and sort of the distributed systems industry has clearly been built up around open source and it has to empower teams like as it's doing today. So I'm not telling you this because it doesn't do it obviously it does but there's more we have to do. So my real argument to all of us in the room is we have to think about how we empower teams better going forward to build operation software build software that can be operable and it really means be consumed by one of these team by one of these organizations that's building microservices. So I actually had a ton of stuff on the first version of slide like we can talk about like what's hard with versioning and deployment of stuff I tried to boil it down to like a very small set of like three things that everybody could like kind of take away and the point of this I think is more to have a discussion than actually like a mandate. So this is just my mental model like number one this is really a reflection of what's hard. It's actually very hard for people to consume software and deploy it obviously Kubernetes and Docker really have changed the world and that does help a lot. So I think that one is the one that is the easiest line of sight to of great we have we do that that makes that problem a lot better but we still have to do it. The other two I think are not quite so simple. So observability is always like that when you have to operate software right like the the Yeager stuff is awesome it really is. You know we've you've all been building crap from scratch for the last 10 years to operate software. Observability really is like the root important thing you have to have but there's always kind of this thing I think that people think well because I get this like x-ray or microscope I now understand everything and the reality is you're not going to like in the system space we've had detracing system trap and all these other system tap all these other like amazing tools that help us understand what's going on in the operating system itself but the reality is unless you're a kernel developer or Brendan Gregg you're probably not gonna figure out what all that stuff is. So you know the real burden then is sort of like giving the person that has to consume this operation the software the ability to operate it in their environment when what they care about is building like an an emoji service or whatever. So I think the way that we get there is by saying as software authors like it's on us to pre-instrument software it's on us to like basically say okay I understand what people need to look for more than better than anybody I understand what people need to look for I understand what they need to do when one of these alarms gets caught certainly give me the trace points and tell me what the trace points are as ideally give me a dashboard that I can just quickly drop in and get going with like I think that's a huge one like more than anything that'd be the number one thing I think we as open source authors need to start focusing on when we build projects that are meant to scale and the other one is it kind of just goes hand in hand with that is testability and I don't mean that like when I run make test I get Unicode check marks or whatever the cool thing is to do these days it's really that I can do actual proper chaos testing and chaos testing as we all you know chaos monkey everybody kind of runs an AWS and it blows stuff up and that's actually kind of easy to reason about I always find like what's difficult is blackout versus brown out testing brown out testing is much harder to deal with so you know what what we have to do though is like and you know you have to assume that if you're building one of these online massive you know consumer facing web scale systems you're going to be doing production testing you're going to be blowing stuff up in production because if you're not doing that you actually have no idea if your system can recover like most places the production environment is so large you literally can't recreate it so it's the only place you can actually do this so it's on us as software authors to assume that that's the environment these things are going into and give people sort of the the explainability and the consumability to go run that and tell them like what to look for how it's going to break and like how to you know validate that's recovered and obviously this goes hand in hand with with observability so I told you to be a lightning talk I think I'm two minutes ahead of schedule so I'll get through this in 60 seconds or less so puppies how is Oracle going to help is I think kind of like tie this all back up so you know we're announced we're in the CNCF today we're going to make that mean something I'm sincerely we've already announced that we're putting a lot of people and a lot of effort into Kubernetes we've been doing a lot on Federation and SIG testing specifically we're going to keep doing a lot more there we've put out some tools ourselves that we find super useful specifically it targeted sort of operability and security so one of these is crash cart that's my personal favorite let's you kind of like debug post more to many crash containers it's awesome check it out it's on our github we're going to keep putting a lot into this this is you know very passion this is a super huge belief of mine that like operations is actually the hard part of building cloud systems and building microservices so this is where we're going to see a ton of effort from us we believe in it very frequently and you know of course open world is coming up that's like Christmas time at Oracle so there's going to be a ton of announcements but stay tuned to us around stuff around open source specifically and you know over the next year there's going to be a lot more from us coming on this so thank you everybody