 All right, this looks like a really, really awesome setup. There's a reason why these books are here. What can I tell you? But you will figure it out. All right, welcome to the upstream panel. My name is Michael Hausenblass. I'm a developer advocate at Red Hat. You have to earn it, Chris. And we have an awesome team here. We have Christian here. We have Chris here. We have our very own Paul here, who you already know. We have Gary here, and we have Dan here, who you also know from early on. You probably remember, right? Do you have your swear jar still around? Yeah, it's down there. We need the swear jar. Is that what that is? I'd have to use bills in a minute, though. All right, so the format is, I have a few questions like simple, easy ones prepared. I'm going to expect that you give them the hard ones, right? Can you give them the hard ones? Oh, come on! All right. I'm going to start the beer right now. Oh, yeah. Can we have beer for everyone? Okay. It's a quick warm-up, like everyone, maybe, you know, a minute or so, about your background, who you are, what you're doing, and, yeah, what do you do for a living? Let's start here. Hi, I'm Christian Posta. I'm a chief architect of cloud application development at Red Hat. I work closely with our customers to help them build distributed systems. I guess we're calling that microservices, DevOps, all those fuzzy word type things. I have a background in integration and messaging. I was a contributor to projects like Apache ActiveMQ and a little bit of Kafka. And I enjoy working on open source and helping our customers. So what's that? The project that I've been pretty excited about for the last eight or nine months is called Istio. What's that? I keep on asking you. Oh, I wasn't sure if you were... No, no, no, I want to learn. So Istio is an implementation of an architectural pattern we're calling service mesh that moves some of the more complicated distributed systems, patterns that we have typically associated with language frameworks or centralized orchestration engines and so on into a decentralized set of infrastructure. Cool. I expect we're going to learn more about Istio. I hope so. Chris. Who are you? What do you do for a living? My name is Chris Anizik. Excuse me, I just got off the plane so trying to revive myself here. But I have the fun job of running the Cloud Native Computing Foundation. So I serve as CTO there. I'm not sure. People are familiar with the organization, but projects like Kubernetes, Prometheus, you know, essentially live inside this foundation where we support these projects under a neutral setting where a bunch of companies come together without worrying that one entity controls everything. So we put on KubeCon, Cloud NativeCon and so on. In previous lives, I've worked on Fedora, Gen2, done a lot of stuff in the JVM, worked on Eclipse a long time ago. I've been involved in open source for quite a while. So it's kind of fun kind of being in the foundation neutral party setting these days instead of working for a company. And kind of like everyone joined the CUNSF? Everyone's welcome to join. We especially love end users. So for us, obviously we have a lot of vendors involved since they were kind of some of the progenitors of a lot of these projects, but we definitely love seeing companies using Prometheus or Kubernetes at scale. And we love to have our end user share. Kind of their horror stories away from vendors just amongst themselves. But feel free to reach out to me if you have questions on that. And if there is no Cloud Native Meetup yet in your town or city nearby, I consider doing one. I'm running one for Ireland. Yeah, definitely we sponsor Meetups. If there's a city that you're involved in, we help pay for pizza, beer and all that wonderful stuff that's required to fuel these things. Awesome. Paul needs no introduction. Next. Tell us a little bit more. What do we have to learn yet? So I work, in addition to Service Catalog, I also work on Multicluster at Red Hat. I have a team actually that works on both Service Catalog and Multicluster. I've actually been working on OpenShift since OpenShift v1. And it's been quite a journey. Things have changed pretty significantly. Before Red Hat, I worked in equities trading and health insurance enrollment. Awesome. So we know from then that he doesn't like the D word. But what do you like? I guess now I like the C word. Briar? You like to freeze out? Anything you want to add to early on, no? No. That's fine. That's cool. Gives more time to Gary because Gary represents an awesome project, the Jäger project. With my German background, I say Jäger. You say probably Jagger. No, Jäger's fine. Jäger, all right. Take it away. Yes, I'm Gary Brown. I'm a principal software engineer. I work for middleware management group in Red Hat. The main area that I work on with some others is distributed tracing technologies. For almost two years now, I think we've been involved in a project called Open Tracing, which is one of the projects under the CNCF foundation. And more recently, Uber, they open sourced the project, which is an open tracing native distributed system called Jäger. That was open sourced in April last year. And more recently in December, that also became a CNCF project. So we're working very closely with Chris on those. Cool. The one-on-one on what is distributed tracing, I would say if you have your developer mode in the browser, and you look down there, you see all these lines. Kind of that just for many machines. How do you sell distributed tracing? Distributed tracing has become more and more important in a cloud-native environment because in a highly distributed application, where you've got a large number of microservices, it becomes very difficult to actually understand how your application is executing, what performance problems are, maybe where the faults are occurring. So distributed tracing essentially allows you to capture an execution, the execution path of your business transaction as it's going through the microservices. And it does that by passing context information with all the requests that are propagated through the microservices. So you can see an overview how something flows through a different microservice. You can use it for troubleshooting, performance issues, whatever. Cool. Awesome. I have a first very easy question, and then I hand over to you. Dear audience, so prepare yourself. Warm up. So what... The first open-source project you were active in, any kind of governance doesn't matter. And any kind of... It doesn't have to be code or whatever. The open-source project you contributed to were active in. Let's start in the middle here. Paul. Open shift. Seriously. I never doubted that. Dan. I don't know, it's so long ago. Vax. Red Hat Linux, like GNU-C, I don't know, it's back probably before Red Hat. So, been a long time. Nice. Chris. Probably Slackware a long time ago would be my guess. I had to do it for messes, but okay. A project called Spring Integration from the Spring communities. Cool. Gary. I think the first one was an Eclipse plugin for a W3C choreography description language. Awesome. A long time ago. Distribution here. First question. Actually, I work back on Kerberos, X-Windows, Project Athena, so back. Love it. Back when they were being done at MIT. I'm friggin old. We're roughly the same age, right? mid-40s. First question from the audience. Who has a question for the gentleman? Go on, guy. Whatever. There is, oh, wonderful. Can you please with the best? So, you figured the pattern, right? You ask a question, you get a book. Hello. Actually, we should have heard the question first. I have some Esteele questions. I was just wondering, is it so already to actually roll out and to put into play? So, I've been playing with it in my own name space with a demo app that comes with it. But I'm just wondering about putting it into our dev integration environment and then do I go beyond that? Okay, so I'll take that first. I would say you should be playing with it. Maybe POCing it. Exploring how it works. I would not put it into production right now. I think there's still a little bit more that has to happen in terms of hardening, in terms of performance, in terms of stability and security that need to go into place first. Cool. Cool. You don't get a second book, right? That's fine. That one looks big enough. Also, with the Steele, I noticed while installing it, it seems to have it's watching for events. I've noticed that I tug at it for my own name space but it seems to be creating secrets in all of the name spaces. So, I'm just kind of wondering what does it actually do? What is the impact of installing it? So, I see these secrets appearing everywhere and people asking me questions what the hell is this Steele doing? I'm just wondering if there's other things. It's installing secrets. Oh, when you have the certificate authority stuff turned on or just in general? Just in general from what I've seen. Well, so Istio has a component in the control plane that's responsible for certificate management and certificate distribution and rotation and we use that internally to build features like mutual TLS between services and from that perspective there are going to be certificates there's going to be secrets installed into the different name spaces where your applications might be deployed for the purposes of mutual TLS those should be mounted into the different pods but if you're seeing other random stuff then I'm not really sure maybe you could open an issue Let's continue the discussion over the beer it sounds like a very many many questions it looks like a very deep dive here Can we do that over the beer? Yep Any other questions in between? There's a question in the back Hi, this is probably a question for Chris How do you successfully structure a neutral company that brings other people together? So CNCF is technically a non-for-profit we have a 501C6 organization under the Linux Foundation so it is a neutral entity based on US law so like every all the IP developed and associated work is forever will be part of a non-for-profit like CNCF so by law all that work is neutral the organization is done which levels of membership governing boards, technical boards and user boards and so on and those are all kind of baked in when we form the organization but essentially the rules are done in a way where we try to give everyone a voice from our end users to our technical people and to the people that actually are writing the bigger checks to fund kind of the good strapping organization I hope that answers your question to kind of define their own governance and independence in some way we don't force integration amongst projects you'll see projects like Yeager, Prometheus and Corinides all kind of differently define how they run all they need to really have is a published governance document and fairway to elect additional maintainers that's all we require of our project which is very different from other kind of foundations out there but we'll get a lot of wrong so if you want to go ahead, sure I remember a while back I read a really interesting paper called Pivot Tracing and I wonder if I remember rightly trying to characterize it it was a tracing framework that carried information around at runtime to establish causality between RPC calls is tracing going to be able to learn from that and build on that or what sort of direction might it go in the future I don't know the working detail but I mean from what I understand one of the core principles is being able to propagate certain information from an upstream service downstream so that you can then add value to the information you're recording locally in open tracing there's a concept called baggage that is primarily intended to do that so you can as well as propagate in the trace context the information between services and add your own information so I think the Pivot Tracing mechanism they provide agent technology that will kind of work out what needs to be done for you so I think the same kind of mechanisms could be built on top of open tracing I mean as part of the open tracing project we also have a Contrib repo where there's a large number of instrumentation framework instrumentation done so we have the beginnings of a Java agent project there so maybe that's something that we can add on top of that Awesome, thank you Okay a quick one from my side if you compare infrastructure stuff like for example cryo, monitoring system level and so on on the other hand app level stuff different kinds of programming languages middleware stuff and so on which one in your opinion which one is more important and or covered in the context of Kubernetes or CNCF right now and maybe what should change is something lacking whoever has Was the question clear? No, it was too long More important to whom? Well as an expert do you probably be able to say there's a spot where there's something missing or is everything covered already? Infra versus like DevOps working together that kind of thing Yeah, I'm trying I could formulate potentially an answer I mean you know from my perspective I'm also involved in other initiatives out there there's something called the open container initiative which I also help spin up in parallel with CNCF but that's mostly focused on container standards in my opinion my hope is that the infrastructure stuff just like it's there Kubernetes essentially is going to be the analogy we like to use is like POSIX of the cloud or Linux of the cloud it's just going to be the default distributed systems API that every cloud supports that applications will actually take advantage of so in the future app developers probably will not care how infrastructure is laid out so to me I see the app layer becoming more exciting in terms of how we actually write apps now that we kind of have this Kubernetes being POSIX of the cloud for lack of a better analogy so that's my perspective at least but I've been in the thick of this standardization and making Kubernetes pretty much available in all different clouds since for the last couple years so you're going to say something full, sorry Yeah, so my own personal theory is that I'm calling this the quantity theory of importance there's only 100% importance total right and the things that you're working with say as a developer are important to you day to day and sorry Dan but unfortunately you work in an area that becomes important to people when it breaks and so you mean like meltdown inspector? Yeah so if Dan here does his job right you'll never even think about it and Dan's very good at his job so odds are you probably won't think about it and if things go well you won't have cause to so day to day most people will relate the most to application level things and since developers tend to experience and develop strong opinions about those things Really? What do you mean? You sound crazy I have some empirical data to support this though that's basically it you tend to think the things that you work with most are most important but as soon as something breaks important, bam Any other opinions Dan? Yeah One of the things I think is interesting is I think there's a constant sway between for the first couple of years when I was working on containers it was such a focus on devs and ops was just a total afterthought and so last couple of years I've been more concentrating on the ops side saying how do we make these things more stable and you weren't here for my earlier talk but my goal is to make the ops side boring cause it just works and so I tend to agree that we need to continue to work on the dev side what I want to see on the dev side though is to lose the focus on doing this engineers are too concerned about building containers and doing things if the primary focus of developers is on doing containers we sort of lost the primary focus on the developer should be on building his app building his cell phone building the equivalent of cell phone apps and getting micro services to that level and I think when we get to serverless and some of this other stuff that will hopefully happen but right now I see too many people saying how do I get access to the docker socket or how do I get access how do I build the docker file what the hell are you doing you're a php app not only did he swear but he said docker at least it would be time I stopped counting anybody got any change any other I think with the availability of this type of infrastructure that we're talking about this elastic infrastructure on prem or in the cloud I think once you move up to the level of the application not only has the stuff on the ops changed but the way we build and design and architect those applications on this infrastructure has changed also and I think as we start to make the lower levels boring we're going to start to move up and say well now we got to tackle these distributed systems problems at this level at the application level and I think there's a lot of opportunity for improvement there I think service measures are a great example for that as well before you folks get another chance to earn books quick question who does not know what upstream means okay who knows what upstream technically everyone should upstream from our point of view from vendor point of view that is the open source project where we draw the things and build our product from just in case that was not clear so open source audience question way in the back not sure he needs this probably he writes books on his own but it doesn't matter still a good read maybe you could write this book trust me I do need to read it I've got a very broad question that I think applies to the whole panel how do you keep up with everything that's going on in the container space I know that for myself I get an email every day from someone in my organization saying what do you think about technology X and I'm curious as to how you manage this challenge I'm also curious a lot of airplane trips and uh and it tips for I would be lying if I said that I actually caught up with the entire container space yeah I think it's impossible to know at least in depth of what each technology does and so on I kind of have a unique perspective where I have a very broad perspective on the tools out there and what people are working on but it's hard to have I'm not a security person so I have really no idea how container security works I think long plane rides are good ways to catch up on things and diving deep on the projects that you kind of care about you know, background that's how I try to keep up I'm aware that I'm not on the panel but it's part of my job and I do it, I believe that it's a full-time job right, you could just the whole day just consuming stuff but like you also do hang out on Twitter a lot and Slack or whatever you catch up with you know, oh have you seen that, I'm using that and that's probably a really good way to filter let others do the filtering and you do the filtering in your domain for others, newsletters or whatever that makes him quite popular again yeah, I mean in CNCF we try to at least our projects tend to be the projects that we think are good examples of projects in relative domains you should look at or care about but the landscape is massive, like there's so much stuff we do it also internally like newsletters for certain domains saying that happened last week so it takes you like 10 seconds to go with that and pick out the two or three things that might be interesting for you what my hope would be is that you could come and watch some of the weekly briefings because a lot of those what I try and do is hit those topics so that you don't have to and can watch them in the leisure of your own homes on YouTube of course the best answer is what they have just said we have another question Kites we have two instant delivery so a lot of talk about technology the bit that concerns me more with financial services is the whole enterprise workflow side of things I think it's probably a question for Paul more than anything the service catalogue and how do you actually decide what kind of recommendations you've got from a security perspective or how do I know what I can put in my production cluster a lot of effort we have is around saying well that's suitable, that image we can certify it that can go in the service catalogue, that can't it's great allowing all this autonomy for developers but we find that it's the same group that's emerging code review and they're just becoming more and more pressure to do the quality control and how do we handle that side of it it's not just a technology problem bit like Grafias, Paul are you okay? yeah so that's a really good question I kind of look at the if I play the movie forward like 6-12 months in an organization like financial services that has important requirements for audit that I start looking at things like services that you might integrate with service catalogue as things that themselves have to be certified and have to go through does it scale very well just having to certify everything we'd like something how do I carry on from that and make this scalable so that I can actually put the number of services that we'd like to put on the platform passing all those audit requirements et cetera so presumably a lot of the services that you might choose to integrate are ones that you already have and they're currently being wired together via some form of credential sharing I'm sure not in anybody's organization here but maybe post-it note so there's already