 Who's excited for KubeCon and CloudNativeCon? Let me hear y'all. Woo! I like the wooers. That's really awesome. All right. So I'm Michelle Norrely. I'm all the things that Dan mentioned. Thanks for that intro, Dan. I'm also, so I'm co-chair of this conference. I've had a great time putting together the schedule alongside Kelsey Hightower. We've got lots of great sessions scheduled for you. In addition to normal talks and sessions, we're also going to be getting updates from SIGs. If you're wondering what a SIG is, SIG stands for Special Interest Group. This is how we in the CloudNative community get together and talk about specific areas of interest. There will also be SIG deep dive sessions. These are longer, more collaborative sessions for you to participate in, as well as for you to get to know one another in person. In addition to that, we'll also have technical salons today, tomorrow, and Friday. These are an opportunity for you to get your hands dirty. There are contributors and experts from all of the CNCF projects here at this event ready and willing to help you get started, answer any questions you might have, even walk you through tutorials. So really take advantage of that. And also, we work on these complex distributed systems, ourselves being distributed all over the world and across time zones. So take advantage of this opportunity. And I encourage you at any level of experience to take advantage of this opportunity to meet people in three dimensions today, tomorrow, and Friday. So the CNCF currently hosts 14 projects. This is insane, one, because the CNCF is less than two years old and two, because the CNCF was hosting just eight projects in March of this year at the last Cloud NativeCon and KubeCon event in Berlin. It's remarkable to me how quickly things are moving, how many organizations are coming together, and how much the community has grown. I mean, seriously, from up here, I have an amazing view. Raise your hand if this is your very first Cloud NativeCon or KubeCon conference. That's actually more than I was expecting. That's a lot of y'all. That's a lot of y'all, because the last event was less than half the size. And all of the conferences before this one don't even match the number of people that are here today. So I think this cloud thing is like working out, which is good for me. And check out what the Cloud Native landscape looks like today. Like Dan mentioned, the box items are official CNCF projects. Today, I'm going to prime you with some information about these 14 box projects so that you can have a solid foundation to take with you for the rest of the event. And also, so you're not totally completely lost when you hear all the acronyms. But let's get into how in the world we even got here in the first place. Well, we had the rise of microservices, which is this idea that you have loosely coupled services that implement business logic and are independently deployable. This is great for your organization because you can now have teams that have specialized areas of concern. And those teams are also enabled to deliver changes faster and more frequently. We also saw the increasing need to decouple the application from the environment that it's running in, so the infrastructure it's running on, with the rise of the cloud. And Amazon had a lot to do with that. It suddenly became a waste of time to manage your own physical infrastructure. And being in the cloud, renting servers, this became more and more of a default choice. And that in itself pushes us to decouple our applications from the infrastructure that it's running on. And whether you're a Solaris Zones OG or swore the container-native life because of Docker's adorable whale, you got to admit that containers have really pushed this movement forward and have allowed us to package our applications and services in a way that makes them more portable. And Kubernetes takes that concept of portability to a whole other level, because you can deploy, scale, and manage your containerized applications in public, private, hybrid, multi-cloud, and bare-metal environments. Kubernetes was also the first project to be donated to the CNCF. And it also inspired the creation of the CNCF because Google wanted to donate this technology to a vendor-neutral party. And from there, the CNCF has been building this powerful toolkit that enables us to build and scale these modern application architectures and environments. Look at where we are now. Three of the largest cloud providers announced their own managed Kubernetes services. Yeah, that's right, you can clap for that. That's amazing, that's amazing, because Kubernetes is less than three years old. In a week or so, we'll be announcing the release of Kubernetes 1.9, bringing us a stable core workloads API. 1.9 will also have beta support for Windows Server containers. That means customers will be able to run Windows-based and .NET-based containers on Kubernetes, making this space even more open. We'll also see a big win in the cloud-native storage space with CSI support in alpha. The container storage interface enables out-of-tree volume plugins in Kubernetes. Now, if you're thinking, well, why does this even matter? Let me tell you, enabling CSI support in Kubernetes lowers the barrier of entry for storage vendors wanting to support Kubernetes, giving us all more storage options in the long run and further preventing the risk of vendor lock-in. And like we're seeing with the CSI initiative, these higher-order systems like Kubernetes depend on other loosely-coupled cloud-native technologies and components. CoreDNS is a great example of that. CoreDNS is a generic DNS server that can talk to multiple backends, like XB, Kubernetes, caching middleware, and others due to its plugin-style architecture. Today, I get to announce that CoreDNS is 1.0, and it'll be available in 1.9, Kubernetes 1.9, on an alpha basis as a replacement for Kube DNS, adding even more features than we currently have and more performant functionality. There's a CoreDNS technical salon happening today at 340 if you're interested in learning more. Another component to know about is container D, an industry-standard container runtime. A container runtime is the thing that enables you to execute containers and manage images on a host. So it's a pretty central component in this space. Container D was originally built by Docker with feedback from the five largest cloud providers. It was integrated in Docker as a version 1.11, and you can even use Container D in Kubernetes via an internal API called Container Runtime Interface. And if you're not familiar with CRI, it's the thing in Kubernetes that makes it possible to swap out container runtimes without the need to recompile Kubernetes itself. And today, I get to announce that Kubernetes is also 1.0. I think I already mentioned that. All right, so, oh, no, I didn't. And today, I get to announce that Container D has reached 1.0 with a stable API exposed in GRPC. Yeah, that's really awesome. Container D and Rocket were donated to the CNCF at the same time in March in Berlin. Rocket was originally introduced by CoreOS. Currently, this team is working on Rocket Let, which is a CRI implementation to integrate with Kubernetes. This one project has had a huge impact on the cloud-native community. Rocket was built to implement the original spec for container runtimes called APSI, which was built for security in mind. And then that culminated into OCI, which is the current Container Runtime industry-accepted standard. Another project that came out of Rocket was the Container Networking Interface, or CNI. And most of this, take this one for granted, actually. Most of us take this one for granted. And this is the specification and API for connecting and improving containerized workloads from the network. It's used by Rocket, Kubernetes, OpenShift, Cloud Foundry, Meizos, and the list goes on. CNI is also approaching a 1.0 release. The maintainers are looking for feedback on what you'd like to see in 1.0. So if you have something, get in their issue queue and let them know. We've talked so far about projects and the CNCF that enable you to work with containers and make your workloads portable. And that's important for building the services that power your business. But just running containers isn't going to be enough. You're also gonna need intelligent ways to figure out what the heck is going on inside of these complex systems. I found it interesting that Twitter, a company known for its microservices architecture, has its own observability engineering team that focuses on this area. Let's switch gears and talk about a set of projects under the CNCF that fall into this category of observability. Starting with container monitoring with Prometheus. Here with us today is Tom Wilkie, Prometheus developer. Please help me welcome Tom. Thank you. Thank you very much. Hello, everybody. Wow, this is a very big crowd, isn't it? So Prometheus. Prometheus is an open source monitoring system and time series database, really specifically designed for dynamically scheduled environments like Kubernetes. It's got a powerful, concise query language, a really efficient storage engine, a really simple operational model, and was the second project accepted into the CNCF after Kubernetes. I'm really here today to talk mainly about Prometheus 2.0, the second major Prometheus release we did on November the 8th. Fabian noticed basically that with Prometheus 1.0, really highly dynamic Kubernetes environments would put a bit of strain on the storage engine, especially if you're doing things like continuous deployment and had a high churn of very short-lived containers. So he developed along with Brian and Gutham, who's in the crowd somewhere, a brand new storage engine for Prometheus, significantly improving the performance. This is a graph. The top line is Prometheus 1.5, the bottom line is Prometheus 2.0. We see about a 3x reduction in CPU usage, a 2x reduction in disk base, and over a 100x reduction in I.O. It's hugely performant, and these are real-world use case numbers that you'll get if you run it. Next, I'd like to talk about the Prometheus community. We've had an absolutely stellar year. We had some impressive stats from the Grafana Chaps who announced that our growth in the Prometheus usage has been over 5x since 2016. We've also, like all good GitHub projects, measure our success in stars, and we've had more than, well, we've had almost a 2x increase in our GitHub stars in the last year. Finally, we run our own Prometheus conference, a little bit smaller than this one. The latest one in August in Munich had over 220 attendees. That's almost three times bigger than our previous conference. I'd just like to thank Richard and Julius for organizing this. Finally, today, we've got a jam-pack schedule for Prometheus events. We're running a Prometheus salon at 10 past 11, various other talks, you can see them on the slides. Finally, we're going to finish up with some drinks at 8 p.m., go to the Prometheus blog if you want to sign up and find out more details about that. Thank you. Thank you. Thank you, Tom. All right. Here with us today to talk about logging and fluency is Eduardo Silva, software engineer at Treasure Data. Please help me welcome Eduardo. Hello, everyone. Wow, this is huge compared to Berlin, as Michelle said. Well, before to start talking about fluency, I would like to start talking about logging. And logging in general exists because just one reason is because you want to do some kind of data analysis, which means to try to get some insights from your own applications. And how do we accomplish that? There are many ways, but everybody knows that if we want to do or extract data from applications, we have some ways in which we take our applications and let our applications generate some messages to the file system or we ship these messages over the network to a distributed place, like a database or a remote server. But that's one thing. But if we want to do data analysis and if you have multiple applications and you consider now that we have containers, orchestrators, you have to consider that you need to centralize all of this information in one place. So how to solve that problem? It's not easy from the beginning, but every problem, if you understand it very well, you can come up with the right solution. And the strategy from an engineering perspective was, okay, we have a problem, we have different messages from different places, and we come up with a design which is called the Logging Pipeline, which said, okay, you have some messages that are arriving through some input. Maybe you need to filter this information, parse this information, store it somewhere, and then ship this information back. At the beginning, it's not that easy to understand, but basically you have an input and you need an output. But in the middle, when working with logging, there are many things that are really relevant. Data transformation is one of them. And that's what Fluendee is. A full solution like log aggregation, log transformation that allows you to connect to the data from one place to another, with a strong focus on logging, of course. And, okay, we understand what is the problem with logging, and most of you or half of you are using Fluendee, from my perspective. So we have some news. And today, this morning, our principal maintainer of Fluendee, Masahiro, has announced that Fluendee 1.0 has been released. Thank you. Thanks also to the whole feedback that we have get from the community. You, as an individuals, as representatives, companies, has made this possible. You have to understand that behind us a project, they are engineers, they are team, but also there's people who's providing the right feedback. You know what? You are doing this wrong. And from a project perspective, if we can fix that, it means that we are solving the problem better. And that's the whole thing. Today we are doing this, but tomorrow we have to make it better. So let's move forward to the, what else? Okay, 1.0 also comes with multi-process support. It supports also sub-second time resolution. We have included native TLS and encryption inside Fluendee. So it will be more easily to make the plugins ecosystem work in a secure way when sending data from one place to another. Also performance is a critical thing. So we come up with a new design for storing the data into the local buffers for reliability. And of course, when you talk about containers and cloud, also you have to consider that sometimes the logs needs to be sent to a different place, needs to be cheaper. And sometimes the network fails, you have power outage and things does not go well. That's why we have a protocol, which is called Forward, which aims to be used to send logs from one place to another. But also, that is not at all. It includes security and allows to allow us to do some kind of load balancing so you can have your own HA scenario in your own architecture. And when you talk about logging, logging is one thing. From the beginning it sounds like, okay, I'll look at the logs, but it's times more than that. And if you think carefully and you look at the future, you can imagine that the data is flowing from one place to another over and over. So the next natural step, of course, is to keep continue making Fluendee more integrated with services like Kafka for data streaming. So the strong focus for Fluendee will be to make more friendly with data streaming services like Kafka. And as of now, we have a secure Kafka connectors. It's very secure. We have a support Kerberos and all the things that in the Kafka site are required for security. Okay, so one thing is logs, the others connecting to data streaming. But also, when you think about how your systems are working, you would like to have some notions about how your logging pipeline, on this case, Fluendee, it's working internally. And that's why our investment has been also in a better integration with Prometheus. And we're happy to know that with all this coming out of 1.0, we are also improving our monitoring for our logging pipeline. So data streaming, monitoring, but also cross-platform is really important. That's why Fluendee 1.0 has been imported to Windows systems and can work reliable for your Windows services. And you can imagine from the beginning that this does not sound too natural, but there are many Windows services running around. And also, they have the same problem. They have a lot of feedbacks, a lot of log messages from different applications, and they need to centralize these, partition information, and ship this out. So that's why Fluendee 1.0 on Windows also is a huge win for everybody this year. And some community stats. You have to consider that Fluendee as a core project. I will say that it's a little bit small, but the huge value is in the ecosystem. You have to consider that we have more than 700 plugins available for Fluendee. And I'm happy to say that maybe the core developers maintain no more than 20. All of the older plugins are made by individuals and companies to solve different integration. And when I say plugins, I mean a plugin is used to collect information from an input to apply a special filter or a custom parser or send this information to a custom output or destination. And in our official repository, there are many repositories for Fluendee. We have around 50,000 Docker pools a day and growing. That is a huge number. And in the last month, it's increasing like crazy. And also from a core perspective, we have more than 100 of developers who has contributed this year. And from an ecosystem project or service, we can see that we have 500 contributors. So I would say that Fluendee is growing a lot. And that means it's not just about growing. Also it's about solving the problem better, as I said before. And I'm happy to say that Fluendee, it's becoming an industry standard for logging. And it's not just me saying this, it's the companies who are adopting this technology. More than 2,000 companies are using Fluendee open source in their own production environments. Okay, Fluendee is a project. We have plugins, we have community. But also I would like to say that Fluendee is a whole ecosystem. And as part of the ecosystem, we have more components. And always we are looking for, okay, how we can make it better? How we can improve performance? How we can make it lightweight? Sometimes we get some feedback. Fluendee, it's right, it works. But sometimes it's consuming a lot of memory. Okay, sometimes it's improved the configuration. But in the other scenarios, it's okay. There are critical environments that generate 1,000 and 1,000 of logs and messages per second. So how we can make it better? How we can make it stable? So I'm happy to introduce you to a new child project of the Fluendee ecosystem, which is called Fluendee. Fluendee has been developed for a long two years. It started as a lightweight solution to forward logs to Fluendee. But it got a lot of traction in the last time. And with a lot of traction, I mean the feedback from the users was, please, can you start integrating with this database? Or can I use InfluxDB? Okay, so Fluendee started to grow and also behave as an alternative to implement the same login pipeline. That does not mean that it's competing with Fluendee, but has a different focus for different needs. And Fluendee is reading in C language, which I said that is one of the most attractive things for most of the users. Also it has a pluggable architecture. We have 35 plugins. So if we have C developers here, you can start contributing your plugins, but also I have to say that you can write your own plugins in Golang. That is a really, really good thing. So we have some community members writing their own Golang plugins and being integrated inside Fluendee. And Fluendee also is fully asynchronous, event-driven, and of course, it has a small memory footprint. That is part of the core. But from integration with all services, this month we are releasing an next release, Theriodar 13, which will have full integration with Kafka, with Prometheus. It already can work really well on Kubernetes, append filters, you can append labels, metadata, and all that kind of information that is really relevant for the login scenario. So the roadmap is December. We're having the next version. And for the next year, we want to focus on a really new buffering mechanism and advanced filtering for everybody. And if you are around the conference, we would like to invite you to participate and get with the Fluendee community. We have the Fluendee Salon today. We have the whole Fluendee. Most of the core developers are here. They flow from Europe, from Japan and South America. So take advantage of that. So join us and thanks for coming and keep using Fluendee. Thanks. Thank you. Thank you, Eduardo. All right. We make a request in a distributed system. Tracing can help you understand the journey of that transaction. The Open Tracing project consists of vendor-neutral open standard and instrumentation for distributed tracing, making it easier to integrate within your projects and without having to be locked into a specific tracing vendor. The Open Tracing project consists of language APIs that implement the distributed tracing standard. The project recently added four new language APIs, bringing the total number to nine. This team has also been working on Nginx, SDO, Envoy, and LinkD integration. Open Tracing has a vibrant ecosystem as well. Check out these stats. Jaeger is a distributed tracing system that implements the Open Tracing API. You can use Jaeger to trace a single transaction throughout your architecture and visualize it. Jaeger was originally built by the engineering team at Uber. And this project's team members have been working on UI improvements, usability improvements for viewing large traces. It also supports multiple backends like Cassandra and Elasticsearch and has several new client or several client libraries with C++, the one being the newest. And they're currently working on integration with other CNCF projects. So we talked about all the container bits, which allow you to run lots of different services in a portable fashion. And then we talked about how you observe what's going on with all of those services. But the whole goal is for your system to act reliably in the first place. In the world of complex topology of services, this concept of a service mesh has become increasingly important to ensure reliable communication between services. Here to talk to us about one such service mesh called LinkerD is Oliver Gold. He's a maintainer of the LinkerD project and CTO at Boyant. Please help me welcome Oliver Gold to the stage. Hi, KubeCon or CloudNativeCon, I guess. Ah, how's everyone doing? Okay, before I talk about LinkerD, I know a lot of you have laptops out and phones and you're not really listening to me, so I wanna ask you to go to emoji.voto and just vote on your favorite emojis throughout my talk. Hopefully you'll do a really cool demo in about a minute. We'll see. So the last six months since I last spoke to you in Berlin, the LinkerD team has been exceedingly busy. We've done a release about every two weeks, but more importantly than the LinkerD team, you folks have been really, really busy. You're adopting LinkerD, you're testing it out, you're bringing it into production, you're writing really good issues and PRs that get to the heart of the things and are helping us learn a lot about the service mesh ecosystem. Most importantly, a lot of you have shared the challenges working with LinkerD, you've given really honest and constructive feedback, and you've told us about the challenges that your organizations and teams face in moving to new technology systems. And so I know it's a scary new world of new technology, but I really do appreciate that a lot, many of you have been so open in sharing your problems and challenges with us. And that's led us to think a lot about the service mesh and LinkerD's place in it and the future of the service mesh now that we have this word for what that is. And talking with many of our customers and users that we've kind of come on something new. Now I'm really happy to introduce a new project called Conduit. It is not related to LinkerD explicitly, except many of the people who worked on LinkerD and have been supporting LinkerD for years also worked on Conduit. There's a lot you can find out online about this, and I'm not gonna get too much into the weeds in this talk, but I do wanna try to show you what it does and why you might care about it. So let's see if this demo actually works. All right, so hopefully you've seen this page at this point. If you have a computer out and you've played with it, I would bet that if you've clicked around enough, you've probably found some problems. It doesn't quite work perfectly because it's software. And so if we look at Kubernetes, we'll see that we've got this service running. And Kubernetes says everything's running, it's healthy. It's green, wonderful. But we still have these problems that I can't know about. And so I'm gonna show you how easy it is to use Conduit to figure this out. First things first, I'm just gonna install Conduit. I'm not touching the running system. This is running in its own namespace, totally aside. Totally separate. You can see we installed Prometheus as part of that. And that'll start over the next few seconds. You can look and see it's running just a small little cluster there. And so the magic really happens when we add our new sidecar, our Conduit sidecar, into the running app while it's running and while you're clicking on it, and hopefully nothing goes down. That's the trick, huh? Okay, here we go. Boom, we're redeploying our app while you're clicking on it. And now there's a sidecar running dealing with all of the inter-service communication. And I just wanna start getting some information about what's going on in the system. And so I'm gonna do this watch command. And hopefully as the cluster rolls and as you click around, we're gonna start to see some things here. I can't force that to happen. This could be a Wi-Fi issue, of course. Oh, there we go. And we see some things staying out. Most of the things are 100% success rate. One of them is at zero. That's bad. What is going on? And so we see request rates and we see latencies, latencies quite low. And below, what can we do to find out more about this failing endpoint? And so what I think is maybe the coolest part of conduit or really makes me happy is this tap command. And what tap does is it sends this query. I say I wanna look at this one method. It sends this query into all the proxies as they're running and fetches them out and puts them right on my CLI for you if the wireless works. Oh look, immediately. Thank you. Yeah, we see we have failures. These are GRPC failures. We see a 200 status code. So most things would say this is fine but we see a GRPC status of unknown. We have found the problem. And this is all running live, just pointing an RPC into the proxy and pulling data at it. It's beautiful. So that must, you know, most of you link to users say, well, that might be at least 100 megs to do that, right? Well, no. We've got much better. Here we see the proxy running at about a meg. That's really good. That's the, yeah, really, really cool. Oh, one more thing. This also has a really beautiful dashboard that I don't have time to show off. It does a lot of cool stuff. And it's brand new. Okay, I'm ready to go back to the slides. Oh, thank you. So Conduit does not replace Linkerty. Well, Linkerty is supported. We're gonna continue to make it better, make it work for everyone. Conduit is a new kind of clean break for us to really focus on the lightweight zero configuration aspects of what we think a service mesh should be. We're gonna be talking about a bunch of the conference. We have a booth, the buoyant booth, over in the sponsor area. There's a panel, my co-founder, William's on right after this. And I'll be doing an Linkerty salon tomorrow where I'd be happy to talk about any of the details of this. So please go to Conduit.io, check it out, download it, run it in your mini cube, and give us feedback. That's really what we want. So yeah, thanks again everyone, have a great day. Thanks Oliver. All right, let's talk about Envoy. The network is inherently unreliable. This problem gets even worse and more painful in a complex system of distributed services. Envoy is an edge and service proxy that makes the network transparent to applications. It runs alongside an application and abstracts the network by providing some common features in a platform and language agnostic manner. Envoy was originally built at Lyft and ended up being deployed as an edge proxy and then as a sidecar service or service proxy forming a mesh over hundreds of services at Lyft, transitioning millions of requests per second. When all service traffic goes through a mesh like this, it becomes much easier to visualize the problem areas, tune performance, and ensure that your services are communicating reliably. Envoy is at a 1.5 O release today with a GRPC v2 API now production ready. Envoy is also experiencing amazing community growth. That project is now at over 100 contributors. GRPC is an open source RPC framework released by Google. In GRPC, a client application can directly call methods on a server application on a different machine as if it were a local object, making it easier to create distributed applications and services. Google has benefited from having something like GRPC internally, something like a uniform cross-platform RPC infrastructure that has allowed them to roll out fleet-wide improvements in efficiency, security, reliability, and behavior analysis, a key for supporting the type of growth that Google has experienced. The team is currently working to support more languages and improving connectivity and performance. The community release cadence is really strong and there's several sessions on GRPC today if you're interested in learning more. So so far, we've talked about projects that enable you to run containers in your cloud-native environment. We talked about how to observe what's going on with these cloud-native applications. And then we talked about ensuring reliability with your services. This last one is important because nobody likes to be stepped, not even your microservices. Those are my friends. Yeah, there's more cheesy cloud-native metaphors coming up by the way, so get ready. Last but not least, let's talk about two projects new to the CNCS in the realm of security. One problem with the internet is that everyone's trying to lie to you. And this gets even more terrifying when you're trying to update your software. I don't know if you've noticed lately. TUF is a software update spec perfect for people who are paranoid. It expects that attackers will compromise server's keys and anything that they can get their hands on. The goal is to retain as much security as possible when that happens. TUF has been making strides in the IoT and automotive space as well. This team has been working on minor extensions with the community around key rotation and other security feature enhancements. Notary is an example of a project that has adopted TUF. Notary can be added to any digital distribution system to ensure the secure download of packages and we look forward to a 0.6 release from them very soon. The Docker platform, the Moby project, Motorola Solutions, VMware, Linux, KIT, Quay, and Kubernetes are all using Notary in TUF. I'll just end with, it's super important for us as a community to provide a solution for trusted cross-platform delivery of content to ensure no bad blood at the end of the day. I know that one's bad too. If you want more cheesy, cloud native metaphors, I'll be here all day. That's all I have for the project updates. I hope that you learned something and it's really cool to me to see this community grow. A lot of these projects that are coming up have been around because people have been experiencing all these problems in a cloud-native environment, but they've just, we're coming together, we're learning from each other, we're sharing problems and solutions, and yeah, it's cheesy, but it's really cool. And so I'm really excited to have all of you here today, your first cloud native gone in KubeCon conference. And next up,