 So good morning, good afternoon, or good evening, depending where you're watching this today. My name is Alex Kirkup, and I'm one of the co-chairs for the CNCF storage tag. And we're talking today about cloud-based storage, the landscape, and some of the projects in the CNCF. I'm here with my co-presenters, Xin Yang and Raphael, and I'll let them introduce themselves. Thanks, Alex. Hello, everyone. Xin Yang, I work at VMware in the cloud-native storage team. I'm also one of the co-chairs in storage tag. And my name is Raphael Spastoli. I am a consulting architect at Red Hat. Cool. So let us talk a little bit about what we're covering today. So we're going to cover an overview of the tag and talk a little bit about how you can take part in the community and how you can contribute. We're going to have a little look at why cloud-native storage is something you should be considering and why that's important. And the review of some of the white papers that we've been working on in the tag, including the cloud storage landscape documents, our data on Kubernetes documents, the performance document, and the disaster recovery document. So a little bit about the tag. The tags in the CNCF are the technical advisory groups. They help advise and act as subject matter experts in different areas that the CNCF works in. So we're the storage tag. There's also other tags like security and runtime and app delivery and networking. Our meetings are all open. We would love to hear more from you and to hear from maintainers of projects as well as other people who are interested in storage. All of the calls are open. They're on the second and fourth Wednesday of every month. And those are the links, which you can use to join the meetings and look at our previous minutes and presentations. A little bit about who we are. It's a mixed group of people. Some represent vendors. Some people represent projects. And some people are independent. We have a number of co-chairs, the three of us, which are in the group today, as well as a number of technical leads. So Louise, Sheng and Nick who join us as technical leads. And each tag also has a number of members with the TOC that we work with that are liaisons for our tag. And that's Nikita and Matt in our case. So if you have any questions, please feel free to reach out on our main list. And we're also on the CNCF Slack groups. So a little bit about what we do. As the CNCF continues to grow and the number of projects within the foundation continues to grow, one of the challenges that we're trying to overcome is the sheer amount of scale and diversity and variety of projects in the CNCF. So part of the reason why we have the tags, it's to help the CNCF scale, to help provide extra levels of expertise for the TOC as we continue to grow the huge interest in cloud native computing. So what does this actually mean? We do a number of things, including educating like in sessions like this, but also in the white papers and various documents that we writes to kind of explain the ecosystem and explain the options and the projects in the environment. We work with the TOC to review projects that are going through the various phases within the CNCF. And we obviously engage with the user community whilst we provide the subject matter expertise. There are already a large number of projects within the CNCF in the storage ecosystem. We have a large group of projects at the sandbox stage and you can click on the link to have a look at some of those sandbox projects. And we also have incubating and graduating projects. Incubating projects include projects like Dragonfly, KubeFS and Longhorn. Dragonfly provides a container services. Longhorn provides a variety of hyper-converged block services and KubeFS is a new, interesting distributed file system. And then in graduated projects, we have everything from projects like at CD, which is a key value store that you're all going to be familiar with in the running of your Kubernetes clusters. Rook, which is an operator for Ceph. Vitesse, which is a very large scale distributed MySQL option. TIKV, which is a distributed and sharded key value store and Harbor, which is another container system. The sandboxes go through different levels. Most projects start their voyage in the CNCF as a sandbox. The sandbox projects are there to help build the community, sometimes just provide experiments and help guide the foundation and the project. Incubation projects are where projects reach a maturity level where they're used in production. Incubation is also where the vast majority of the due diligence takes place. And the projects have to be in use in production in a large number of sites to get to incubation status. And finally, the last step is graduation where we have a final phase of security audits and other final checks on the governance of the project, for example, to get to that final stage. A little bit about cloud native storage. So why are we talking about cloud native storage and what does it even mean and why should you be interested? So ultimately, and I'll be a little controversial here, but I'll come out and say it, I think every application needs to store state summary. There is no such thing as a stateless application and we talk about the way we run applications in Kubernetes, but ultimately your databases, your message queues and everything else at the database, sorry that the application needs to run, needs to be operating somewhere and they can operate using cloud native patterns and they can obviously operate in the more traditional sense. So as organizations mature, they're cloud native environments, they're moved to Kubernetes. They want to adopt the capabilities and the facilities that we have in our declarative environments in Kubernetes where we use those cloud native patterns to enable automation and scale and facilities for failover and performance. And we'll talk a bit about some of the disaster recovery and other functions here. And certainly we're now at the stage where there's a very broad ecosystem that supports cloud native storage with very robust and mature CSI driver capabilities within Kubernetes itself, as well as a vast variety of operators that allow you to run databases, message queues and many other services in Kubernetes too. So the time is now to come and try these things. So I'll hand over to Jing now who's gonna talk a little bit about our storage, landscape white paper. Thanks, Alex. Now I'm gonna talk about the CNCF storage landscape white paper. In this white paper, we described storage system attributes, different layers in a storage solution and how they affect the storage attributes. We talked about the definition of data access interfaces and management interfaces. Next please. Storage systems have several storage attributes, availability, scalability, performance, consistency and durability. Availability defines the ability to access the data during failure conditions. Availability can be measured in uptime as the percentage of availability. Scalability can be measured by the ability to scale the number of clients, throughput, the number of operations, the capacity and the number of components. Performance can be measured against latency, the number of operations per second and the throughput. Consistency refers to the ability to access newly created data or updates after it has been committed. A system can be either eventual consistent or strongly consistent. Strong consistency is synchronous. Eventual consistency is asynchronous. Durability is affected by the data production layers, levels of redundancy, the endurance of the storage media and the ability to detect corruption and recover the data. Next please. There are several storage stacks or layers that can impact the storage attributes, including host and operating system, storage biology, data production layers, additional data services provided by the storage system and the physical non-volatile layer. That's all for the storage landscape web paper. Next please. We are collaborating with the data on Kubernetes community on a web paper that describes the patterns of running data on Kubernetes. The paper is complete, ready for review. In this paper, we talked about attributes of storage system and how they affect running data in Kubernetes. We compiled running data inside versus outside of Kubernetes. What are the common patterns and features used? And in the first version of this paper, we are focusing on databases. However, most of the things we described here apply to other data workloads as well. Next please. We talked about the storage attributes of a storage system earlier. For a cloud native database, the kind of a backend store used the number of replicas and so on all have an impact on attributes such as availability and durability. Cloud native databases typically use Sharding to facilitate horizontal scaling. For example, Vitas graduated CNC project has a built-in Sharding feature to scale MySQL horizontally. We added two new attributes, observability and elasticity here. In cloud native environment, typically there are different services running in a distributed fashion. When a failure occurs, it is hard to know which component is causing the problem. It's even more important to have comprehensive observability system. This will help us detect problems early and prevent a failure from happening. Elasticity means the ability to scale up and down quickly. This is on-demand infrastructure. It refers to the ability to release resources that are no longer needed. This can also mean storage tiering where you can move data to different tiers depending on how often you need to access them. For storage stats, I mentioned earlier that the different layers of storage system can have an impact on the attributes. This is true for running data in Kubernetes as well. Regarding the art, Rafael will talk more about this one later. Next, please. There are options to run data inside or outside of Kubernetes. Deploying and operating databases without proper automation is not recommended. So there are many two alternatives. There are major databases services provided by most cloud providers. You can also run databases inside Kubernetes. This can provide multi-cloud and cross-cloud capability. Running databases inside Kubernetes is typically facilitated by an operator. It leverages Kubernetes declarative API and reconciles the active state with the desired state. With databases managed by a Kubernetes operator, it automates day-to-day operations such as backup and restore, upgrade, migration, et cetera. It can also leverage other tools such as permit fears and Ravana for monitoring. Next, please. When running data in Kubernetes, we see many common patterns and features being used. And operators are usually used to facilitate the running data in Kubernetes. There are things to consider when writing an operator. When you decide what configuration knobs should be visible to the user, there are trade-offs between flexibility versus complexity. An operator should handle periodic operations, support, upgrade, especially non-destructive upgrades, handle different CRD versions and so on. Many operators use persistent volumes to store data in Kubernetes. Those persistent volumes are typically provisioned by a CSI driver. CSI driver allows underlying storage to be consumed by containers running in Kubernetes. That's all I want to cover for the data on Kubernetes white paper. Now I will hand it over to Alex to talk about the performance white paper. Thank you, Jing. So as we built up the various white papers and we discussed the attributes, after availability, one of the most common things that we get questions on is performance. So we started writing the performance white paper and you can see more in the link on screen now. In the performance white paper, we look at some of the definitions and the terms so that we can bring you up to speed on a lot of the concepts that we can use for measuring performance for both databases and volumes. And then we pick those two as they seem to be the most in demand at the moment. Some of the things that we cover during in the white paper include understanding the differences between, for example, the operations per second which are typically needed in applications which might be latency sensitive or do lots of small operations together. So, you know, think database queries, for example, versus throughputs where you're probably doing a smaller number of operations with larger data sets and are more focused on the amount of data that can be processed perhaps measured in megabytes per second or gigabytes per second. So think more analytics or data warehousing type operations. And of course, in all of these systems, the way the storage is laid out, the data protection in terms of, say, erasure coding or replicas, the data reduction in terms of D-Tube or compression or even the security in data encryption all play a part in the overall performance. And there are often compromises to be made across all of those. But ultimately, the latency of the different components is probably the single biggest determinant, both in terms of throughput and operations per second. And because latency is that challenge, we have to understand how to make the storage environments as concurrent as possible and that involves, you know, managing the depth of the queues but also the number of queues and the number of clients and the number of backends that the system can scale across. If you only have one thread hitting one connection, there's only so many things you can do in parallel because you're serialized by that single connection. So concurrency is one of those big challenges but the other big challenge is understanding the caching at each of the different layers, you know, and you have caching at file system, database, often storage system and even at the individual disk level and therein lie some of the traps, I guess, if you wish, in how you understand the performance benchmarking for your environment. So when you're testing that, it's very important to understand if you're testing a dataset that actually does need to fit in memory and does need to fit in cache or whether dataset is going to be expanding and be larger than that cache. The most important takeaway here is not to trust blindly the numbers that any particular storage vendor or database vendor use in their environments because your environment will differ in many different ways and it's important to test your own application in your own environment with your own use case when you're comparing different systems. In fact, as we were writing this document, we realized that performance benchmarking is actually pretty hard and there are so many caveats to be aware of and there's also a large section on caveats within the document. And with that, I'll hand over to Raphael who is going to give you an update on our Cloud Native Disaster Recovery paper. Thank you, Alex. Yeah, so we wrote this white paper, the Cloud Native Disaster Recovery White Paper to describe an approach to doing disaster recovery which of course we call Cloud Native Disaster Recovery. This is an option that we think is important and that we think people should know about. It's not obviously, we are not saying that you have to use it but we think it's a good option, we think we should consider it. So let's see how Cloud Native Disaster Recovery is distinguished from traditional disaster recovery and we have this table here to point out some of the differences. So for example, for the deployment of your application with traditional disaster recovery, you often end up doing active passive, especially for the stateful application for the storage application which is what we're talking about here. And instead, Cloud Native Disaster Recovery, we say you should do active active, you should pick a product, a middleware database that can do active active. For the trigger of detecting the disaster and the trigger of the recovery procedure with traditional disaster recovery, it's usually a human decision. With Cloud Native Disaster Recovery, this middleware software can detect a problem autonomously and start the recovery procedure autonomously. And under disaster recovery procedure itself, it's usually in traditional setups, it's usually a mix of manual and automated tasks. If you're very good, maybe they're all automated, just the initial trigger is manual but otherwise it's a mix. In Cloud Native DR, it must be, everything must be automated. And then the two main metrics regarding disaster recovery, which are RTO and RPO, the recovery time objective is how long it takes for your system or service to come back up after a disaster and the recovery point objective is how back in time, do your data goes in terms of transactions that you lost because of the disaster. So for traditional disaster recovery, you will have between zero and a few hours, depending on the disaster recovery procedure for RTO, for recovery time objective. But for Cloud Native Disaster Recovery, we want it to be close to zero. And essentially it's not zero just because there are some health checks that need to detect that there is a problem in one of your availability zone or in one of your failure domains and these health checks then start directing the traffic to the healthy locations. And then for recovery point objective, again, depending on how you implement your traditional DR, it could be from zero to hours, but for disaster recovery, for Cloud Native Disaster Recovery, we want it to be exactly zero, especially for strongly consistent middleware. And then who is the owner of the disaster recovery process? So in terms of business continuity, the formal owner is always the application or the service, but what often happens in traditional organizations is that the application team just looks at the storage teams and inherits their disaster recovery procedure. So the de facto owner ends up being the storage team. For Cloud Native Disaster Recovery, we want the application team to be the owner. They will design, they will use the underlying platform capabilities to design the disaster recovery procedure, but they are the owner of the disaster recovery procedure. And then what capabilities do we need? What technical capabilities do we need? Historically disaster recovery has always been a part of, implemented with storage capability, like backup and restore or volume replication, whether it's a synchronous or synchronous. But what we found out to implement this Cloud Native Disaster Recovery architecture is that we needed the capability mostly from the networking domain, and especially the ability to do east-west communication so that we could stand up cluster of storage or cluster of middleware, stateful middleware across different regions or different failure domains, and that we needed strong, smart global advances. So global advances that have health checks and are able to swing the traffic when there is a disaster. Okay, next slide, please. So what can you find in this white paper? So we have this Cloud Native Disaster Recovery approach, definition explained a little bit with more detail than what I just did. And then we have some fundamental definitions of the terminology that we use in the white paper, like failure domain, HA and DR. Then we talk about the CAP theorem, which is the theorem that underpins the behavior of stateful work or distributed stateful workloads, which is what we need to use if we wanna go with active, active approaches. Then we take a look at the anatomy of these distributed stateful workloads. They all have similar characteristics, so it's worth knowing what they are by understanding what replicas, shards, and shards are. You can easily figure out the behavior, the most high level characteristics of a workload. And then we look at the consensus protocols, which these are the way that these replicas and shards coordinate. So we take a look at Paxos, Raft, and this is the most common consensus protocols. And then we have some reference architecture for Cloud Native Disaster Recovery deployments with true flavors, strong consistency, and adventure consistency. Next slide. Here is an example of what you can find in the white paper. This is a slide about the anatomy of Cloud Native of stateful workloads, distributed stateful workloads. And as you can see, we have usually three layers, an API layer that exposes the capability, like Messaging or SQL. And then we have a coordination layer, which is used to coordinate replicas and shards. And then we have a storage layer, which is actually interacting with volumes and persisting the data. And when you have multiple of these instances, because we have both replicas and shards, you need protocols and coordination protocols between these instances. And this is where interreplicate state synchronization come into play or interpretation coordination come into play. So we go in the details of what the options are there. Next slide. And we also have a little analysis of the most common products, distributed stateful workloads products that are out there. These are all open source. And what they use for the consensus protocol between replicas and what they use for the shard consensus protocol. And you can see some of them don't have shards, but when you have shards, you need, if there is an operation that a transaction or an operation that involves multiple shards, there needs to be a way to coordinate those shards. Next slide. There is an example of the reference architectures that we have. This is a very high level way of deploying a stateful workload in Kubernetes across multiple data centers, or if you will, across multiple geographies. So you see there are multiple Kubernetes cluster, there's the stateful workload with persistent volumes. Of course, you can have multiple replicas. As you can see from the yellow arrows, bi-directional arrows, this stateful workload, they need to communicate with each other. So you need to have an East-West strategy, communication strategy, maybe it's a tunnel, maybe it's some other networking capability, but they need to be able to talk to each other. And then you probably have a front-end application and some way of ingressing traffic from the outside, from the clients of this application. And you need a global advisor that is smart enough such that when, for example, maybe if you lose data center number one, traffic will not be directed there and all the traffic will be redirected to the data center to entry. Here we have three data centers because of the CAP theorem, we need at least three instances to have a successful leader election, which is what all of this strongly consistent stateful workload need in order to function correctly. On the right side, we discussed also the example of what happens if you have a network partitioning and what's the behavior in the case of a strongly consistent stateful workload, the partition that has the minority and cannot elect a leader will take itself offline. And so the global advisor also in this case needs to be able to detect that situation and just send the traffic to the data centers that are still able to work. That's slide. And back to you, Alex. Thank you, Raffaele. So on this last slide, we'll just reinforce our drive to continue to grow our community and to welcome you to join our community. We'd love to see more of you join the storage tag meetings and help contribute, whether you're on the project side or whether you just have a technical interest in storage. But also joining our calls is a good way of finding out about other projects in the ecosystem and understanding more about blockstores, file systems, databases, key value stores and a huge variety of other systems. Finally, if you want to help contribute to the tag itself, you can consider a role with the tag either as a tech lead or perhaps as a co-chair in the future. I would love to hear more from you if you'd like to contribute. So we hope to see you soon and thank you for joining our presentation.