 Okay, so we're on we're on slide six. I think the agenda speaks for itself Again, we have proposals. We have sponsor requests and we have a backlog Today we'll be hearing from TIKB But as somebody associated with poor sex, I feel no shame in requesting a second sponsor I believe that Ken has offered to be a sponsor for the project having spoken to the team and I don't know if Brian Cantrilla's on it was able to speak to them, but I think he may Be able to act on their behalf Otherwise, I'll have to tell the team to go find someone else. Hey Alexis. Yeah, I intend to do that That their availability is limited by time zone So in my availability is really bad in the morning right now. So we're trying to find a time to talk. Okay. Thank you, Brian I appreciate that cool and open metrics and Honour but we're okay. So that means that we can go straight into the TIKB presentation Do we have Ed and Kevin Available. Hey Alexis, this is Kevin. How are you? I'm very well. Thank you as I was saying on Twitter, you know Open source is a global phenomenon. I saw you talking about that. So indeed. We hopefully represent a part of that globe With this presentation Well, good luck. All right, so I guess I'll share my screen and launch this slide For everyone and go for it. Yeah, make sure everyone can see me share screen. All right, hopefully everyone can see this slide Yeah, looks great. Awesome. Awesome. Okay, great. So once again, my name is Kevin I am from the company pink cap. I'm their general manager here in North America and I along with Our co-founder and CTO Ed Huang will be presenting TIKB to everyone. Thank you again for the opportunity Thank you for not watching the World Cup and listening to us at this very moment Very excited to be presenting this open source distributed transactional key value store for everyone to consider for CNCF and quick Here, let me see. All right So a quick agenda for today's presentation I will go through a history and a community update for TIKB a Fairly detailed walkthrough of the technical and architectural aspect of TIKB a use case with Ulama, which is one of the largest food delivery platform in China right now Serving more than 260 million users and they're using TIKB right now to serve about 80 percent of their Improduction traffic if you have time I will also do a quick demo on my laptop to give you a little bit of a feel of how to spin up a TIKB cluster on Your laptop and when we have time happy to take any questions from everyone on the call So a quick history about pink cap It was founded in April of 2015 by three infrastructure engineers We're working in some of the largest internet companies in China like net ease and JD calm and was of course one of them And we set out to build the TIDB platform tie for your curiosity just stands for titanium There are several components to the TIDB platform One is TIDB itself, which is actually a stateless sequel layer that is my sequel compatible The focus of today's presentation is TIKB, which is a distributed transactional key value storage layer We also built something called ties spark recently That is a spark plug-in that also talks directly to TIKB to help a lot of our users Process more complex analytical queries last but not least We also have a project called placement driver, which is a cluster that does the metadata storage layer That communicates with TIKB to do scheduling Auto-balancing and also timestamp allocation TIKB the project was open source a little over two years ago on April 2016 Its current version is 2.0 its license is patchy 2.0 and here is the link for you to check out the repo In terms of the community progress so TIDB as a whole is actually one of probably the most popular active Open source database project out there. It has more than 13,500 stars TIKB itself has more than 3,300 stars with 70 plus contributors and roughly around 3,000 commits right now We also have the benefit of enjoying contribution from a lot of outside Institutional contributors from other companies like Samsung like MoBike, which is one of the largest bike sharing platforms out there as well as Folks like totiao.com, which is one of the largest tech companies in China with a super popular news Aggregator app the whole company is valued around 20 billion dollars right now as well as two public cloud vendors like Tencent cloud and UCloud and The big pain point that we wanted to address to build TIKB is to have a open source Distributed storage layer that can unify a lot of the desperate data that has been stored right now in multiple different kinds of database solutions But have a layer that supports strong consistency that really supports distributed transaction with asset compliance That can be easily scaled horizontally in either direction and of course has a cloud native architecture That was of course a lot of what drew people's interest with the Google Spanner project That is also where we got our original inspiration from for TIKB as well But unfortunately Spanner is an open source and isn't so Accessible and our vision for TIKB is to build a building block for other cool Amazing powerful systems to be built on top of it. So far we built TIDB and Tyspark ourselves Totiao has built their own metadata service on top of their S3 Implementation and the ULMA which I will go into in more detail actually build their own Redis proxy on top of TIKB So now I will do a dive into the technical architecture of TIKB as I mentioned TIKB currently lives within the whole TIDB platform among many a few different other components But the focus of today's presentation will be in this middle red part Where all the TIKB clusters are where all the data is actually stored and persisted and Communicates with the placement driver cluster Here is a layout of the TIKB Architecture so TIKB the component uses GRPC to communicate with the placement driver as well as any clients That can be built on top of it. It exposes two kinds of API one as a raw Key-value API one is a co-processor API that facilitates a push-down computation It uses the RAV consensus protocol to provide data replication and high availability and Underneath each TIKB instance, and you can essentially imagine each instance as one single machine We also have a RocksDB instance where we leverage that community For this as our storage engine for TIKB and Here are some of the technical highlights of TIKB as I mentioned it does scheduling and auto balancing We also have a multi raft implementation because each TIKB node has Several of actually Oftentimes many different raft groups that are replicated across different TIKB nodes So each TIKB node has multiple raft groups that it has to facilitate the communication between Different TIKB nodes. We also have a dynamic range-based partitioning Feature that allows these raft group to be split merged or the leaders can be automatically Transferred in order to remove and resolve hotspots the way we implement asset transaction is through a two-phase commit with optimistic lock and TIKB is written entirely in Rust which is a relatively new systems level language that is getting a lot of traction and adoption and The nice thing about Rust as many of you may know is that it does not have GC stop time or a lot of run-time cost In fact, I think TIKB is one of the largest Rust Improduction project out there aside from of course FireFox Here is one example where SQL can be realized on top of TIKB using TIDB, which is what we built Internally well with the community as well So the way it works with TIDB is that TIDB actually has several layers That we built ourselves with a MySQL compatible layer a parser a cost-based optimizer and a co-processor distributed executor API that talks directly to TIKB nodes and each of these little color blocks are Basically a raft groups that are in the same group and they are evenly distributed across multiple TIKB nodes and replicated for a high availability and The way we map a relational table to a key value pair is Essentially, we have a encoding system that maps the key and the value or actually the the IDs and the indexes of each row of data into key value storage pairs that can be essentially imagined or Visualized as a giant sorted map that are broken down again into multiple smaller chunks that are replicated using a raft and All the keys here are sorted according to their byte array order and we did that intentionally in order to support useful operations like scan and this is a pretty important difference Comparing TIKB to say some of the other other similar projects that you go by for example which I believe uses hash to generate these keys and therefore cannot support things like scan and Sorry about the mix-up with the graphic, but here is a visual representation of how the co-processor works So what essentially happens is that when TIDB receives a SQL query it will go through the parser to break down each of the query into different physical plans and partial and each of the The plan the partial aspect of the plan are actually pushed down into multiple TIKB nodes Simultaneously where all the computation is actually done inside TIKB nodes at the same time to compute partial results for a particular query these partial results are again returned back to TIDB and TIDB does the final reassembling of all the partial results that can be sent back to the client and This is an implementation that we worked on a lot to be able to take advantage of the distributed nature of TIKB and all the computing power that it has Access to in order to speed up more and more queries and in one of our future roadmap plans We actually plan to support more built-in functions that we can push down into TIKB nodes as well Here's another example of how TIKB is being used so far. I alluded to this a little earlier, which is how totiao com uses TIKB They have their own S3 implementation They have a bunch of S3 buckets with a lot of blob storage and they are also using TIKB as their meta data storage Right now for their in production mode And here is the latest benchmark the YCSB benchmark that we did just last month And here is the environment and the hardware that we use to do this benchmark And you can see the insert TPS results as well as the read QPS result here One thing to note is that this is a standard default three TIKB node deployment and You know, of course in a actual in production environment Most of our users deploy way more than three TIKB nodes to store more data to increase their capacity So, you know, this kind of this result will be much better And I think the throughput will be much higher in even in production environment But this is the the benchmark that we did last month for TIKB Oops Here is a quick overview comparison between TIKB and some of the other Popular no-SQL databases out there. Of course every single database Tries to solve different problems in different ways using different technology So not everything can be compared in a completely Apple to Apple sort of a way, but TIKB is original and still the current goal is to first and foremost support this stupid transaction that has strong consistency and that is Sort of the first goal and the first level priority that TIKB looks to support Which is different from some of the other no-SQL databases out there Here is a visual overview of one of the features that I mentioned before which is dynamic splitting and merging As many of you know, RAF groups and RAF regions can get quite big and as a RAF region gets big It could form a lot of hotspots and provide and produce performance issues And one thing that TIKB can do on its own is that if a region say here in region A Gets too big and by big currently we are defaulted to mean 96 megabytes Of course that can be configured depending on your usage But if it gets larger than 96 megabyte we will automatically split that region into two regions and put them in different TIKB nodes and The reverse is true as well if a region is too small and that is currently defined as 10 megabyte then we will look for the closest adjacent region and then merge those two together Into one larger region to you know improve the performance of that region And here is another visual illustration of a core feature which is automatic hotspot removal One of the best use cases for TIKB and TIDB as a whole is if your access mode Doesn't have hotspots or wants to avoid hotspots TIKB is a great solution for that and the way that's being done is if You know several regions where the leader here denoted in blue Is in one particular machine then all the workload is going to all the leaders while the followers are not You know doing a whole lot and if this is starting to form a hot spot Then the system will facilitate an automatic RAF leader transfer where we will do a logical leader transfer Here in region B to move the leader from the first machine to the second machine And there's no actual data movement here It's just a transfer of leader within the RAF consensus protocol Then we will have the workload split onto two different machines and thus the hotspot is removed And to go a little bit over our cloud native integration and progress like I mentioned We've always imagined and built TIKB to be having a cloud native architecture that works closely with Kubernetes To be integrated in all kinds of cloud deployment Scenarios currently TIKB is integrated with Tencent cloud and you cloud and most recently We are also got on JD comms cloud provider or cloud a solution And of course in the future we look to integrate with all the major cloud vendors you know all over the world and As far as cloud native synergy is concerned with other Components currently we have a Docker compose deployment for testing and development on local machine Which will be part of my demo We have a tool called tidy be operators that works closely with Kubernetes to help deploy tidy be in all different public or private cloud scenarios and we will actually soon be open sourcing this tidy be operator tool as Well, we of course use Prometheus and gRPC in our standard deployment we our team is actually one of the largest maintainer of the rust implementation for both Prometheus and gRPC and we also use a lot of Etcd and is a active contributor to Etcd Because we have really been leveraging Etcd since they won when we started building TIKB Because it had a very mature raft implementation and also very rigorous testing regimen that we really leveraged and We didn't fork it completely because we wrote TIKB and rust so we kind of have our own rust implementation of Etcd We are active contributors of the Etcd community We do a lot of bug fixes and we are also leading the charge in Forming features new features like the raft learner a quick overview of TIKB usage Currently, there's about 200 companies Give or take that are using TIKB in production right now. A lot of them are using TIKB in Combination with other components like tidy be anti-spark, but quite a few companies are using TIKB by itself and one of those companies that I want to talk about is Ollama, which like I mentioned is a food delivery platform with 260 million users So it's bigger than a lot of the perhaps more well-known food delivery platforms that we hear here in North America and Europe are combined it was recently acquired by Alibaba for 9.5 billion US dollars and The problem or the pain point that they were facing is they had a lot of data in key value formats And they were using a hodgepodge of different solutions like Mongo, Cassandra and Redis And they were looking for a solution that can really unify all these different data sources into one and They found the TIKB and deployed TIKB as this unifying storage layer Which currently is a serving and affecting about 80% of the entire platform's traffic TIKB is currently holding more than 25 terabytes of data spread across four different data centers for Ollama and What's really interesting is that Ollama built their own Redis a layer on top of TIKB because they wanted to continue using Redis A lot of the application developers love using Redis So that's what they did to make TIKB work for them And if you're interested in digging deeper into how they use TIKB We recently published a use case story written by Ollama's engineers that you can look at via this link All right, so I've done a lot of talking blah blah blah and If we have time I would do a quick demo of how to spin up TIDB cluster on your laptop and the context of this demo is to show you number one how easy it is to deploy TIDB right now I already downloaded or get cloned a TIDB cluster repo on my laptop So I'm deploying it right now using Docker compose and as you can see Prometheus is installed by default and So our three standard TIKB nodes right here and what I will do now is Spin up a mySQL cluster as well as a spark cluster So that you can see how TIKB can be the underneath storage layer to facilitate both components talking to each other and reading from the same data source But before I do that, I want to show you real quick Monitoring mechanisms so each of these Deployment has a Grafana implementation defaulted to port 3000 and if you log in using just admin admin again This is just for testing and development purposes. You can monitor your entire clusters You know metrics and current status if you go into TIDB cluster TIKB You can look at the store size the available size And things like that. So there's a bunch of stuff that you can play with inside the Grafana implementation And one more tool which we built in-house is something called TIDB vision and This is defaulted to port 80 10 and Here you have a cool little data visualization tool That as is a ring and each of the partial ring is basically one TIKB node If you look a little bit deeper in there, you see a bunch of empty blocks These are just empty store spaces the dark green are Raft leaders and the dark gray are raft followers And you can essentially visualize raft as it goes through the entire TIDB or TIKB deployment So this is how that works Now back to terminal the demo What I would do is launch a mysql Instance so in the interest of time, I would just do a lot of copy and pasting of commands So launch mysql and as you can see this is TIDB compatible with mysql and I will also launch a spark instance This will take a little while so I will let this run Let's go back to mysql and I'll show you what is in here. So show this So we have a few databases and we'll actually use this one called TPC H-01 for the demo. It just has a bunch of Sample data in there. So TPC H-01 And Let's see what's in this database. So it just has a bunch of different tables in here One of them is called nation orders things like that. So let's see what is in nation All right, just a list of countries with some random information in here and Right now we have our spark plug-in ready So I am going to input a couple of commands to Launch Thai spark which like I mentioned is a spark plug-in that works directly on top of Thai KV as well so these are the two standard Thai spark commands and Last one we will tie this instance to the same database called TPC H-01 So they should be talking to the same data source and Let's just see if that is the case We'll use SQL again spark SQL select star from nation show 30 We'll have the exact same table of country information as the one that we saw on my SQL site So let's make some edits to this table Since Belgium has such an epic World Cup match yesterday where it advanced to the next round We should probably add Belgium to this list of countries. So let's insert Belgium into this table and If you see that we have Belgium on the bottom right here a new new member of this country list and If we do the same command you will immediately see the change being made and Visible on the Thai spark side as well so you can easily imagine where multiple updates and changes are being made on the my SQL side and the Thai spark side can immediately do queries and analytical processing on the spark side all being supported and stored inside Thai KV All right, so now I'm gonna go back to my slides and Of course the goal of our presentation is that we will love to have CNC if CF accept Thai KV as either an incubation or a sandbox level project and With this acceptance and to be part of CNCF We're looking to build really not just a bigger community But also something that is more vendor neutral that can help us build this project With better governance with better structure and ultimately more contribution to build a very useful and important components That's really is you know beyond the strength of the current community right now. We will love to see more Language support right now. We only have a go client for Thai DB and a Java client for Thai spark One of our community members has already started building it open source Redis proxy He caught it Titus so you can check out his repo here But of course it's still very much a working progress We wanted to support column family structure as well So there's a lot of things that we will love tidy Thai KV to have and with CNCF support. I'm sure we'll be able to accomplish that so again Thank you for your time we'll love for you to be our TOC sponsor and of course reach out to me and Ed anytime if you have any questions And we will actually preparing for the technical proposals right now and we will share that with everyone Hopefully within the next week. So that's about my presentation. I had one quick question Kevin on The PR it has sandbox slash Incubation which one was it going incubation or sandbox. So that's just depends on the sponsor. I guess or yeah What do you feel like is blocking you from one or the other? I guess I mean, I think I'm sort of leaving this up to the TOC to give us what you think will be the best I guess level of entry for the current status of the project You know given the number of adoption that we've seen so far for Thai KV I think it probably would work for incubation But then again, you know, I'm not too familiar with the different kind of criteria and what goes into these considerations So we're being you know open and receptive to your opinions about which level is most appropriate Hey, Kevin. This is Brian suit One question. Have you done a Jepsen run on tidy V or take a V? I just going around it looks like you've done at least some experimentation That doesn't look like Kyle's at the opportunity to do a full run Yes, we've done our own Jepsen test, but we haven't done it with Kyle going through his process specifically And is yeah, you might have already right. Is that something that you that you're looking to do? That's definitely something that we're looking to do and I guess we just haven't quite got around to it It's definitely a process that you know requires some resources from our side as well But we would definitely love to you know go through that process as well You know since our own process probably gets us somewhere along that way But I think having him do it with us is probably you know, we're definitely open to doing that Yeah, I mean I would really encourage you to do that I know it's it's time-consuming and it's expensive both the terms of resources and potentially monetarily But I do think it's really worth doing because it Jepsen has I mean as you know Jepsen has has become the gold standard for actually Did allowing people to understand the true consistency guarantees are of these various projects But this is honestly very impressive and it looks like you've got a lot of production use So I think it's I mean really exciting project on that. I'm actually a bit embarrassed that I had not heard of Our fault, but we're working on fixing that right now Well, and I think like getting a Jepsen run is a good example of something that would get you more visibility in terms of how it's In terms of what it can go to because this is certainly for us and for a lot of people This is a really interesting. I mean, we've got all the same problems that you're seeing and the that the GC pauses are just a Deal breaker for me for any So the and I'm obviously I may be not in this forum but aside very curious with your experiences with Ross we're having a lot of really good experiences with rust and It's looking like a very interesting trajectory. So I'd love to get your take on that as well But I would be happy to help that to help facilitate you however I can in terms of Sponsoring would have you that would be awesome. Thank you so much, Brian And yeah, consider it done when it comes to Jepsen has would definitely put that higher on our radar with you That's great Really impressive presentation. Thank you. Thank you any other questions that we can help you with in a moment So how tightly coupled are? Ty KV and Ty DV So you're proposing to donate just Ty KV. That's correct That's correct It's as far as being tightly couples concern. So Ty DB itself is just a stateless sequel layer Right and KV is the layer where all the data is stored and persisted and we've always designed it that way So the two part can be Separated and then used in different ways. So right now, of course given that Ty KV hasn't been So much on his own the current usage very much is connected to Ty DB and you know Ty spark depending on what the user is for but it can be easily adapted like what the Olima folks are doing with the Redis proxy to be Used as a building block for really whatever you see fit right on top of it. So what you see is a in ecosystem of Kind of domain specific or use case specific storage layers on top of K Uh, KIDV. Yeah, like the the Redis and sequel and potentially others in the future. Absolutely. Yeah Hey Chris, my apologies. I got a drop No worries. Appreciate you Brian. I should make the last question. Can you repeat that please mr. Brian? Yeah, I was asking about the Whether they see usage as Having an ecosystem of Domain specific storage layers or adapters built on top like Redis sequel And so on as opposed to direct use of the KV store by applications I guess maybe we might see both but it looks like Right now it's more layers on top like spark and in sequel and so on Um, well one can debate about whether spark is a layer or an application itself, but uh Okay, I had a question about the the region Uh Breakup as well. Yeah, when splitting a region that then forces transactions to go to A two-phase commit is that all happen? Is that correct? Um in terms of splitting the region So the one I was talking about was uh more for removing Um kind of a large region for the purpose of performance issue So what things I guess would be put in the same in together in the same region? Um In terms of so this that would just be data that's coming in Or kind of encoded from the relational side To kv store and then they're broken up into different region mostly by their size So kind of like in this situation where you can see Um different tables go into the key value pairs and then broken down into different regions And then these regions, I guess if it gets too big could be split up Just for you know performance purposes or hot spot forming are removable Okay Yeah, I'm sorry. Maybe I should I can follow up with you. Um in terms of specifically Are you talking about the transaction kind of getting disrupted by a region split? Uh, I was just trying to understand what some of the um Automation was doing in terms of how regions were structured and what's in the same raft group versus different raft groups Right And what the some of the performance trade-offs were with two phase commit versus the same raft group Gotcha, gotcha. Yeah. I mean in terms of region splitting I think the main consideration is the size of the region But also the amount of traffic that this region assuming that this region is a leader replica For that particular raft group, uh, then it could get split up into different Machines to remove hot spot. That's that would be like one scenario. I think where that will get automatically facilitated Okay, thanks. I have two questions. Yeah One is I think pretty easy, which is could you just clarify for everybody Exactly what the relationship is with that cd in the past present and future Just to sort of be understand and and two is just for Just for q&a really interested in, you know, if people Have issues with uh, take a view. What is the number one thing they complain about? I see, gotcha. So First question first so for etcd, right? Um, so like I mentioned, uh, we Have been leveraging etcd since day one because of this uh raft implementation And also, you know, we really leverage its testing Rigger to use it to test our own ty kv system We actually use etcd embedded in our placement driver Implementation to the placement driver cluster directly, but for ty kv We because we use wrap we use rust to uh, code it We didn't fork etcd completely but kind of made our own rust implementation of etcd In ty kv. So that's kind of the past and the current and for the future We are very involved in the different kind of etcd Kind of road map that's going forward One of the things that I mentioned is the raft learner feature that we are really looking to implement For the next evolution of ty kv Because one of the I guess drawbacks and this actually goes into your second question is because ty kv is you know by nature a key value store it doesn't quite support Complex analytical queries and the speed and the performance that say a h base potentially would or any other column family database would And that is sort of the inherent Structural limitation that ty kv has in that implementation So there is a limit for example to how fast our type spark implementation could really go If it sits on top of what ty kv is now and you can see that as being one of the Not so much complaints per se, but at least Consideration or limitations that people use ty kv for when it comes to analytical processing But with the raft learner feature being more mature and being implemented We can see that as a really good solution to support Faster analytical queries to be able to process on top of ty kv Awesome presentation. Thank you very much. Thank you Yeah, once again, please reach out to us if you have any other follow-ups And we will circulate our proposal very soon I Kevin just a quick question Which percentage of the maintainers are outside of pink cap right now? Is it is just primarily pink cap driven or Do you have maintainers from from other companies? Right now. I think in terms of maintainers is probably mostly from pink cap In terms of contributors You know our own ty kv team isn't any it's probably like less than 15 people And we are like 70 contributors. So okay, cool. Thanks Just just one request if you go ahead and do the incubation route Probably start sounding out potential Interview ease in your production user base because I think the more we hear from them The better it is for streamlining the dd process Got it. We'll do Okay, I actually got a jump Chris. Could you Shepard the rest of the cold please. I'm really sorry. Yeah. No, we're pretty much Closing out. So any other questions for for kevin before he disappears. All right Thanks. Thank you Cool. Thank you everyone So, uh, not too many updates. Um, you know, if just go to slide 37 as a pointer to the working groups 30 it's the project review 39 it's just a reminder that Uh, we have a few events upcoming for this year at least we have shanghai in seattle if you Are interested in submitting a talk to shanghai Um, the cfp closes at the end of this week. I think it's on the 7th. So please get your, um, toxin You have a little bit more time Uh for seattle Other than that slide 40 our next meeting. Um, we'll be hearing from Uh, the falco project from siss dig on july 17th And that's about it any other questions before we Close it chris it closes at midna on the 6th So this is just the the the last big shout out. Please consider submitting a talk Uh, or multiple talks for kibkan shanghai, which is november 14th and 15th This week is your last time to do it. You can write it out as you're looking at the fireworks tomorrow If you're in the u.s. And um, please encourage the folks in your organization to submit as well Thanks Yeah, cool. Yeah, i linked the cfp in the chat any other questions from folks All right Thanks all and we'll see you in a couple weeks Take care You too. Thanks