 Hi everyone and welcome to this session, I'm Xiaohuang, I'm the VP of Ecosystem at the Cloud Native Computing Foundation and today we're going to reveal the results of the November's edition for the CNCF end user tech radar, which I'm really excited by. So let's take a look. Basically what I do at CNCF is I work with end users to get them engaged and active in the ecosystem so that they can successfully adopt technologies like Kubernetes and Prometheus. You can find me at Oishelow on Twitter and these are some of the companies that I work with. So this ranges from startups to large enterprises, all fantastic companies that are using Cloud Native and I'm very honoured to be joined by three of them here today to talk about the CNCF end user technology radar. So I would like them to each introduce themselves. So first of all we have Jackie Fong, please introduce herself. Hi everyone, I'm Jackie Fong. I work at Master, took a master for the last three years as their engineering manager leading their Kubernetes platform team and CI CD team observability and most recently developer experience and we developed a platform for our developers to consume and recently we migrated our self-managed Kubernetes to EKS at the beginning of 2020 which shows help. I kick-started a service mesh end user group with CNCF as a co-chair so happy to be here. Great, thank you, Sméin. Hi, I'm Sméin. I'm a DevOps team leader at Dailymotion which is this video distribution platform basically pretty famous in France and Europe. At Dailymotion I'm leading a team which is in charge of the Cloud Native platform. This platform is a hybrid platform mostly on premises and we have some stuff on the cloud and we are responsible of the whole developer experience from the laptop, the CI CD, to how they ship their application to the production and also the organizer of the CNCF Paris Meetup and the CNCF Ambassador and I'm glad to be here too. Fantastic. Maya, you're next. Hello everyone, my name is Maya. My pronouns are she and her. I'm a former principal engineer who worked on indeed compute storage and observability platforms. I recently left and joined a small startup called FX. I'm happy to be here. Awesome, I want to say thank you to all three of you for coming together to be the radar team and be the editors for this edition of the TechRadar. Just a reminder, the TechRadar is a way of assessing upcoming solutions and different technologies. The idea is to pick which tools you would recommend to place in adopt. In other words, they're mature and they're stable. They've been used by many different teams, many different use cases. Or you could put it in trial, which is to say that there's some success but may not be applicable in all cases or assess, meaning it's up and coming or maybe there are other solutions for different use cases. So the topic of this radar is database storage and database storage is very close to my heart because I used to work for a storage vendor. So I want to ask you to the radar team, why did you choose database storage for this radar? I would start by saying that I was wondering how was the adoption in the CNCF and user community in this particular area because the CNCF community is pretty mature on the Kubernetes platform and maintaining and the evolution of the Kubernetes platform. But I wanted to know how databases fit in this cloud native domain. I know on the Indeed side of the world, as we started to shift more and more of our workloads to the cloud, we were really interested in how people were running their databases, whether they were choosing to go manage services or running in cluster and what kind of some of the technology decisions that went into that for as well as how they were handling some of their scale out. Yeah, I'm pretty similar. I would love to pick the community spring. We've been migrating out on-prem presence to the cloud on the database front. We have legacy systems running on Orgo and just looking forward to get some insights from the community and some experiences to migrate out of either Orgo or onto Orgo on the cloud. So super excited for this topic. I'm definitely excited to see the results for this. So just a reminder for how the technology radar is put together. We created a Google spreadsheet with a list of different tools. This was not exhaustive, so people can add their own extra tools in as well. And then we asked each company to add a column for themselves and choose whether, within their own company, they recommend that these tools are adults, trial, assess, or hold. So hold explicitly here doesn't mean we don't use it or we don't have it at all. Hold means we might have it, but we don't recommend that new applications use this technology. And then here we've got some of the companies that have contributed to the results of this and probably a sort of slight bias towards the medium and larger size companies overall. All right, here we go. Ready for the results? Actually, before we do that, let's talk about what did you expect? So what did you think was going to come out from this? Gosh, where did it even start? I kind of expected a lot of people to continue using the technologies that they were familiar with just because you already have a lot of expertise on it from the engineering side. You don't have to rewrite applications. Good compatibility there. Yeah, I expect Oracle, like I mentioned earlier, to be up there as well. And for most of the companies that has entered information in there, most of them said on hold or recommending it to be on hold or faced out. So that was surprising, but otherwise most of those are probably in line with what we've been using. And I was expecting much more experiments on new cloud native databases. This is something that we will see later on, but I wanted to see because we worked on the POCs of the emotion to identify our next native database. And I was expecting that many more companies will do the same experience. Okay, so that's quite interesting. We've got a wide range from we expected Oracle DB to we expect super new technologies. Okay, so let's actually look at the results. Let's just remind again here. Green is votes for adopt. The blue is for trial. Yellow for cess. And then gray is for hold. And then we took probably the top 15 or so answers that we had. Oracle, I believe, was not, didn't end up in here, right, Jackie? No, it was way down there from a responses perspective as well. Yeah, Maya, you were going to say something? I don't think I was going to say much, but I think it was, like I said, to my expectations earlier, I kind of expected people to stick with the technologies that they were comfortable and familiar with. And I felt like the results kind of showed that, right? Definitely a lot of credit. Yeah, a lot of well-known names up at the top. So when we came to choosing which ones ended up in adopt, trial and assess, I think these are just kind of bucketed like this. I know there was a little bit of discussion about some of these items. Were there, are there any here that you use or you thought were, you know, personally you would put in a different place? At Denim Machine, we are heavy users of MySQL. This is our main database. And yes, I can see that most of the adopt technologies, we are using them except using PostgreSQL because we chose MySQL. We use MongoDB pretty heavily at Indeed. And that was one of the things I thought was really interesting to fall into the trial bucket. But there was a lot of people that kind of like marked it as like holds, don't quite adopt yet. And so it was kind of an interesting experience to see that. I remember Jackie, when we were doing this, you thought that PostgreSQL, you were surprised that a lot of people were using PostgreSQL, right? Yeah, I mean, way more than MySQL as you can see here, we've been, we migrated off, not to take a master in my previous place, I believe 20 years ago, we migrated off of PostgreSQL onto MySQL. And since then, I've been working with MySQL almost every place I work for. But recently at Ticketmaster, PostgreSQL came back. I mean, at least from a vendor-supported service they provided. And PostgreSQL was the back end. So to me, that was a little bit surprising, especially looking at the results here that PostgreSQL is more widely adopted than MySQL. Can you go into a little bit more detail, like why did you migrate off PostgreSQL in that previous world? And then why did you do that? That was a long time ago. Our main database, I work for ISP. That was when we had startups and DSL. I'm aging myself here. And basically it was our main registration application. And it was really heavy at the time people were signing up left and right. And supporting PostgreSQL was just too heavy for us. And we're looking for an open source database that we can migrate to. It wasn't an easy effort. When I left, they were still finalizing the migration. And MySQL at that time worked for us. It was super light. And the operation cost for MySQL was so much lower than PostgreSQL. And finding expertise at that time, we called them DBAs. Finding expertise for MySQL at that time was so much easier than PostgreSQL DBAs. So that's when we made the decision. And on the application side of the world, PostgreSQL didn't handle updates as well as MySQL. And so when people were doing updates and it needed to update any indexes, there was a cascading right problem which had performance considerations. Yeah. In fact, I think we talk in a little bit about how difficult it is to move between databases. So I think actually we should go into that. Yeah. So I asked you to come up with some thoughts about what themes or patterns did you see? Not just from the data, but from your own experience. So, Maya, why don't you talk about this first one? Yeah. So one of the big things that I had noticed with a lot of our cloud migration is a lot of we're really cautious with our data. And we tried moving from database A to database B. And there's obviously a lot of overhead involved when you have to move like petabytes or pairabytes worth of data from one data storage technology to another. I remember running math to try to figure out what kind of like write workload capacity we needed to stay on top of our writes for the day. And there was just a lot there to take into consideration. And we ended up like not choosing to adopt any new technologies, but even in our move to the cloud, we're still kind of considering like keeping the same things that we're familiar with. We have a lot of operational excellence around things like MySQL, as well as all of our applications are programmed against that. So it's, you know, the cost of switching from one technology to the other often has a high tax to pay. Right. That's actually also what you mentioned in terms of the hiring, right? Jackie, the finding people with skills. And a lot of engineers, when we interviewed them, they're looking for, you know, challenges, new, the next new shiny toy to play with. And, you know, when they hear about Oracle or when they hear about running on-prem, they shy away. And a lot of times companies make decisions, you know, based on some of those responses and newer technology, it's sort of attraction, right, to work for a company and whatnot. Like, Maya, going to a startup, it's like, you have to choose. You don't want to choose legacy systems and technologies. And that's why the newer technology should be there. I mean, it shows on our radar too. So. So, you said you're in the middle of looking at a change right now, a daily motion, right? Can you talk a little bit about the challenges that you've seen there? You're on mute. Sorry. Yes. Still on mute, I'm afraid. And I know why. Okay. So, okay. So, I was saying that at Daily Motion, and as I said, I'm in charge of the DevOps team. And my team is responsible of giving the main building blocks, I would say, for a developer to build his application architecture. And one of the pieces we want to provide is to provide an easy way to boost traffic database system, database software. And we are looking at the alternative in the market. And we did POCs last year in order to choose the one that fit our needs. And we are pretty happy because some of our internal tools are currently running in production on that solution. Do you mind sharing what that one was? Yes, we are working with TIDB, TIDB, TIDKV, and we deployed it on projection for internal uses. How has it been working with it out of curiosity? How is it working today? Working with TIDB, coming from, I guess, your previous standpoint. Yes. We did that shows because we did a lot of performance tests and reliability tests. We did chaos testing using the tool named the Chaos Mesh. We pushed a lot the solution to be sure that it handles our traffic, big traffic, because we have the target to have a lot of traffic in this platform. And we made everything to be provision ready. So we have backups, regular backups, scheduled backups. The TIDV operator is pretty amazing because it allows you to define really easily how you want to declare your database. And all the observability features are really amazing in this solution. I was pretty surprised by the maturity of this solution because it's not known, really known. But when we did our tests and compared to the other solution, this is the one we shot. Okay. I think let's go to the second theme out of three. Choosing a managed database service depends heavily on your use case. Sméin, do you want to talk about this one? Yes. Actually, when we've seen the results, we were pretty surprised by the fact that managed services were not too much adopt. We were expecting that there will be much more adoptions with the managed database services. But actually, when we talked about that, it depends on the company use case and where the application is deployed, what is the cloud provider. And sometimes the database just runs on premises. So that's why maybe that can explain why we don't see much more adoption in this area. Yeah. I mean, one of our use cases, our database is the biggest one is on-prem. And one of our initiatives for our developers is to migrate their workloads to the cloud. In the East region, it's great. The cloud AEC is pretty close to our on-prem data center there. But on the West side, it's too far. And we're constantly hitting some latency issues. And that's where our use case is kind of a little bit different between our East and our West regions. And so that's why we're still looking into what are the alternatives as we move off of on-prem from Oracle to anything out there that may not be just one-to-one migration. We are looking at multiple solutions. I know that when we were diving in, we looked at things like elastic search for an example and comparing our on-premise configuration versus the cloud configuration that we could get from something like AWS and did some pretty heavy performance testing against it and found that we could run elastic search not only cheaper than what AWS was offering, but also with higher performance. And so we did a lot of things like that when we were diving in. I think when we were doing that evaluation, we were on easy two instances and not necessarily running in Kubernetes. But it was something that was like we kind of again kind of take to heart when we look at some of this stuff and really try to take our use cases in this consideration. We also had like another ephemeral testing type of effort in place where people wanted to be able to spin up kind of like feature branches for ServiceX and then have a database that's kind of fresh and brought up and then torn down when that feature branch is deployed and using like a managed service for something like that can be really heavy and really difficult to get built into that workflow. And that's where we started looking at more things like running it in cluster alongside the service that you're trying to deploy. Yeah, I mean some of our developers even run their own MySQL within Kubernetes instead of using RDS. It's kind of find that interesting even though they are running on EKS. So different use cases really depends. That's actually a really good point because RDS was one of the options that was in here. It was one of the answers that was there but it didn't end up in the final radar. Why is that? I mean it's vague right when I said RDS which database are we talking about MySQL, Aurora right or even all goes there as well. So we took that out and because I mean some of the users they answer they adopt RDS but within RDS there are five or six that they can choose from. So we took that out. It's kind of vague from that sense and it's very AWS specific and we try not to like similar to Oracle. Well not similar to Oracle but we try to take the vendors out and be more what's the word non-vendor specific let's just say. Okay great. I knew that was a question that people are going to ask like where is RDS. So figure let's get it out of the way. Let's go to our third theme. This was just keep an open mind and Jackie what does this mean? Yeah and the reason why this subject was you know while filling out the radar I get a chance to talk to our database team and they're new to Kubernetes and we've been trying to work with them to migrate their services onto the cloud and most of the time onto Kubernetes as well and some of those engineers are Oracle DBAs and we try to talk to them about migrating venture into new technology and don't settle for the first one they look at and reach out to industry like experts like people here right. Maya, Smi and the community and see what their experiences are and learn from that before we you know decide on our own so keep an open mind on what's out there and this radar will really open up some of our eyes. That reminds me of a point Maya you were making earlier you said that cost and performance were some of the things that you were looking at so were there other things or maybe you can go into performance a little bit like what do you remember what the things were that you were looking at? One of the big things I think in kind of this space of like keeping an open mind right like a lot of pieces of technology are coming out that are better suited for certain use cases and kind of keeping that in mind so we have a pretty high right throughput when it comes to things like processing candidate applications or our job aggregation funnel and so we needed to make sure that whatever database technology we chose could handle the right throughput so with the way like CockroachDB works for instance it has a single node that can accept rights at any time and so we found like that wasn't scaling well for a lot of our right heavy use cases and so we needed something that was a little bit more federated to some extent. Even things like I said earlier was like protocol compatibility where it's like a lot of systems are written against my SQL or some piece of technology and going through and trying to like redo a lot of that code to then work with something like Postgrass or another kind of protocol it's a lot of work on our engineers and we don't necessarily want to put that kind of toil on them and that was one of the other things that we had found with like the test was it didn't handle some of the prepared statements that we were using quite as well and it kind of again forced our engineers to kind of go and reconsider like how their code was being written so we were trying to find something that was more you know drop in and replace versus we needed burden our engineers with going and fixing that. Spain maybe you can talk a little bit about this as well because you were saying you you're using tidyb so you did choose a new technology and I would say that apart performances and reliability there are other aspects to take into consideration I would say all the life cycle tools around the software solution the database software solution I mean how do you do backups how do you dig further when you have an issue observability is really important and how you can get the more information from your system these are some considerations to take into account and the adoption of the software the open source most of them are open source and we need to take into consideration how the community is is maintaining and how is growing and what is the adoption of the open source projector so you mentioned you're you're you're using tidyb now right yeah so the community community backing was part of the decision making for you yes we had a look to the to the activity and the repository to the number of contributors the the number of stars obviously and and the the way we can ask for support to the way we can talk with the community and this is something we took into consideration when we we tried the solution Jackie you're Maya do you have thoughts I think one of the things I had for I don't know this section was it was really surprising also to not see graph databases picked up a little bit more there's a lot of cases where like a knowledge graph is really useful and the fact that like I think we had Neo4j show up in there but it was like the only one that people had and it fell really low and there was maybe only a handful of companies that had been voted for it but it was again like one of those more interesting data points kind of to that point or theme of keep an open mind right like find the database that fits your use cases no I mean Maya and Smin you know nailed it we as we migrate from on-prem and supporting legacy systems we need basically keep an open mind what's on what's out there venture into the unknown right a lot of times my team the engines I have every sprint we they I see them picking the tasks that they're comfortable with I keep basically telling them you know jump out of your comfort zone venture into something out there and oftentimes try to contribute to the open source community as well when we bring in technology from there so yeah I of course I appreciate that given that I work at an open source foundation and so let's just kind of look at the whole thing kind of in total so what we ended up with in adopt it's my SQL memcached elastic search postgres Kafka and Redis and then we ended up with another sort of six or seven here and then in assess just a few so actually like do you have any thoughts for people who are who are looking at this and thinking you know if I have to choose a solution today you know this doesn't mean like okay you have to choose something from this section but do you have any words of advice for people reach out to your network or join this community here and pick the brains a lot of brainpower a lot of open source contributors here trying on new things and learn from each other because I learned a lot just talking to you guys here and and so don't don't be afraid to reach out for help and that's the biggest piece that I can say here Spain Maya trying to think of how to follow that that's really well captured okay well let me go to the just overall force how did you find creating the radar as as the Jackie said where I was really I appreciated to talk with you to share our point of views this is it was really interesting thank you and I learned a lot of things I didn't expect to be honest and as Jackie said talking to each other is really important I thought the radar process sorry sorry no I was gonna say a lot gave me a chance to reach out to teams that I have not talked to for a while especially the you know and this time and most of my team work from home anyways so I reach out to a database team and pick their brain for a while to help me build this radar having been like so engrossed on our persistence platforms like the world I was pretty familiar with kind of the indeed side filling in that information wasn't too bad for me the things about the radar process that I thought were really interesting was it was kind of difficult at times to figure out like well should this go into adopt or should this go into trial I think Kafka is one of those really interesting ones where it's like a lot of people said adopt but there were still like a few other folks that were like hold on Kafka and it was kind of interesting that people were submitting hold and so there's kind of some context there I felt that was missing and having some of that context probably could have helped us make better determination about like whether or not something could go in the adopt versus trial category and that's kind of where we typically leveraged comments pretty heavily of like this is how our use case of this is I think I had put Cassandra in there as an example of like we did a trial deployment of Cassandra a long time ago and even though we wound up on a hold for Cassandra you know there's a lot that can go into that right how was it deployed was it deployed with like open configuration or were we recompiling it things internally given indeed comes from a non-prem model like we were most likely taking like whatever was out in open source and rebuilding it running on sent OS and trying to go like our own way versus kind of using the well-known configuration provided by the community awesome I mean I want to say thank you to all of you Maya, Smain and Jackie because I really enjoyed working with you and I'm very very interested and excited to see the results of this same here thank you to everyone here just a reminder so you can go on to radar.cncf.io and read about this in more detail and take a closer look at the projects and if you want to get involved then we want to find out what do you want to hear you can go to this linkcncf.io slash tech-radar that just redirects to a github issue where you can vote on things that you want to see for the next version of this radar if you want to come in and be part of the radar team like Jackie and Smain and Maya or you want to contribute your viewpoints then please come and join the end-user community we'd love to have you and if you have thoughts about how this could be better then please send your feedback to info atcncf.io or just to me and that is it I want to say thank you once again to all three of our radar team today for putting the work and the time into this so really appreciate it and really appreciate your time today thank you so much thank you Cheryl thank you thank you Cheryl