 Hi everyone and welcome to this session, I'm Xiaohuang I'm the VP of ecosystem at the cloud native computing foundation and today we're going to reveal the results of the November's edition for the CNCF end user tech radar, which I'm really excited by so let's take a look. Basically, what I do at CNCF is I work with end users to get them engaged and active in the ecosystem, so that they can successfully adopt technologies like Kubernetes and Prometheus. You can find me at oyster on Twitter. And these are some of the companies that I work with. So this ranges from startups to large enterprises and all fantastic companies that are using cloud native. And I'm very honored to be joined by three of them here today to talk about the CNCF end user technology radar. So I would like them to each introduce themselves. So first of all, we have Jackie Fong. Please introduce yourself. Hi everyone. I'm Jackie Fong. I work at master take a master for the last three years as their engineering manager, leading their Kubernetes platform team and CI CD team observability and most recently developer experience. And we develop the platform for our developers to consume and recently we migrated our self managed Kubernetes to EKS. At the beginning of 2020, which shows help I kickstarted a service mesh end user group with CNCF as a culture. So happy to be here. Great. Thank you, Smeen. Hi, I'm Smeen. I'm DevOps team leader at Dailymotion, which is this video distribution platform basically pretty famous in France and Europe. At Dailymotion I'm leading a team which is in charge of the cloud native platform. This platform is a hybrid platform, mostly on premises, and we have some stuff on the cloud. And we are responsible of the whole developer experience from the laptop, the CI CD to how they ship their application to the production. And also the organizer of the CNCF Paris meetup, and the CNCF. And I'm glad to be here. Fantastic. Maya, you're next. Thanks, Maya. My pronouns are she and her. I'm a former principal engineer who worked on indeed compute storage and observability platforms. I recently left and joined a small startup called FX. I'm happy to be here. Awesome. I want to say thank you to all three of you for coming together to be the radar team and be the editors for this edition of the tech radar. And just a reminder, the tech radar is a way of assessing upcoming solutions and different technologies. So the idea is to pick which tools you would recommend to place in adopt. In other words, they're mature and they're stable. They've been used by many different teams, many different use cases. So the topic of this radar is database storage and database storage is very close to my heart because I used to work for a storage vendor. So I want to ask you to the radar team, why did you choose database storage? Maybe I would start by saying that I was wondering how was the adoption in this NCF and user community in this particular area, because this NCF is a very, very, very long-term, so I want to ask you to the radar team. Why did you choose database storage for this radar? Maybe I would start by saying that I was wondering how was the adoption in this NCF and user community in this particular area because this NCF community is pretty mature on the Kubernetes platform and maintaining and the evolution of the Kubernetes platform. But I wanted to know how databases fit in this cloud native domain. I know on the Indeed side of the world, as we started to shift more and more of our workloads to the cloud, we were really interested in how people were running their databases, whether they were choosing to go manage services or running in cluster and what kind of some of the technology decisions that went into that for as well as how they were handling some of their scale out. Yeah, I'm pretty similar. I would love to pick the community spring. We've been migrating out on-prem presence to the cloud. On the database front, we have legacy systems running on Oracle and just looking forward to get some insight from the community and some experiences to migrate out of either Oracle or onto Oracle on the cloud. So super excited for this topic. I'm definitely excited to see the results for this. Just a reminder for how the technology radar is put together. We created a Google spreadsheet with a list of different tools. This was not exhaustive so people can add their own extra tools in as well. And then we asked each company to add a column for themselves and choose whether within their own company, they recommend that these tools are adults, trial, assess or hold. So hold explicitly here doesn't mean we don't use it or we don't have it at all. Hold means we might have it but we don't recommend that new applications use this technology. And then here we've got some of the companies that have contributed to the results of this and probably a sort of slight bias towards the medium and larger size companies overall. All right, here we go. Ready for the results? Actually, before we do that, let's talk about what did you expect? So what did you think was going to come out from this? I kind of expected a lot of people to like continue using the technologies that they were familiar with. Just because you already have a lot of expertise on it from the engineering side, you don't have to rewrite applications, good compatibility there. Yeah, I expect Oracle, like I mentioned earlier, to be up there as well. And for most of the companies that has entering information in there, most of them said on hold or recommending it to be on hold or face down. So that was surprising. But otherwise, most of those are pretty in line with what we've been using. And I was expecting much more experiments on new cloud native databases. This is something that we will see later on, but I wanted to see because we worked on the POCs at the emotion to identify our next database cloud native database. And I was expecting that many more company will do the same experience. Okay, so that's quite interesting. We've got a wide range from, you know, we expected Oracle DB to we expect super new technologies. Okay, so let's actually look at the results. Just a reminder, again, here green is votes for adopt. The blue is for trial, yellow for cess, and then gray is for hold. And then we took probably the top 15 or so answers that we had. Oracle, I believe was not, didn't end up in here right Jackie. So it's, yeah, it was way down there from a responses perspective as well. Yeah, Maya you were going to say something. Oh, I don't think I was going to say much but I think it was, like I said, to my expectations earlier I kind of expected people to stick with the technologies that they were comfortable and familiar with and I felt like the results kind of showed that right. Yeah, a lot of well known names up at the top. And so when we came to choosing which ones ended up in adopts trial and assess. I think these are just kind of bucketed like this. I know there was a little bit of discussion about some of these items where they are there any here that you use, or you thought were, you know, personally you would put in a different place. So we are heavy users of my sequence. This is our main database. And yes, I can see that most of the adept technologies we are using them, except using was good, was good because we chose my sequence. We use for MongoDB pretty heavily add indeed. And that was one of the things I thought was really interesting to fall into the trial. But there is a lot of people that kind of like marked it as like holds don't quite adopt yet. And so it was kind of an interesting experience to see that. I remember Jackie when we were doing this, you thought that Postgres, you were surprised that a lot of people are using Postgres, right. I mean, way more than my SQL as you can see here, we've been, we migrated up, not a ticket master my previous place, I believe, 20 years ago, we migrated up a Postgres onto my SQL. And since then I've been working with my SQL almost every place I work for. But recently at a ticket master, Postgres came back. I mean, at least from a vendor supported service they provided and Postgres SQL was the back end. So to me that was a little bit surprising, especially looking at the results here that Postgres is more widely adopted than my SQL. I want to go into a little bit more detail like why, why did you migrate off Postgres in that previous world and then that was a long time ago. Our main database, I work for ISP. That was when we have dialed ups and DSL I'm aging myself here. And basically it was our main registration application. And it was really heavy at the time people are signing up left and right and supporting Postgres SQL was just too heavy for us. And we're looking for an open source database that we can migrate to. It was, it wasn't an easy effort. When I left, they were still finalizing the migration and my SQL at that time works for us. It was super light. And the operation cost for my SQL is so much lower than Postgres SQL and finding expertise at that time. We call them, you know, DBAs, right, finding expertise for my SQL at that time was so much easier than Postgres DBAs. So that's when we made the decision. And on the application side of the world's Postgres didn't handle updates as well as my SQL and so when people were doing updates and it needed to kind of update any indexes. So that was the cascading right problem which had performance considerations. Yeah, in fact, I think we talk in a little bit about how the different, how difficult it is to move between databases. So I think actually we should go into that. Yeah, so I asked you to sort of come up with some thoughts about what themes or patterns that you see, not just from the data but from your own experience. And why don't you talk about this first one? Yeah, so one of the big things that I had noticed with a lot of our cloud migration is a lot of, we're really cautious with our data and we tried moving from database A to database B and there's obviously a lot of overhead involved and you have to move like from petabytes or perabytes worth of data from one data storage technology to another. I remember running math to try to figure out what kind of like right workload capacity we needed to stay on top of our rights for the day and there was just a lot there to take into consideration and we ended up like not choosing to adopt any new things. Even in our move to the cloud, we're still kind of considering like keeping the same things that we're familiar with. We have a lot of operational excellence around things like my sequel, as well as all our applications are programmed against that so it's, you know, the cost of switching from one technology to the other often has a high, high tax to pay. But that's actually also what you mentioned in terms of the hiring, right Jackie the finding people with skills. And a lot of engineers when we interviewed them they're looking for, you know, challenges, new, the next new shiny toy to play with. And, you know, when they hear about Oracle or when they hear about running on prem, they shy away and a lot of times company make decisions, you know, based on some of those responses and newer technology it's sort of attraction right to work for a company and whatnot like my going to a startup it's you have to choose. You don't want to choose legacy systems and technologies and, and that's why the newer technologies should be there. I mean it shows on our radar too so you said you're in the middle of looking at a change right now a daily motion right and can you talk a little bit about the challenges that you've seen there. You're on mute. Yes. Still on mute I'm afraid. Okay, so, okay, so I was, I was saying that at the emotion and as I said I'm in charge of the DevOps team and my team is responsible of giving the main building blocks I would say for a developer to build his application architecture. And one of the pieces we want to provide is to provide an easy way to, to bestrap database system database software. And we are looking at the cloud native alternatives in the, in the market, and we did POC is last year, in order to choose the one that fit our needs. And we are pretty happy because some of our internal tools are currently running in production on that solution. Do you mind like sharing what that one was. Yes, we are working with IDB type TV and we, we deployed it on production for internal uses. How is it been working with that out of curiosity. How is it. How is it working today working with the IDB coming from like the, I guess, like your previous standpoint. Yes, we did that shows because because we did a lot of performance tests these tests and reliability tests with it cows testing using the tool named the cows mesh. We, we pushed a lot the solution to be sure that it handles our traffic, big traffic because we, we, we have the, we have the target to, to have a lot of traffic in this platform. And, and we made everything to be a production ready so we have backups, regular backup scheduled backups. The IDB operator is pretty amazing because it allows you to define really easily how you want to declare your, your, your database. And, and all the observability features are really amazing in this, in this solution. I was pretty surprised by the maturity of this solution because it's not known, really known. But when we did our tests and compared to the other solution. This is the one we shot. Okay, I think let's go to the second theme out of three and choosing a managed database service depends heavily on your use case. Smeen, do you want to talk about this one? Yes, actually, when we, when we, we've seen the results, we were pretty surprised by, by the, the, the fact that managed services were not so too much adults. We were expecting that there will be much more options with the, the managed database services. But actually, when we talked about that depends, it depends on the, the company use case and when where the application is deployed, what is the cloud provider, and sometimes the database just run on premises. So that's why maybe that can explain why we don't see much more adoption in this area. Yeah, I mean, one of our use cases, our database is the biggest one is on-prem and one of our initiative for our developers is migrate their workloads to the cloud. In the East region, it's great. The cloud, AZ is pretty close to our on-prem data center there, but on the West side, it's, it's, it's more, you know, it's too far and we constantly hitting some latency issues. And that's where our use case is kind of a little bit different between our East and our West regions. And so that's where we're starting looking into what are the alternative as we move off of on-prem from Oracle to, to anything out there that's, you know, may not be just one-to-one migration. We are looking at multiple solutions. I know that when we were diving in, we looked at things like elastic search for an example and comparing like our on-premise configuration versus the cloud configuration that we could get from something like AWS and did some pretty heavy performance testing against it and found that we could run elastic search not only cheaper than what like AWS was offering, but also like with higher performance. And so we did a lot of things like that when we were diving in. I think when we were doing that evaluation, we were on EC2 instances and not necessarily running in Kubernetes, but it was something that was like we kind of, again, kind of take to heart when we look at some of this stuff and really try to take our use cases in this consideration. We also had like another ephemeral testing type of effort in place where people wanted to be able to spin up kind of like feature branches for, you know, ServiceX and then have a database that's kind of fresh and brought up and then torn down when that feature branch is deployed and using like a managed service for something like that can be really heavy and really difficult to kind of get built into that workflow. And that's where we started looking at more things like running it in cluster alongside the service that you're trying to deploy. Yeah, I mean some of our developers even run their own MySQL within Kubernetes instead of using RDS. Kind of find that interesting, even though they are running on EKS. So different use cases really depends. That's actually a really good point because RDS was one of the options that was in here. It was one of the answers that was there, but it didn't end up in the final radar. Why is that? Well, I mean, it's vague right when we when I said RDS, which database are we talking about MySQL, Aurora, right, or even all goes there as well. So we took that out and because you, I mean, some of the users, you know, they answer, they adopt RDS, but within RDS there are, you know, five or six that they can choose from. So we took that out, it's kind of vague from that sense and it's very, you know, AWS specific. And we try not to, you know, like similar to Oracle, I tried, well, not similar to Oracle, but we try to take the vendors out and be more, what's the word, non-vendor specific, let's just say. Okay, great. I knew that was a question that people are going to ask, like, where is RDS? So figure let's get it out of the way. Let's go to our third theme. This was just keep an open mind. And Jackie, what does this mean? Yeah, and no reason why this subject was, you know, while filling out the radar, I get a chance to talk to our database team. And they're new to Kubernetes. And we've been trying to work with them to migrate their services on to the cloud and most of the time on to Kubernetes as well. And some of those engineers are Oracle DBAs. And we try to talk to them about migrating adventure into new technology and don't settle for the first one they looked at and reach out to industry like experts like people here, right? Maya, Smi and the community and see what their experiences are and learn from that before we, you know, decide on our own. So keep an open mind on what's out there. And this radar will really open up some of our eyes. That reminds me of a point Maya, you were making earlier, you said that cost and performance were some of the things that you were looking at. So, were there other things or maybe you can go into performance a little bit like what do you remember what the things were that you were looking at? One of the big things I think in kind of this space of like keeping an open mind, right, like a lot of pieces of technology are coming out that are better suited for certain use cases and kind of keeping that in mind. So we have a pretty high right, high right throughput when it comes to things like processing candidate applications or our job aggregation funnel. And so we needed to make sure that whatever database technology we choose could handle the right throughput. With the way like Cockroach DB works, for instance, it has a single node that can accept writes at any time. And so we found like that wasn't scaling well for a lot of our right heavy use cases. And so we needed something that was a little bit more federated to some extent. And so one of the things like I said earlier was like protocol compatibility where it's like a lot of systems are written against my sequel or some piece of technology and going through and trying to like redo a lot of that code to then work with something like post grass or another kind of protocol. It's a lot of work on our engineers and we don't necessarily want to put that kind of toil on them. So that was one of the other things that we had found that like the test was it didn't handle some of the prepared statements that we were using quite as well and it kind of again forced our engineers to kind of go and reconsider like how their code was being written. So we were trying to find something that was more, you know, drop in and replace versus we need to burden our engineers with going and fixing that. I mean, maybe you can talk a little bit about this as well because you were saying you you're using tidy be so you did choose a new technology. I would say that apart performances and reliability. There are other aspects aspects to take into consideration. I would say, all the life cycle tools around the software solution, the database software solution. I mean, how do you do backups? How do you dig further when you have an issue observability is really important and how you can get the more information from your system. These are some consideration to take into account. Yes, the adoption of the software, the open source, most of them are open source and we need to take into consideration how the community is, is maintaining how he's growing and what is the adoption of the open source project. So you mentioned you're you're you're using tidy be now, right? Yeah, so the community community backing was part of the decision making for you. Yes. We had to look to the to the activity in the repository to the number of contributors, the number of stars, obviously, and the way we can ask for support to the way we can talk with the community. And this is something we took into consideration when we tried the solution. Jackie, you're Maya, do you have thoughts? I think one of the things I had for, I don't know, this section was, it was really surprising also to not see graph databases picked up a little bit more. There's a lot of cases where like a knowledge graph is really useful. And the fact that like, I think we had Neo4j show up in there, but it was like the only one that people had, and it fell really low and there was maybe only a handful of companies that have been voted for it. But it was, again, like one of those more interesting data points, kind of to that point or theme of keep an open mind, right, like find the database that fits your use cases. I mean, Maya and Smin nailed it. As we migrate from on-prem and supporting legacy systems, we need basically keep an open mind on what's out there, venture into the unknown, right? A lot of times my team, the engineers I have, every sprint, I see them picking the tasks that they're comfortable with. And I keep basically telling them, you know, jump out of your comfort zone, venture into something out there and oftentimes try to contribute to the open source community as well when we bring in technology from there. Yeah, I, of course, I appreciate that given that I work at an open source foundation. So let's just kind of look at the whole thing kind of in total. So what we ended up with in ADOPT, it's MySQL, Memcached, Elasticsearch, Postgres, Kafka and Redis. And then we ended up with another sort of six or seven here and then in Assess, just a few. So actually, like, do you have any thoughts for people who are looking at this and thinking, you know, if I have to choose a solution today, you know, this doesn't mean like, okay, you have to choose something from this section. But do you have any words of advice for people? Reach out to your network or join this community here and pick the brains. A lot of brainpower, a lot of open source contributors here trying on new things and learn from each other. I learned a lot just talking to you guys here. And so don't be afraid to reach out for help. That's the biggest piece that I can say here. Spain, Maya. Trying to think of how to follow that that's really well captured. Okay, well, let me go to the just over force. How did you find creating the radar? As Jackie said, I was really appreciated to talk with you to share our point of views. This was really interesting. Thank you. And I learned a lot of things I didn't expect, to be honest. And as Jackie said, talking to each other is really important. I thought the radar process. Sorry, sorry. No, I was going to say a lot, give me a chance to reach out to teams that I have not talked to for a while, especially the, you know, and this time. And most of my team work from home anyways. So I reached out to the database team and pick their brain for a while to help me build this radar. Having been like so engrossed on our persistence platforms that world I was pretty familiar with kind of the indeed side. Filling in that information wasn't too bad for me. The things about the radar process that I thought were really interesting was it was kind of difficult at times to figure out like, well, should this go into adopt or should this go into trial. I think Kafka is one of those really interesting ones where it's like a lot of people said adopt, but there are still like a few other folks that were like, hold on Kafka and it was kind of interesting that people were submitting hold and there was some context there I felt that was missing and having some of that context probably could have helped us make better determination about like whether or not something could go in the adopt versus trial category. And that's kind of where we typically leveraged comments pretty heavily of like this is how our use case of this is. I think I had put Cassandra in there as an example of like we did a trial deployment of Cassandra a long time ago and even though we wound up on a hold for Cassandra. You know there's a lot that can go into that right how was it deployed was it deployed with like open configuration or were we recompiling it things internally. So given indeed comes from a non-prem model, like we were most likely taking like whatever was out in open source and rebuilding it running on sent OS and trying to go like our own way versus kind of using the well known configuration provided by the community. Awesome. I mean, I want to say thank you to all of you. My ass main and Jackie because I really enjoyed working with you and I'm very very interested and excited to see the results of this. Same here. Thank you to everyone here. Just a reminder so you can go on to radar dot CNCF.io and read about this in more detail and take a closer look at the projects. And if you want to get involved, then we want to find out what do you want to hear. You can go to this link CNCF.io slash tech dash radar. Just redirects to a GitHub GitHub issue where you can vote on things that you want to see for the next version of this radar. If you want to come in and be part of the radar team like Jackie in Spain and Maya or you want to contribute your viewpoints then please come and join the end user community we'd love to have you. And if you have thoughts about how this could be better, then please send your feedback to info at CNCF.io or just to me. And that is it. I want to say thank you once again to all three of our radar team today for putting the work and the time into this so really appreciate it and really appreciate your time today thank you so much. Thanks Cheryl. Thank you. Thank you show. Let me try and stop this record. Okay.