 Well, hello, everyone. Thank you for coming out. This is the, you know, gorilla tactics for building scalable e-commerce services with Apache Cassandra, Apache Pulsar, and Vector Search, of course. I actually have an image up there for LangChain at Datastay. Actually, I've been doing a lot of work with LangChain in terms of using it to abstract a lot of the things that are around working with LLMs and, you know, getting to generate embeddings. So, yeah, I'm gonna talk a little bit about that today, too. Yeah, so just a real quick poll of the room here. Is there anyone who's like kind of new to Cassandra? Oh, yeah, a couple. Okay, okay. All right, how about Vector Search? Anybody who is really new to that? Oh, okay, yeah, a lot of the room. Okay, all right, all right. Good, now I know what I can't skip. All right, thank you. Okay, real quick about me. I'm a developer advocate with Datastacks. I've done a few books, two on distributed databases, and I have one on Java 21 that's coming out hopefully by the end of the month. In terms of, you know, my experience with e-commerce, I spent a lot of time working at WW Granger. Actually, that was my first interaction with Cassandra back in 2012, and then I spent about six years as the engineering lead for the NoSQL team at Target, and that's where this whole talk kind of came from a workshop idea that we had of building on a lot of the use cases and things that I supported while I was at Target to kind of show the way for other folks trying to do the same thing. All right, so the tech stack that we're gonna work on today, or look at today, I should say, is using Spring Boot, Spring Data, for the controller layer and the data layer, and then a Node.js React simple layer on top using Tailwind CSS. I know Tailwind CSS gets a little bit of hate on Twitter or X or whatever they're calling it these days, but as a backend engineer, I kind of like Tailwind CSS, so I don't have to think about how things look and it just kind of does a lot of that for you. One thing in particular that we're gonna focus on or dig into a little more deeply here in terms of driving at product recommendations using vector search generative AI is this service layer in particular. Now to go ahead and integrate this with a large language model, there's a couple of extra steps here. One, because of the state of current AI tooling where they kind of focus on JavaScript and they focus on Python, the support for a lot of those things, a lot of those frameworks with Java just isn't there, so I had to actually branch out and do this one in Python just to get the data loaded for the vector search backend. And I do that by working with a model that's from the hugging face, hugging face libraries, use that to generate the embeddings, returns them and persists them in Cassandra, of course. And then of course, when we have our service layer, go ahead and run a query, so it'll take like a product and grab that product's vector and then run an ANN approximate nearest neighbor based on that vector. That's gonna actually do a vector search in Cassandra, return the results and then we display it. Simple, simple. We'll actually get more into that and see what that looks like on the CQL side, as well as a little bit of the code as well. All right, so the reason that we're still talking about, building good data models for e-commerce in 2023 is because nobody wants to see this, where you go to a site and you're waiting a little bit and then maybe, uh-oh, ah, there we go, helps you if you turn it on. Then maybe that shows up, then you wait a little more and maybe that and that and then it's like, you know, Internet in 1996, it's, you know, everything else kinda, oh, kinda comes in. So, we wanna make sure that nobody goes through this, which is why we still talk about building good data models and making sure that things run well. Of course, the magic behind all this is Apache Cassandra. It's been around for quite a while, engineered for high levels of read and write performance. It's linearly scalable, you know, so that means if you, once you have a, maybe you establish that you can get 10,000 reads a second at a six node cluster, well, if you need to reach 20,000 reads a second, then you know you need to double that cluster to go ahead and match. That's linearly scalable, highly available, so it can withstand, you know, minor outages, which is really good, platform agnostic too. And of course, my favorite feature of Cassandra has always been the geographic distribution and awareness, you know, being able to have like an East Coast data center, a West Coast data center, a central Europe data center, and, you know, right once and it'll replicate everywhere. I've always thought that was pretty cool. And then of course, you know, we also have DataSex AstroDB, Cassandra Database has a service, deploys all three major clouds, and our free tier is actually really generous. You get about 80 gigs of storage, so I highly encourage you, if you haven't, to go ahead and check out Astro. It's pretty slick, pretty lightweight, pretty easy. And you don't have to worry about running Cassandra on your own if that's, you know, if being a DBA and an infra person isn't your thing. All right, so Spring Data Cassandra, I did mention is the data layer for this project. Over the years, I've had a bit of a love-hate relationship with Spring Data Cassandra. The love part is that it really does make things easy. Configuration's easy with the application YAML or application properties, you know, if you prefer the properties way. And it reduces just a lot of boilerplate code in terms of, you know, connecting and getting useful CRUD operations by default. I mean, there's just so much you don't have to build just to basically use your database when you use Spring Data Cassandra. Now the hate part of it, of course, comes from using things that don't run well in Cassandra and still having those available. Oh, and I made the slide, I couldn't decide on, I was down to like two return of the Jedi memes and in the end I decided to use them both because it is a trap and if you're running in Kubernetes many containers will die to bring you this information if you're, you know, doing a find all or count or something like that. But basically Spring Data, especially with Cassandra, you should be leery of any method that says all or count, just gonna throw that out there. All right, so Apache Pulsar. Lorena, here you go, this is for you. Hey, this gives you, you know, public subscribe, message queue, event streaming, kind of all in one. The cool thing about Pulsar is that it's geographically aware like Cassandra is. So it kind of makes it a natural fit to use with Cassandra, also highly scalable. You know, I didn't work on the Kafka team at Target but I can tell you that I do know we had all kinds of issues in terms of going multi-data center with Kafka and a lot of those headaches Pulsar just kind of takes care of, so highly recommended if you haven't checked that out. All right, and of course Vector Search. Vector Search is new with Cassandra as of 5.0. Essentially, you know, if you haven't used it a lot it works on data that's been transformed or tokenized into embeddings and it allows you to do approximate matches or searches on that data inside of, you know, like Vector Space and you can do that with a few different algorithms. So, you know, you have things that are like cosine based. Dot product is a little bit faster version of cosine if all your vectors are normalized properly and then of course there's Euclidean in terms of getting Euclidean distance between vectors as well. One thing I found out the hard way while trying to help someone on Stack Overflow is that if you have a vector that is all zeros in Vector Space, Euclidean is the only one you can use because all the cosine based ones will throw a divide by zero error, which, you know, well, nobody wants. So, there's a small little tip for you. As far as like Vector Search in general, you know, kind of how does it work? Where, you know, you think once we have certain strings, in this case, the names of products. So, this is an example for like a pet supply website, you know, where we have like Love My Dog, Ring Chew Toy, and Love My Dog, Tennis Ball Toy. And we can do, and this example is using the cosine based index, we can do an approximate nearest neighbor based on one of those vectors and it'll have an understanding of how close it is approximately to the other. So, that's kind of sort of how it works. I always like to show new people like this slide because wrapping your head around it in two dimensional space is a little easier than trying to wrap your head around it in 1536 dimensional space. So, you know, that's always a good start. All right, so when we're talking e-commerce, you know, it's kind of a general term. E-commerce subsystems, however, there's a lot that goes into building, you know, a good e-commerce site. There are things you need, there are things that are kind of cool and there are things that are nice to haves. The things you absolutely need are, you know, like a shopping cart, you need a product system, a way to manage your users so they can log in, a way to process orders. And then I have the recommendation system up there. The recommendation system, you know, it's not necessary, but it's the perfect, I think, perfect use case of kind of demonstrating how we can use vector search in something like e-commerce. So, all right, so, you know, a lot of this in terms of use cases is gonna be pretty self-explanatory. You know, like our product catalog is, hey, we can kind of navigate through different categories, we can go to our product page, we can get pricing and sometimes pricing, you know, will be a little different, depending on if it's a web price or if it's a price in, you know, San Francisco versus a price in, like, Sheboygan, Wisconsin, you know, they're gonna be different in terms of just kind of how things are priced and the expenses that go along with that. Shopping cart, of course, you're gonna need one. The idea is that in this case, we have data on user carts. And I say that plural because when you allow users to have multiple carts, that's kind of the first step in having like a wish list kind of a functionality. And then, of course, you know, you wanna be able to track products inside of carts as well. And then finally, recommendations. The recommendation service that I've kind of built out here that I'll show you is that, you know, when you're on the product page and you kind of pick your size, in this case, it's an extra-large track jacket. And you say add to cart. That's when it kicks off the product recommendation service and says, hey, you might also like this Datastax Black hoodie. Yeah, by the way, Lorena, I took all these pictures myself. If you look closely, you can see there's dog here on, well, the hoodie specifically, but. All right. And then, of course, user profile. You wanna wait for users to log in. In this case, I've built in Google single sign-on. So, yeah, so that's an easy way that you can kind of go in and have just say, hey, click, use Google. You can also do like GitHub. And I think Facebook as well, although that might be falling a little out of popularity, but yeah, definitely can be done. The actual data model, or ER model if you will, kind of looks like this, where when we wanna view our subsystems, this is what we're looking at from how we build our data model, how we're gonna support things out, where we have our users who have things like addresses and shopping carts, and those carts can contain products, and products can recommend each other, and orders also contain products. So, yeah, it's a little bit of a complex relationship. I see people snapping pictures, so I'll wait. Hey, all right. So, of course, if you haven't done a lot of Cassandra data modeling, the thing you need to think about the most is, which queries are you going to support? So, you kind of have to have this little discussion with your team about, well, for our shopping cart, what does it need to do? In this case, well, user needs to be able to view all the products in their cart, and they also need to be able to add and remove products from the cart as well. So, all of that kind of figures into how you're gonna build your table, and more specifically, how you're gonna build your primary key. In this case, this is how, for this project, I should say, this is how we solve the cart query, where we have just a UUID as the partition key. We have a product timestamp next, and the idea with that is, this way, we'll see the products in the cart in order as they're added, and it'll maintain that order. And then, if by some strange coincidence, you get two products added at the exact same time, which you shouldn't, but just in case, we have product ID there as a tie breaker, just to kind of make sure we're separating those out. And I also have a TTL on here. This one was a bit of a lesson learned, but when you don't have a TTL on your shopping cart, what happens is they become avenues of attack by other malicious operators. The general theory that we had at Target at the time was that somebody was using an old cart to scrape prices. So suddenly, what we'd see is a big spike in write activity on one particular table in the cart model. And all of a sudden, this cart would have like 1,500 products added in just a few seconds. And of course, we know that that's not human activity. That's a bot that's gone through and has some of our API URIs and is able to inject some things there. So, highly recommend in this case a TTL on the carts. In this case, I have it set to 60 days. 60 days seems long enough that, if you have a shopping cart out there and you haven't looked at it in two months, I mean, you're probably not terribly interested in it, so it seems like a safe enough number, but that's gonna be something that will be more of a business decision than anything, like a hard and fast rule. All right, so product recommendations. In this case, we're building a special table just to kind of handle the vectors for the products. It's still just keyed by product ID. The main thing here is that our vector type is using a 384 dimensional float. Now, your dimensions that you'll use for your vector are gonna differ with each use case, but more specifically on whichever large language model you're using to generate your embeddings with. And then, of course, what really makes this work is the storage attach index that is also on the vector column as well. Getting into specifics about the model for this, we're using one of HuggingFaces models known as the all-mini-lm-l6-v2, excuse me. It's technically a sentence transformer, which basically means that it takes phrases and can tokenize those into an array of floats. It works with 384 dimensions. The reason that I chose this particular model is that we're just going on product name here, so it's pretty simple. There's not a lot of moving parts to that. But the thing is that the more dimensions you have, like if you're using open AIs at a O2 model, that's 1,536 dimensions. The more dimensions you have, the more accurate a lot of your results will be. But there's that trade-off there where your queries will also be a little bit longer as well. So 384, I felt like, was a nice balance of performance versus accuracy to kind of drive the best results there. And actually, we've gone through, or I've gone through in this, and wrote the interface with LangChain to do a lot of the heavy lifting. The code will be up there in a second, but LangChain, if you haven't heard of it, it's a framework for building applications with large language models. And I've said this before a couple of times, that it abstracts a lot of the more difficult aspects of working with LLMs away, so you don't have to worry about that so much. LangChain specifically works with, it's like API access out to hugging face and open AI. And I don't know if it works with cohere or not, but I was able to do a little bit with cohere for the Taylor Swift GPT model that just went out. So the Python loader, I apologize for the font size here, but I did try and get it as big as I could, but simply put, I just had a file of product IDs and up here is where I specified the model. And really, we just rip through the product IDs, read them out, make a call to Cassandra to get the product back by ID, and then we take the product name from that and we use this model.embed query, that's what generates our embedding right there. And with this generated and below, you can see we're doing our session execute and it's inserting those product vectors into the table. Oh, too fast. That produces data that kind of looks like this. So in this case, I have a T-shirt on the website that says go away or I'll replace you with a spring boot annotation. When I use the vector for one of those and run a approximate nearest neighbor, you can see that I'm getting all the different sizes of that shirt back. And then I get a couple other different ones on the bottom. That's important to remember for later because in this particular example, when I have a large or an extra large of a shirt and I get a recommended product that's the small of the same shirt I already added, a small's not gonna fit me. So it's not terribly useful. So the idea is that we kind of have to look at what we're getting for our next level of matching. As far as building out with spring data Cassandra, this is working out of the product vector repository. I was able to use, if you use the query annotation, you can go ahead and build out custom queries. As far as spring data Cassandra goes, it doesn't quite have a native vector search in yet. So to get this to work, you do have to do a custom query for now. So that's what I've done here. And like I said, I did a limit of eight because I wanna make sure that I'm getting more products than just the ones that are the different size of the same shirt. So we actually call this in this find products by vector here. This is the get promotion product or get recommendation product service. We go through and create a list of product vector pojos from our product vector entities. If you've used spring data enough, I'm sure you've done the entity to pojo conversion before. And then down here, we again, go through the list that we have and as long as, and I'm sorry, I should say, once we get to one that has a different product group, product group than the one that we searched on, we know we have a different product and then that's the one that we return. So that's kinda how the whole product recommendation service works with vector search on this. So hey, getting a Pulsar. I'm not gonna get too much in a Pulsar today, but I will talk about kinda how it's used in this project is, I did talk about how it's message ordering, guaranteed delivery, good scalability. The idea is that this is what's going to move our order once we place it around to the different business units. So when you place your order, you have all your products in your cart, that gets sent out to the pending orders topic. And the idea there is that somebody on a forklift or a robot in a warehouse comes back to, or gets a message sent to it with a list of products to go pick and it turns around and takes that list and goes down to different aisles of the warehouse, gets the products, brings them back and then that order goes from pending to picked. So it goes from like one topic to the other. Well, then it's in the shipping department. Shipping people have the picked order and they take it and packages up, put it in boxes, slap a label on it, send it on the conveyor belt and they can mark it as shipped. Simple enough like that. Eventually that package goes on a truck, maybe it goes on a plane somewhere along the way, then back to another truck and it ends up on your doorstep. The driver gets out and snaps a picture of it and that marks it as complete. So Pulsar is the way that we're kind of moving that those orders around so that we know where they are throughout the different business units. Again, just kind of a way to kind of keep track of where a particular order is once it's been sent through. All right, so I've talked for an awful lot. Now I can show you how this all works. Essentially, I'm gonna have this on a resource side later but this is the Git repo that has all of the code here. But basically we're gonna fire this up in a spring boot. I have six minutes, which should be plenty of time. I'm gonna sign in with single sign on with Google. I'm gonna navigate to a product, add it to my cart and that should trigger a product recommendation using Vector Search. So let's give this a shot here. Here we go. Yeah, I know, I'm one of those people who still uses Eclipse and you know what? I'm not apologizing for that. I know everyone likes VS Code but... Oh yeah, there you go. I was just gonna say I've been using Eclipse for like 16 years and I have just, yeah, it's just, yep, what I do. All right, so our spring boot project is running. So I can say HTTP, oh. Slash, slash, local host, 8080. Hey, there we go. All right, so we have our site. I can go to login and I'll select my Google account. There we go. I can check my profile if I did everything right. Oh, it knows who I am, outstanding. There we go, that's good. All right, so if I go back to the main area, come on. I can go to my Apache Cassandra 3.0 contributor t-shirt and I'll go into that. And I'm an extra large, so I'll say that. And once I add this to the cart, again, if everything's working properly, it should recommend another t-shirt, although I can't remember which one. So if I say add to cart, oh, there we go. That's the go away or I'll replace you with a spring boot annotation shirt. So that's, again, that's running vector search under the hood there. So there we go. Yeah, oh, and I can also look at the, oh, dismiss. Have a look at the shopping cart. Oh yeah, I tested earlier on the Datasax black hoodie, so of course you can see that my cart is persisting, which is excellent. All right. And if I make my way back to the slides here. All right, like I said, I'd have the GitHub repo up there along with some others. This Ecom product embedding loader, that's the Python script that I wrote that actually like kind of works with the hugging phase. What is it, all many L6 LLM? Here's a YouTube channel for Datasax developers. If you haven't been out there, we have all kinds of tutorials and instructional videos. There's a lot of great material out there, especially if you're new to like gen AI and vector search and you wanna learn how to get started on this. I actually, just not too long ago, put out a video on getting started with lane chain, so it really does walk you through a little bit of that as well. And then of course, the Cassandra.apache.org main Cassandra site, lots of great things out there, including all the documentation. Thank you, Lorena. I know you're right in front too, and I'm picking on you all day, yeah. And hey, otherwise, highly recommend, check out AstartUB. Like I said, we have a great free tier and unlike a lot of places, we don't require payment information, so if you just wanna use it, you absolutely can and try some things out. I highly recommend that. So, are there any questions? All right, well hey, thank you all very much. I know this is one of the last talks of the day, so thank you so much for coming out. Absolutely appreciate it.