 Hello everyone, thank you for coming here. I'm Stefano from Datastax. Today, I'm going to show you a library which was built with the purpose of making Cassandra more easily accessible to generative AI folks, which as you will see, does not always coincide with Cassandra people. So we know that Cassandra 5 is coming, packed with very good new stuff, AC transaction, SAI, vector search, vector search. So vector search, in case you don't know it, is a big thing, which is basically the workhorse of this generative AI revolution, which is taking place. So looking at the timeline, you notice that CEP 30, which is the Cassandra enhancement proposal, introducing vector support, happened during the course of this year and was very quickly made its way to Cassandra 5, which is about to be released. And you probably know that vector search is a big deal and it, so you know vector search probably, but just as a quick reminder, it is a way that it is a search method that enables retrieval of basically anything as long as you can turn it into a vector. In particular, it's very useful for retrieving pieces of text because there are now embedding methods to very cleverly make text into points in a space and then vector search basically translates the hard task of semantic similarity search into a geometric task. So you can have a sentence and you can look for similar sentences and that has become a problem of geometric problem, which is great because we know how to treat that. So developers, especially generative AI folks, there might be an overlap with people familiar with Cassandra, but you probably can imagine this overlap right now is not that big. And since Cassandra 5 is so great and it starts to pack support for all of this gen AI oriented thing, I think we need a way to enhance this overlap, especially because in this picture, Cassandra and gen AI people are not drawn to scale and they are not drawn to scale even in this picture. So generative AI practitioners is a notion of people and if we find a way to make Cassandra more accessible then it's a big win, right? So there is a concept which probably DevRel people know very well, which is the time to hello world. And today this has to be very short. I'm speaking minutes. Many times if, let's say someone lands on your website on your quick start and in five minutes they do not see anything working yet, that might be enough to make them flee to someone else, some other vector search database and we don't want that, right? So raise your hand if you feel familiar with CQL. Of course, we are at the Cassandra Summit. Can you imagine how would CQL feel to someone used to a two line of Python code? You can admit it might feel intimidating a bit. Verbose, I mean, I love CQL, don't get me wrong. But we need a way to make at least the time to hello world much shorter, much easier. And this is where Casayo enters the picture. We want to make using Cassandra easy and quick for folks who don't have time to learn the ways of CQL. But still they want to exploit the various, very awesome technologies that are behind Cassandra and its vector search support. So let me show you a quick start. I'm sure you have seen plenty of rather quick starts and this is no exception. Actually, this is two quick starts in one. So let me jump to, okay. I have a notebook, by the way, sorry. You will see there is a link here. You can run it yourself either with Docker, Cassandra 5, beta image or with Datasux AstroDB. If you don't want to get involved with the local running your database, you will find all links in here and in the presentation online. So this is a very standard rug quick start. I'm not here to show you how rug works, but so I have already executed the setup steps which are starting a Docker, Cassandra, checking the contact point, okay, standard stuff. So you see here, this is Python code, of course. This is where I create a session. This is all done. I create a key space, so this is already. Now, I also set open AI key for later usage. And now, this is how the Casio part works. There's one line where I set up a session for later use. We can skip the AstroDB case because today I'm running with Docker. You can see it here, Docker, Cassandra 5.0, Alpha 2. So I can run a SQL just in case it wouldn't be needed here. So this is where I create. Well, a vector store, but it's a table. It's a table directly made the right shape to serve your vector needs, directly made with your indices, ready to be queried, and this is everything it takes. A bit easier, a bit more Pythonic than SQL. So behind the scenes, a lot of things have happened. Table creation, indices, all that. Then we will populate the store with a few vectors. This is a sample dataset with 30-something fake insect descriptions. Let's look at one of them. So there is a name, an order, and a description. Like this is some kind of made up butterfly. So you see, this is how I put my entry in the vector store. There is a put method that takes care of everything. It will take care of prepare statements in search from everything, schema, translation, pre-post-processing, everything. The only thing I have to do is I have to make, to create an embedding vector. So this is not a library, this is not a long chain. This is not a library that makes JennyI things for you. This just makes it easier to use Cassandra as if it were exactly the shape of your JennyI needs. So, okay, let me run this one insertion. This is it. So we can maybe just for the fun. We can try desk tables. There is a Casio demo. Let me, so you see, this is my row. I'm not showing the vector because it's very long, but this is what ends up in the database among other columns. Let me go back to the demo. I want to complete the insertion, so I will compute the remaining embeddings and still with this Casio abstraction run the insertion concurrently to save some time. So that's it. Now, how do you do search? Again, you don't have to deal with the SQL full syntax for vector search, which I'm going to show you later. You see, again, this cell is what, where the vector search is run, but the vector search is this line. So it's exactly a method built in Casio exactly with the shape that you need. So you have a vector, a number of entries, and possibly a metric to measure similarities, and there we go. So I started with a sentence describing some kind of dragonfly. I make it into a vector because that's where, that's what I need for running the search, and there we go with the results, or assorted by similarity score. I can do something more. I can run some metadata filters because the appropriate indexes have been built for me by the Casio table creation. So we are only about, this is only about bugs, and there we go, colioptera, colioptera, colioptera. Again, results. Okay, so the last part of this demo is the full rug thing. So it's not only retrieval, R. We also want to do AG, so full rug. And in this case, I don't, I think we have seen so many of this chat with your docs things. Okay, so this is a bit different just for the fun of it. This is a field assistant bot. Imagine you are a amateur entomologist. You spot some insect in the fields, and we use rug to retrieve entries from this fake textbook about biology, and we help the entomologists with identification of the species. So we have a prompt, which says, you are an expert entomologist assistant, so and so, and then we will stuff this prompt with the observation by the user and the candidate found by the retrieval, just for the fun of it. So I have a function here, suggest observed species, and you see it's basically doing a vector search, then to find the relevant matches according to the observation, then it formats the prompt template with observation and matches, and then it will ask an LLM to come up with a final identification and degree of accuracy. So this is how it works. I just call suggest observed species with some description. It will, of course, here you see it takes some time, but that's the LLM interaction that takes some time, because the retrieval is very fast. And you see, after a bit of text, there is a, the why and the how this observation matches this insect with a moderate degree of uncertainty. So this is basically it. Of course, again, I can run it again with a different remark, and I can also run in debug mode and you will see how the prompt is created. That's, of course, not specific to Casayo, but it might be fun to see. So this is the actual prompt that is then sent to the LLM, and the answer is, again, based on your observation because of this and that, this is the insect yourself. So that's it. There's nothing specific to Cassandra here in the rug part, but that was to give you the sense of how you get all the way to a working application. By way of comparison, I have here, I happen to have here a similar notebook with the same application done with pure CQL. And it's nothing extremely verbose, but still you remember the one line to create the vector store, that's it here. So it starts here, if you do CQL, and it goes all the way here. I'm sure everyone in this room has no trouble doing that, but again, try to put the head on of someone new to Cassandra. This is a big difference. This fosters adoption, you know? And besides, it already packs the best practices in a sense, it will do prepare statements, it will do optimization, it will take care of all of the right schema, and let's say it makes it more difficult to shoot you in the foot. And the same for when you run the search. So again, this is where you insert, again some CQL that might scare people away. This is where I do the search and so on. So this is somehow, so Casayo has a mission of trying to get, trying to hide away what is avoidable at least in a first interaction with Cassandra while keeping the goodies. So Casayo is more than just a vector store abstraction. It's actually a family of tables, which from a GNI perspective look like vector store, chat memory, LLM cache, so very specific and whose purpose is specific to individual typical needs of GNI application. But actually what Casayo does is a more abstract kind of thing. So Casayo is offering abstractions for a table with a vector, a table with metadata support, searchable metadata support, a table which is just a key value store and so on. And then you can use them for GNI purposes or anything actually. So there's nothing specific to language, there's nothing specific to GNI actually in Casayo. Indeed, the principle are just to show no CQL unless it's necessary and use the best practices in Cassandra but trying to avoid showing them to the user unless it's necessary again. Because GNI people don't have time to, they don't like boilerplate like many. They don't have time to learn a new language like CQL if they can avoid that. And they might like to see their GNI language being used in the API. But still, we don't want to compromise the best features of Cassandra so we still expose partitioning, we expose TTL, we use prepare statements and whatnot. Okay, so how long do I have? I might spend a bit, yeah, okay. I should be able to spend some time and a bit of dive into how Casayo is structured. So there are basically three big modules. One is, I'm starting from the ones on the right. So there is a logic to easily read from several tables at once doing client-side join to fill a prompt typically. So it's a very typical task in a prompt-based LLM application that you read from, let's say you read from, you want to try to convert a set of primary keys into the actual values with stuff in the prompt from user ID to names or name and date of birth, something like that. So this is a one line to do that. And then there is a config module to easily configure your session before you start using Casayo without having to specify it anymore. But the biggest part is what you see on the left here and that's a tree of classes built on a base table class which act like a mixing. So you can mix and match your classes and create the abstraction for your table that suits your needs, no more, no less. So you want metadata or not, you want clustering, you want the vector support. You can just mix these mixings and you will get a table that works. And this is based on a tower of methods that might optionally do something and defer to the higher method in the hierarchy. So let's say the vector table that you have seen in the demo is this metadata vector Cassandra table on the upper right. And that's a base table with vector on top, with metadata on top. So every time you run something like an ANN query, you see this second line in the yellow block. There is the ANN search in the vector mixing which asks the metadata layer to handle some of the input arguments to make them into the where close. And then the thing goes back to the vector support which runs the query. And then again, so every data input and output goes through all the layers which potentially might have something to say in it or not. So this is how you can get full composability of these abstractions. So where is Casayo used right now? You probably have heard of LungChain and LamaIndex. These are the two most used frameworks to write applications with LLMs. Their support for Cassandra in both cases is built on top of Casayo. And even though users would not see Casayo directly, the rationale behind is that there are so many patterns that are best captured once in Casayo. And then the integrations for LungChain and LamaIndex are just thin wrappers specific to the idioms of LungChain and LamaIndex. We just defer for the bulk of managing the table to Casayo. So which opens the way to easy integrations with more frameworks. Not only that, but there is even a legitimate usage of Casayo itself because not everything is language. I keep forgetting that, but you might have needs for vector search even outside of language, right? So sound similarity, image similarity. You just go straight to Casayo and you have your ready-made methods to use like you saw in the demo. Again, easier than doing all the CQL. So the project is evolving. We hope to have zero to out very soon. And yeah, you can see here the features that are being added at various stages now. I invite you to go to casayo.org for quick starts, docs, tutorials and everything. And of course it's an open source project and we welcome contributions and issues and everything. I close by reminding you that if you prefer to work with a cloud database built on Cassandra, we have AstraDB of course. And Casayo has a first class support for that as well through a very easy authentication method. So you are invited to try it again. And I think I can stop here and I thank you for your attention. Thank you. Any question? Sorry, there is one there. Yes. His talk, sorry. Yeah, Casayo is Python specific. That was built out of the need of offering something to Python practitioners. There might be, of course, the same could can be built in different languages but as things are now that would be an undertaking from scratch. But see Jeff's talk later today at five, right? 450, there is a very enticing way to cut the shortcut and get to the point of multilingual support. You had a question? Okay, it's okay. Okay, thank you.