 Hi, and welcome back to another episode of Visual Studio Toolbox. I'm back again with my guest, Jeremy. Hey, Jeremy, welcome back to Toolbox. Thanks. Glad to be here. It's awesome to have you, and it's almost like we never left the studio, or we wear the same thing all the time. That's right. This is the only, actually have a closet full of hangers that have nothing but the Azure logo shirts on them. That is amazing. The cat's out of the bag. There you go. We're doing two episodes today, but they'll be published separately, so folks might have not even seen the previous episode. That's totally fine. So today, we're going to really focus down on Cosmos DB, right? Absolutely. Cool. So what are we going to talk about? Give us a summary. So I want to just drill into a lot of the aspects of Cosmos DB. I've gotten pretty excited about it. In the other episode, we talked about link shortening tool that I used. Yeah. I enhanced that tool to store data into Cosmos DB so I could find insights from my application. What I found is just works incredibly, it works really well. As a .NET developer, there are a lot of ways to interface with it. So I want to talk about what Cosmos DB is, why it's important to .NET developers, and show a little bit about how to get plugged into Cosmos DB and start using it as a service. Right. Folks didn't see our previous episode, that's fine. We'll link it. It's not a dependency or anything. You showed a cool Azure function thing, and that was a really good thing to watch if folks are interested. But let's talk about what Cosmos DB is. Again, I want to make sure folks coming in, maybe really not knowing what this database technology is all about. If you can give us a quick summary, that'd be great. Sure. So when we talk about functions, we talk about serverless, which I talked about flipping that less server. Cosmos DB is what functions are to compute. Cosmos DB is to storage. It's the idea of a hands-free ability to host an enterprise-grade database. It's no sequel. That's the defining way to describe it at a baseline, and then it has a lot more because of what we built into it. Right. So it's no sequel. It's document-based, but what's neat about it is it supports a bunch of different interfaces to connect to it. One of those interfaces is what everyone who has followed the history is familiar with, DocumentDB. So that was the previous incarnation. And what's interesting about DocumentDB is it supports SQL syntax. So even though in the database we're storing documents that can have different schemas and properties, I can still go to my comfortable select star from group by, order by, and use that familiar syntax to query the database. If, however, I'm a traditional NoSQL developer and I'm used to something like MongoDB, we have a MongoDB driver that works out of the box. In fact, I can take an existing MongoDB application, update the connection string, and connect to Cosmos DB and use that. Yeah, that's very cool. And then there's two other APIs. There's a TableStorage API, which, again, you update the connection string. Your app that uses TableStorage is completely transparent. And then there's one that I always love to bring up this interface name because it's called Grimlin, right? So Grimlin is what we call a graph interface. And the idea is that you're storing nodes and vertices that connect the nodes. And the canonical example is airports. So you have airports as nodes, and then there's flight paths that connect them. And Grimlin is a language that can describe how to traverse those paths and then find information, things like what's the shortest path, what is aggregate information along the path, et cetera. That's really cool. And for folks that might be Googling later on trying to find some information about this product, the rename is always like a moment in time with Microsoft and it changes something like that. So to be clear, like DocumentDB was what this product was called. It's not called Cosmos DB, but we still refer to DocumentDB interface, right? Of the way you can interact with it. But that does just a historic factor, right? Of what it was called. So the name might be dead from the product name perspective, but it's still alive from a interface perspective. It is still alive. In fact, I did a workshop that was a node workshop based on Mongo or based on DocumentDB. And of course it's iterated now to Cosmos DB. So I created Cosmos DB and was able to migrate that existing application without changing code. I just updated the connection string and that existing DocumentDB driver worked as is out of the box. Yeah, we changed the name because it was honestly just causing confusion. I remember the conversation. Folks didn't realize the power of this technology and hopefully the new name gives us some new life. It's a lot more fun, that's for sure. Yeah, I was impressed. Sometimes we don't pick the best names. I'm in marketing, so I've spent time naming stuff. You can blame me for CodeLens, for example. Oh, okay. You can praise me or blame me. I was on that project. CodeLens. Yeah, this was a good one. I like Cosmos DB and let's jump in to see what it's all about. Sure, so the first thing I'm going to show you just from the portal is the experience of creating a new Cosmos DB. I'm not going to go all the way through it because I have an existing one. But if you click into add a new resource and just pick Cosmos DB, it's going to ask me for an account ID. So this is just a unique identifier to access the database. Like most Azure resources, there's some URL and that URL needs it. And you can then find a beginning. You can always change it, kill it later. Right. So we'll do my VS Toolbox DB. And then you pick your API. This is what we were talking about. It supports different ways to interface. The traditional document DB, the MongoDB interface, the Gremlin or Graph interface, and then key value storage, table, simple storage. So we can pick any one of these. Let's just do table storage. I pick my subscription. That's my billing. I can create a new resource group. I pick my location. And then I can enable geo-redundancy. What this is key, because this is going to give me the ability off a map to basically click different regions and have it automatically span those regions for me, which is a pretty cool feature. Yeah, for what I often hear people describe this technology of ours as a globally distributed database, something they can live in lots of data centers. And I believe you mentioned there's different kind of retention policies, right, of how much replication you need. We kind of concurrency need all that stuff. Right, exactly. In fact, this is one that I've created. And this is a live one that I'm using for my link shortening tool and it stores metadata. And you can see that for my purposes, because I'm the only one who's actually exploring the data, I just have this in the Atlanta region, which is close to me, the Eastern US. I've got an activity log that'll show me how it's been accessed so I can quickly look at, basically, have there been any errors or issues or anything along those lines. And this is just basic, what's available in Azure by default. You didn't configure any of this telemetry, for example. Correct, this is completely out of the box. What's really nice with this is the data explorer. And this allows me to look at my collections, which are containers of documents. They're kind of the corollary of tables if someone's used to relational databases. You can see I have these URL stats. And this is using a SQL-like syntax, like star from C. And it's giving me IDs so that I can expand these on demand. So if I click on one of these, you can see the actual document that's stored. In this case, there's a unique identifier. It redirected to a page, has a timestamp, and this particular document was a Facebook redirect. If I click on another one, though, this is the power of Cosmos DB and document databases, is this one has a property called Twitter. So I'm not locked into a specific schema. I'm counting different types of items for that. Yeah, yeah, this is a great power. I mean, as a person that's used to creating a table back in the SQL world and having that table be sort of the definition of what kind of data you can store, this is a different way of thinking about it in terms of no SQL. Well, and here's what's really nice. I want to jump into some code and show an example project that I created. But what I love about the way that both the document DB and the MongoDB interfaces are written is developers used to SQL are also pretty used to object relational mappers. You know, NED framework is a very popular one. And the whole idea is I deal with my code in classes. The database stores them as relational tables, so I need something to map between the two of those. What's nice when you take a document approach is even though the document is schema-less and you can have open data and you could do a dictionary key value pair to set the document, you can also create strongly-typed classes and you don't need to go through any intermediary to store them. The driver just accepts that class as is and stores it right into the database. Cool, let's take a look. So if we take a look and I'm going to come back to the link shortener in a second, but I've got a Visual Studio Code window opened here. And in this project, what I did was took the USDA food database and import it into Cosmos DB and then stood up an application so you can search food items. And if we look at these links in the configuration, I'm going to just click on this food group link. You can see the way it's stored actually on the website is this format of these weird squiggly lines and arrows and whatnot. So part of what this application does that I won't spend too much time on says a parser and that parser is going to read the lines in and split them up. And then it has an importer that actually imports the collection. The transformer is how it maps them to this is a food group item or something different. But the importer, this is the actual code that I'm using to populate the database. And you can see that we have a helper method that gets our database connection. So we're pulling in our secrets, making a connection. Then what I love about this is I have Git Collection. So I'm getting a collection of a particular type. It might be food group, food item, something like that. If that collection exists because this is the import utility, I'm going to drop it and recreate it so I can just import fresh. This is the code that inserts an item. And that's all there is to it. This item is passed in. You can see this is a generic type of T. So the item might be a food group class and I'm going to show this in a second but literally collection insert one async. And that loads it into the database for me. I've got a completely separate project that has my models, my classes. So here's food group, for instance. This could be a completely Poco, right? Plain old C-sharp object, CLR object, if you will. But I've chosen to go ahead and add some metadata that says, okay, this is a unique identifier. Right, so you use an attribute to make code the BSO and ID. And some people might want to do this fluently. So keep it out of the class completely and you can do that. But for the purpose of this demo, we did metadata. The other thing... Great power and flexibility comes great responsibility. Exactly, with great powers, great responsibility. The other thing I wanted to show, and this is a nuance I think it's important to call out because this is what people run into when they move from that relational model to the document model. The first time I decided to create a food item, what happens is every food item in the database has a set of nutrients. And that can be protein, fat, carbohydrate, vitamin C, whatever. The database doesn't necessarily have all nutrients for all foods. So I may have a food that only has three. I may have a food that has 200. So the way we would typically model that in a C-sharp class is just like this, right? I have a nutrient and I have an array of nutrients, which is straightforward. The problem, though, is when it stores that in the document database, it's just an array that can have whatever inside of it, because remember this is a schema-less. So if I wanted to ask a question like what are the foods with the top protein content, it would have to go into each one of those records, scan the array, find the entry, and pull that out. It becomes a very intensive operation to complete because of the distributed nature of the data. So what I'm gonna do is search for this Cosmos database that I have preloaded so that I can show you exactly what the data looks like inside the database. And once again, I have this experience that I can go into the Data Explorer, I can expand my collections, and here you see I have food groups, nutrient definitions, and food items. The way this is stored in the database, and we'll pull up this document, you can see that I have an array of weights, which is fine, but this nutrient doc has nutrients, and then water here is a property. It's not an entry and an array, this is nutrient doc dot nutrients dot water, or dot nutrients dot energy. So now I'm referencing just a path on the document, and I can do that indexing. I still wanna be able to work with it as an array from C-sharp. So all I had to do for that is just put a helper method that basically when we pull it from the database, we look at this free form document format, that's this nutrient doc, and build up our strongly typed list of nutrients. And then on the flip side, when we're saving, we just go through our strongly typed list of nutrients and create a more loose form JSON type document. Yeah, so this makes sense. I mean, I've seen this kind of pattern before, and like processing XML in some cases, you wind up in very similar, like as some other systems XML data, right? And you're converting it into some structured thing. Right. In your database and there, the way they've organized it doesn't make sense for you as a query, you know, performing thing. And you have to reorganize it, so. Right, so that will actually map in, and I wanna show the simple case of accessing this data now. So I showed the experience of importing it. I've got a little test application, and this is just making sure that I have my connection set up correctly. So you can see here, this is the power of .NET Core, right? We've got a very flexible configuration model. I can have my keys stored in app settings. I personally don't wanna check those in and have someone steal secrets. So in this case, I've set environment variables. You can see here, it'll pull from the JSON unless it's overridden by the environment. And then those secrets come in, and when we deploy this into a website, we can always set them up in the application settings. So it's very flexible like that. This is getting the list of groups, and this is the code. This is using the MongoDB driver. It's basically getting the name of the collection that's just the helper I have, as courierable to a list. This is link syntax that everyone should be familiar with. Looks a lot like entity framework, only I don't have to add that as an intermediary. So if you're a development developer, all of this is like being home, nothing unusual. So it just iterates the groups and writes them, and then here we're gonna grab the first food item. So again, it's getting that collection as courierable first or default. And then I just pull some information out. So let's take a look at that from my terminal and run it. Let me see where I'm at. I'm in the console test. And of course, this is .NET. You can just do the .NET build release, and it'll come through. And in this case, I've got a helper for connecting to Cosmos DB, and I share my models between several projects. So you can see I've built the models, the connector, and now I've built the test. So let's run that test project really quickly. So we'll go, and this is usdaconsultest.dll. And so it's going through the connection string, getting that list of food items. It's listed it out, and then I tap that and it grabs that last food item for me. Cool, so we know it's working. The next logical step in most applications is to build a web API. So I've got this web API, and this is where the scenario that I talked about with, you know, give me the top nutrients comes into play if I want to sort, for example, from protein. So if we look at the controller, and again, what I love about this is the .NET developers are going to be just very familiar with this type of syntax. So here, I'm getting a specific food group. So that's my route, passes in the code. I'm gonna return not found if you pass me an empty code, that's just logical, right, just being safe. Now I come through and I grab that collection, and here we go, find async, food group, where code equals code, just link syntax, straightforward. When it grabs that, I decided to enhance this as well and grab all the food items that are in that food group. So if you get that group detail, you can get a list of items and drill into those. And then it just returns an okay result. That's it, that's the API endpoint. So if we come inside of this application now, let's actually expand this out. And control K is the secret code to clear the screen. I did not know that. Yep, so. I think there's some right click equivalent that I use. We'll go into this and again, .NET build. We'll go to release, you can build the debug if you want. And we'll let that spin, again, models connector being reused across the projects. And then we'll do .NET and we'll go and run this. One of the cool things when looking at your code is, like you said, it's very familiar to .NET developers, the Cosmos client library that the Cosmos team builds gives you a lot of that link sync syntax against it. So as a .NET developer, you can come in here and be like, yeah, in a couple of hours, I'm building against Cosmos. You don't have to really learn some new thing you have your explorer in the cloud to see the data. That's the usual question, like my app's now functioning. Is it writing the data? Is it reading the data? Exactly, what is it? So you have multiple points that are just there. Well, there's actually a quick start too that I love because when you build a new Cosmos DB, you can literally click a button in the portal and it'll create a populated list of items and give you a download for a project to hit the ground running. So it's not building it yourself, but it's having an app that you can look at and interact with. So we're going to just hit this food groups endpoint to get the list. And again, what's happening here is my API is going out, grabbing Cosmos DB, it just pulled back the list of food groups. Let's pick one of the food groups. I'm going to do 200, which is a baby product. So for the, oh, actually spices and herbs. So then we got this list of foods with spices and herbs. Maybe your baby likes spices and herbs. I'm not here to tell you. Exactly, so now that we have a database and we have an API, we have everything we need to do to build a front end. The focus of this is Cosmos DB. So I'm not going to go into too much detail with the front end. But what I do want to show you is actually, let's keep this running and we'll open up a new terminal. And in the same project, I have a web, actual web page. It's a single page application that just uses VJS. I'm not here to say angular, react, whatever. You're not picking sides, that's fine. Quick way to get up and running. So what I'm going to do is see if I have any Docker pieces running. Let's actually make sure Docker is good to go. That little detail. Yeah, it is good to have Docker spinning for that. It's getting there. We'll give it a second. But what I've done is I've just created a really tiny application. You can see it's got a bunch of HTML that uses templates and data binding to do things like list nutrients and list food items. Just one page of information. Yeah, there's no backend code. This is all JavaScript using VJS. You run there off the public endpoint that you've built for this example. Yeah, exactly. And for this, the JavaScript include is just using very simple code to make calls out to the API endpoints. So we've got queries and that actually reminds me, well, Docker's spinning up, it might be done here. I wanted to show you this inside the web API for the foods controller. This is the one that allows me to do things like sort or filter by a group. So I've got from the query string, I've got a group ID, I've got a from the query string a search. And so this will come through and basically project the item. So MongoDB has something called projections where you can say, even though there's this much data in the database, this is what I'm really gonna show from my UI. It's a subset of records. Exactly. Based on the filter of some sort. Yep. So it sounds just like a link that I'm used to, you know, entity framework and all that. Exactly. It's the same thing. This should be very familiar. Description contains the search. Yep. And the food group ID equals the food group and you can see I can chain these together or basically do one, the other or both. And then I go out, get the query, order it and list it. So again, very straightforward in that sense. Let's come back and see how our Docker's doing. We'll do Docker. All right. So I'm gonna run the image I created. There's a very simple Docker file. They use something called busybox, which is a really tiny web server because all I'm serving is an HTML page and a JavaScript file. A JavaScript page basically. Right. So I'm gonna map port888 because port80 is inexplicably taken on my laptop. And we call this USDA web. So we run that. I'm just gonna do this to make sure it's still running. It looks good. Let's come back into our browser and hit local host. Docker for dev is so powerful. It is. Under values sometimes. And here's my simple Vue.js app and I can select a food group. Let's do spices and herbs. Click filter, boom, it pulls these. And then when I click on them, it's going to give me all the nutrient information. So zinc, manganese, vitamin C. And then as a person who eats 100% plant-based diet, what I might be interested in is breakfast cereals. Which one has the most protein? So I'm picking the group breakfast cereals, nutrient protein, give me the top foods. Look how fast that was. It just came back, sorted. And these, when I click on this, presumably, it's gonna have a higher protein content. Well, actually that's kind of small there, but it's also a small serving size. So if we change to something that has a bigger serving size, like a cup, you can see the, okay, well, let's do one and a half. Let's actually take the maximum cup amount. There we go, now we've got good protein. And then we can search with the tech. So this is a fully functional app on Cosmos DB. And what's really neat is for the front end, I don't even have to care. I'm just working with straight APIs. For the back end, I can use either DocumentDB driver or Mongo. Now, what I showed you in this.NET Core app was the MongoDB driver. I want to take one quick second and show you what I use for my URL shortener. And that is the DocumentDB. And what's really nice, and we talked about this in the episode with Azure Functions, is the fact that we have these things called bindings that make it really easy to interact with external sources. There's triggers and bindings. So if I open up this project, I'm using the DocumentDB interface to store all my information. And when I look at this function host, what I did was added this attribute for DocumentDB. I gave it the name of the database, the name of the collection. Create it if it doesn't exist. I love that about bindings. The first time it calls, it can do that. And my connection string is a secret that I store in app settings so that no one else sees it. And notice I have this out dynamic doc. So remember schema list, so I can create the doc however I want. Using this API at the very end of my method, I create an expando object, which is just a key value pair basically for dynamic. I'm adding a GUID to make it unique, putting the page and the count. And then based on the custom event, and this is if it's Twitter, Facebook, LinkedIn, my blog, I add that. Notice that there's no other code. This is an out parameter, so by setting the value of the parameter when this function exits, automatically writes the document into Cosmos DB form. Now even though Cosmos DB is a document database, I can plug it into Power BI and pull up rich metrics and data and filter. So what I've got here is where my redirects are coming from. I'm just doing one day, so I'm seeing where are the most click-throughs, what hour of the day is popular. And these are my popular links and I can come into the filter for example, and let's look at change this from last one day to last one month and apply the filter. And now you can see this is Sunday, Monday, Tuesday, so Wednesday, wins for click-throughs. This four o'clock UTC, which is around noon Eastern time is popular for clicking during time of day, and then I've got all my links here. So I get all the information I want, pulled out of that Cosmos DB. Cool, very nice. So you've got the whole story basically end-to-end. If you can store data, your flexibility and how you query it, you've got the ability to add Power BI dashboards on top of that data, what more can people want, especially just to give it a try. Exactly, it's working great for me. And again on the site, there's plenty of quick starts that let you download the project, prepopulate the database and get up and running in seconds. All right, sounds good. Anything else you wanted to show about this today? That's it. Everything I've shown today is open source on my GitHub. We'll put the links on the site. There's the Cosmos DB that'll import the data for you from the USDA. It's got that built in as well as the URL shortening project that I have here. All right, well that sounds good. And we really hope people give Cosmos DB a try and thank you so much for coming on yet again. Yeah, thank you. We'll have you back. It's always a pleasure. And again, hope folks give our technology a try. It's all we're trying to show you all the coolest stuff. And this is definitely some of the most cutting edge things that we've been working on, so glad to do it. A lot of fun. Thanks for having me on. I'll make sure to wear my Azure shirt next time. You always gotta wear it from now and if you don't wear it, you can't come on the show. A band from the show. All right, thank you for watching everybody. Have a good one.