 Welcome to lecture 17, where we end the server-side JavaScript part, apart from the testing that comes later. So we have last time created a restful backend. We have designed the different URLs for our to-do application and we'll now go in and actually implement this in Express. Then we'll deploy it to the cloud, so we will not longer just run it on our own machine and then finally we'll add a bit of persistence or a bit of MongoDB. The learning outcomes are the same as for the entire block, now completing them basically. So we go from just designing to actually building a restful application using a database and actually deploying it to the cloud. So that's the same as before. All the literature references here are just tutorials, how-tos that show you how to use the different technologies because now it's really a technical lecture. And we'll continue on our to-do application. So we have users, they have notes, there is create, read, update, delete, access to all of them. We should have multiple users, of course, and we only look at the backend. So we try to do this in a very general way that you could implement many different frontends, for example a mobile application or a regular web application. And we do so, of course, as we have started to build this in a restful way. These are the different end-points that I'm suggesting. So it doesn't mean that this is the only set. You could do this differently. For example, you'll notice that a lot of real APIs avoid, delete all because it's just having a large consequence. So you want to avoid that someone uses this by accident or that someone uses this in case there is a security issue or so on. So many APIs do not have this delete all option. The other thing that is a bit special here that I haven't mentioned yet is that there is a patch request here. And so far we've only discussed put. So we use put to do an update. Here we update a user. And patch is a similar thing. So it's also an update. But there's a difference between the two. So put is a complete update, which means the request body for this request needs to include all the different attributes. So if the user, for example, has a name and a password, then each put request needs both of them. Otherwise, it's not executed. A patch request on the contrast is a partial update. So we do not update, we do not require all the attributes, but it's enough to send whatever you want to update. So for example, if I just want to change the user name, then the request body here would just include the user name. If I just want to change the password, I would just include the password. So there's a difference between the two. Patch is especially useful if you have large resources. So again, consider uploading videos or so on. Then if you just want to change the metadata, the title, the description, then the patch would be much better than the put, because in the put you would send all the data. So you would have to send the video again. This is just convention. So HTTP itself does not say that there is this difference. This is just how it's used in many RESTful applications, that you say patch is partial, put is a complete update. So I just included this, so you have seen it. The way we do this is we start off from the endpoint skeleton that you have already seen. So this is the basic express application. Then we go in and decide how we actually represent our data. We start off by just having that in memory. So I'll just design JavaScript variables, objects that have the right format. Then we implement get and delete, because those are rather easy. I don't need to look into the body, the request body. And then we do post, put and patch, because they are a bit more complicated. After that, after we've done the complete application, we'll deploy it to Heroku. This is one of many cloud providers. I use it in this course because it's easy. So it's easy to get started. You don't need a lot of knowledge in the beginning. And it's also for free. So that's of course good that you can use it. Then we do a slight excursion to middlewares. That's an important concept. And in fact, express is a middleware. So we'll cover what that means and what you can do with it. And as a last step, we'll introduce persistence. We'll do some MongoDB as an example. The endpoints, I will only do partially. So I'll only do the user endpoints. And then one notes endpoint or so, just that I don't spend the whole time implementing. But I'll upload the complete example. So you get the entire endpoints. Persistence will only cover briefly, since this is not a database course. So you should get all the foundations from database course. And then if you want to use this in your own applications, you just have to look up the right resources. So the first step is our skeleton application that doesn't really do a whole lot. We import express using the require statement. Then we initialize it. So that's just what express tells us to do. We decide where we host this application, which port. And this choice is arbitrary. It could also be four, five, six, seven or anything else. And then we design our endpoints. So this is just an example where I say, if someone sends a get request to slash, then I'll send back 200. Hello world. So that's just a basic answer. If someone sends anything else, so use stands for any HTTP verb, get, post, put, patch, delete, whatever. The star is also a wild card here. So any URL, then we send back 405 operation not supported. So basically our server just allows a get request to slash. Anything else is blocked, is responded with a 405. And then we actually start the application. Now, what we'll have to do is replace this one route with all the different routes we need for our application. So that's exactly going back to the, to the design in the beginning and say, what are the things we actually need, which URLs. And then finally, we have one route that catches everything. And that's exactly what we have here already. So if our endpoints are not used correctly, we just send back 405. Now, the next thing, and maybe I'll just start by running this. So I have all the things here. In this case, this is our basic application. If I start that, then you will just see that there is not a whole lot starting. And I probably need to install all the different dependencies first. So I've also created a package.json file that creates or that has all the different things I need. So it has express, it already has the dependencies for later. So I have already included them. But if you want to use any other dependencies, there is also the package.json file that just includes express. So that's sort of the basic one. Okay, now we start this. And I said skeleton is the application. So that's just the one that really doesn't do a lot. And now you'll see if you go to localhost 3000, you'll get hello world. So that's just the slash. If I request anything else, if I do slash test, it tells me operation not supported. And if I go into the network view, you see that it's 405. So this is exactly what I want. Now to test this properly, I have to go into postman because here I can only test the get request and in my postman application I can then test the proper thing. So I'll just use my test here. So if I do localhost 3000 slash, that's my get request. It works. If I change it to anything else, if I do a post request instead, it tells me operation not supported. So here I can test all the different options I have. Okay, so that's my basic application. Now I go in and I actually just create endpoints for all the rest. So for example, going back to my slide with all the endpoints, I just create one endpoint for this, one for this, one for this, and so on. So that's all I do. And basically I just say slash users, slash users, slash ID. And now the format here is in express when you use this colon, it indicates that this is a parameter. So I can later on in my function, I can actually access the parameter. So that's exactly what I'll do later on. These are the two get requests and then I have to set, for example, a post request. So this is what I'll do to all the different things that you see here on the slides. And of course, I don't want you to watch me copying and pasting code. So I have prepared this. You'll see this in the to-dos plane. So here I have created all the different endpoints. And I've also included examples of how you actually access the IDs from the URL. So you see here I say colon user ID. And then I can access that with request req.params.userID. So this is just copy and paste, essentially making sure that you get all the right verbs and all the right URLs. And just to show that this works, I can start that one. It's listening. And now if I do slash test, I get operation not supported. That's good. If I do slash, I get operation not supported. So I'm not supposed to do that. If I do slash users, I'll get something. Okay, and I've started the wrong application. This one already does something. So maybe we use the one that does not return any data. So this one just returns a string. If you look into the code, that's exactly what it does. It just returns 200, get all users. And you see that I already returned JSON. So I return the right format. I just say message, get all users. And that's what you see displayed here. You can also look at it as a raw data. This works. If I do users 23, then you say get user with ID 23. So you see that we already have the access. If I do slash nodes, get all nodes for user with ID 23. And the same if I do a post request to slash users. For example, it will tell me post a new user. So all the different endpoints exist. They already reply with something. Of course, this does not work yet. So that's the next step. So that's our endpoint. And now we decide on the data format. What kind of attributes do we have? What kind of schemas do we need to use if we have a database? For example, if you use SQL, how do our tables look like? As I said, this is not a central part of the course. So I will not go into detail. I'll just decide something now. And I basically say my users is an array. My nodes is another array. And each user is an object, a JavaScript object that has an ID. A username and an age. If I use a database, the ID part is usually taken over by the database, generate IDs and so on. So that's nothing I have to do. But if I do this by hand, if I do this in memory, I have to do everything manually. The nodes I design as an array of objects again. And each object is the user that exists and then an array of nodes. So basically, it's basically a map. It maps the user ID to all the nodes of that user. And then this one is again an array that contains all the different to-dos. And each to-do is again an object with an ID, with a name, with a content. And there is nothing special here. This is just how I have decided it. And by including the user ID here, I have essentially established the relationship to the user. So now I can refer to existing users. Okay. And here I'm missing a curly break. So this is, if you go into the next one I have implemented, get and delete, you'll see that this is just a format I'm using here. In this example, I have created two users, Alice and Bob. User 5, that's Alice, has two nodes to-dos for today, blah, blah, blah. So there are just two nodes included in the user nodes array. User 10, Bob has one node, shopping list. So that's all I've done. There's nothing, there's no magic about here. Now, after I have decided that, I go into and implement my get request. And as I said, I start with this because it's easy. It's usually easy to implement because a get request doesn't have a request body. There is nothing I have to check there. There is basically no input validation. The only input I need to check is if I want one specific user, whether the ID exists, that's all I need. And in our case, the get request for all users is particularly easy because we just return the user's array. So what I do here is whenever someone wants all users, I say response.status 200, so everything is okay. And I return the user's array as a JSON string. So looking into that, you see that our endpoint has actually gotten smaller compared to before. All I do is sending back this array here as JSON. And let's try that. So if I start this one, if I now do a get request to slash users, you'll see that I get, if we look into the raw format, you just see that I get a JSON string back. So this already works. Now, you can implement variants of that as you like. For example, a common thing is to say we don't include all the fields. Sometimes there are certain things you don't want to return. Maybe there are some kind of internal columns in your database that should not be visible to the user. Then you have to exclude them. So these kind of things are very common. In our case, we are a bit lazy here. We just return everything. Now, let's get all. Now, the question is how does get for a specific user work? Well, you have to do a bit more work because you actually have to find the right user. And what we do is we have a for loop. We just iterate through our users array. And we check whether the ID of the user is the same as the provided one as the parameter here. And the important thing in this case is you see that ID here is a number. I don't have any quotation marks around it. So it's a number parameter. And what the user is providing is a string. It's part of the URL. It's always a string. So I only use the double equals here. If I would use the triple equals, then it would actually always return false. So that's just a small side note here. And once I have found the user, I just send exactly that one back. And I return. Now, if I go through my entire user area and I do not find the right one, then I return 404 and I say the user does not exist, could not be found. So now it's already a bit more complicated. I have two cases and I return the user only when I find it. Now, there's one important thing here maybe to mention. Well, let's first try it. Let's say I want user 23. User 23 does not exist. So I get ID 23. If I do user 5, I get Alice back. So this seems to work. User 10 gives me Bob. User 9 does not exist. Perfect. One important thing here, and that's express specific, you see the return statement here. And I can actually also put that in front of this statement. Now, this is important because if I do .json or if I do .send, it's the same. So you can also do .send to send the text string back. But this causes the array to the response to be sent. It causes the server to answer the request. But this is asynchronous. So it happens in the background. Now, if I don't do return, let's remove it. I can just comment it out. What happens is that the response is being sent, but this code continues executing. And then we get here and the response is being modified and changed. And usually this is when express gives you an error. It complains that you are modifying the header after you have sent the response. I mentioned this because it's an extremely common error. So if we just try this, there's no problem with this code. So the compiler doesn't complain. But if I execute this now and if I say, if I do nine, everything works because the for loop, the if here is never entered and we get down here, we return 404. If I now choose a parameter that does exist, then I'll send 200. I do not return and then this will continue executing. So if I do 10, it actually does work. But if you look into the server log, you see that there is an error here. It says, cannot set headers after they are sent to the client. So the framework complains that I'm here trying to modify the headers even though I've already sent them with this JSON request. So in this case, it actually doesn't cause any error. The behavior is the same. But if you do anything here, for example, it might be that I delete something here, then this can cause big problems. It can cause stuff to happen that is very hard to debug. So be aware of this, that you always return when you actually want to send a response. That's an important thing to do. Okay. So these are our two get. Now the other thing we have to do is the delete. We want to remove something. It's pretty similar to get methods. There is no request body. There is no input validation. All I have to check if I want to delete a specific user, whether the user actually exists. And now of course we get into some JavaScript stuff here. Now the way to get the entire array is, for example, to use the slice method. So the slice method just removes elements from an array and returns them. If I call it without parameters, it just removes all of them. So this is all I do here. I just return my array 200. Everything is fine. So that's exactly what you see down here. No, you don't. Here in the delete, I slice the array, I empty it, I return it. Again, we can test this very easily in Postman. So if I get all the users, you see we have Alice and Bob. Now if I delete them, the method returns exactly the same. It returns me all the users it has deleted. If I now do a get request, I should get an empty array. So now there are no more users. I have deleted them. So this seems to work. Now the individual delete is very similar to the individual get. I have to find my user. I have to check whether the parameter, the user ID, is the same. And if I find the user, I just return it. And again, I use the splice method for this. So I say, please remove the element at position i. Please remove one element. So this just gives me the user and removes it from the array. I need to return again. Otherwise, I get this error. And if I do not find the user, I return 404. So that's it. This is all the get and delete. And then if we look into the nodes endpoints. So for example, we would like to get all the nodes for a specific user. Now it gets a bit more complicated because we basically need to iterate through our nodes array here. And we need to check whether we find the object with the right ID. So for example, if we request all the nodes for user five for Alice, we just iterate and check, is this user five? Yes, then return the user nodes array, basically return this array. So you have an additional step to find the right user. And once we have done that, just return the nodes. And similarly, if I want to have a specific node for a specific user, I have two for loops, I need to find the right object in the nodes array. And then within that, I need to find the right node and return. Which also means now I have two chances for a 404. It could be that the user doesn't exist and it could be that the node does not exist. And delete is similarly equivalent. If I want to delete all the nodes, I have to find the right element in the nodes array, splice it, return it. If I want to delete a specific node, I need to find again the right element in the nodes array, the right node in the nodes array and then splice it. So this works similar to the endpoints for users. It's just sort of one for loop more complicated. Okay, so those where the get and the delete endpoints, they are reasonably easy. If I work with my own data structure, with my own in-memory Java object, then a lot of this is quite complicated. I need to make sure that I modify the data structure properly, that I search using for loops. If you use a database, you usually get a lot of this stuff for free because you have all the optimizations that make this very easy, but you'll see that later. Okay, so I'll stop here for now. In the next part, we'll now go into post and put. They are a bit more complicated because we actually need to look at the request. We need to look whether the right data has been included.