 APS, what, why, and how. For this session, we have with us speaker Devi, ma'am. And she is a freelancer programmer, consultant, and a trainer. She has done her M. Tech in computational science from Indian Institute of Science. She is currently working as an architect, lead developer for Power to Fly. I request ma'am to start the session. Thanks for the introduction, Padmish. Good morning, one and all. I hope you had a wonderful keynote session. And the house is full here, and I hope I'll satisfy all of you at least a bit. Yeah, let's start the session. So we're going to talk about REST APIs and why they're useful, what are they, and how do you go about designing a one, and what are the choices one should make while choosing a library to make a one, right? So I work for Power to Fly, which connects women to remote or part-time jobs. Our platform is such that it's a website, like LinkedIn for women, wherein we had to develop in-house APIs for ourselves, for our size. That's where all this understanding for diving into REST APIs has come into picture. Let me share that with you. So what is an API? So APIs are basically a programmable interface. All the context of this talk, I'm going to be only with web services, right? And REST APIs are possible in other contexts, but we are going to stick only for web services. What is an API? It's a programmable interface. As we are being humans, we interact with computers or websites manually, right? But if we want to make those web services, we interact it through the programs. How do we do that? It is via APIs. So for computer programs to communicate with each other, that's via APIs. So what does it mean by designing an API or having an API? It's designing a format for the client to communicate with the server. So a server is offering an API, a service as an API, and it has to have a format on how to request for something to be done. And in return, what to expect as a response? How does it respond? Like our languages or the programming languages, they have certain rules of specification, every language, right? Whether we use it in communication manually or through programs. So similarly, APIs have a set of rules and specifications. So why are they useful? Why are APIs useful? You might have seen that if you are a developer and if you are doing a core of, let's say we are developing a product. Not everything we can build from the scratch, right? We use third-party services like Amazon Web Services or Stripe Payments or Elasticsearch or so on or anything. We have lots of software which has already been written in different technology stacks. So they are offered as services through APIs for us to use. So that we don't need to reinvent the wheel from the scratch or the best part of the APIs is that we don't need to bother of what the technology stack is for AWS. If you are using AWS as storage for storage, you don't need to bother about what is the technology stack they are using or which programming languages it has written. You don't bother about it. You just think of what is the API and how should you request? I just want to dump all my images onto AWS and I'll just look at the API, nothing else. I'm not bothered about the technology stack or anything else. So a public APIs, it lets you glue in different technology stacks or languages of choices, right? And not all APIs need to be public. One can have a company inside. They can have private APIs. They're not exposed to the public. It's not for the outside world to use them, but it is just for them. You might have used Slack for communication or almost all of you use Facebook. You've got mobile app on your mobiles and a website, right? But both of them are separate in their UI, not the whole technical stack. There wouldn't be the application. The application totally wouldn't be separately written for mobile app and separately written for the website, right? There is a lot of common thing, the code, on which there is a layer, the UI layer is separate. So they expose, the core Facebook is exposed as an API on which the UI talks to them, right? We'll see what, how APIs look like. And so now that we know a bit of what API means, we'll see what is a REST API? What is REST there? So REST there as an acronym, it's a representational state transfer. It's all about a style of architecture of how do you model the entities that you have? In object-oriented world or oops world, we talk about entities as objects, right? We call the entities that you play with or deal with as objects, whether it is customers or orders or jobs or talents or whatever you are dealing with. They're all called as objects. But here in REST world, they're called as resources. So the primary focus of the design is on resources in designing REST APIs. So when you think of your application, you should think of the resources. What are the resources that I have while designing a REST API? And then the URLs. So REST APIs, in REST APIs, the resources are identified by URLs. Each resource is uniquely identified by URLs. I will come to there. And the best part of REST APIs is that it takes the full advantage of HTTP. HTTP as a protocol, it's so sophisticated that it has got its own methods. Get, post, put, delete, you might have heard. We'll get into them. And the response codes, along with the status, right? REST takes full advantage of them. We'll see how, right? When I say each resource is identified by a unique URL, I'm in this. So let's take as an example, let's take a e-commerce site or something like that, wherein we have customers and orders, orders placed by them. So what are my resources in my e-commerce site? I've got products. I've got customers. I've got orders. And I've got maybe items, right, in the order. So they are my primary resources, primary resources. And what are the actions? One customer can place a new order or edit an order or delete an order or things like that. They're all actions, right? In a non-REST world, you would say slash customer slash add slash order or slash customer slash order slash add. That add comes into the URL in non-REST world. We'll see how REST world takes care of it. So what are the URLs here? Slash customers. It represents the collection of all the customers that you have in your web service. When I say customer slash five, this is going to identify a particular customer with ID five, okay? How it has stored, whether it is a DB or whether it is a, whether it is a RDBMS or a non-RDBMS stuff, I don't care, right? We are designed in API for our e-commerce site. Let's say that. So customer slash five is going to represent a customer with ID five and all his orders are going to sit at customer slash five slash orders and a particular order can be fetched via its ID and so on. We'll come into the details a bit later. But the main point here is you don't have the actions in the URIs. So you don't have order slash edit, order slash delete or something like that, order slash create in the URLs because it's a REST world and we'll rely on HTTP verbs rather than putting the actions in the URLs. So what are the HTTP verbs that I've been talking about? So HTTP as a protocol, it has got many, many work, many methods. Get the most important things and most popular ones are get, post and put and delete. So which one of them is used where? Get is used for retrieving a resource. So you just want to get the information. You don't want to change the state of anything in the server but you just want to get the information. Let's say you want to see the orders of a customer who has placed few. So you're not changing anything on the server and you're just retrieving. Then you would use get. Post is for creating resources. So let's say when an order is placed, a new order resource is going to be created at the server, right? Then HTTP post is being used and let's say if you want to update an order, let's say you want to change the quantity of something or you want to modify an order, then the HTTP verb that you will be using is put and for delete it's delete. Here you can see that there's a slight overlap between post and put. Both can be used for creating a resource but the choice comes from this. So get, put and delete are idempotent. What do you mean by idempotency? So how many ever times you do that action, getting or putting or deleting? It's the same. It doesn't change the server state but if you post the same thing to a server, the state will change. We'll see that as an example. Okay, this is slightly blurred, sorry for that. So this is what is our example. We have got customers, we have got orders and items and products. So this is the documentation that I'm going to give to my API users. What is it? The customer's endpoint, meaning the customer's URL is going to support, get and post. What does it expect? What does it expect? It expects a name, which is a string as an input in the request and it replies back with a name and the self URL and orders. Orders URL in the response. Let's see how it behaves. Before going into the example, I want to just say that how should I arrest APIB? It should be simple. It should be simple for us as humans to read and understand or comprehend what it means. So JSON is a kind of default standard or XML and HTML can also be used as the language between client and server. But JSON is the most predominant one so that the developers also can read it. The API should be consistent across. What does it mean? So when I say the URLs here, here I don't say slash customer at one place and customers at another place. So the API should be consistent across. The URL should be guessable. Whenever I say customer slash five, I should be able to guess that the orders URL would be slash orders slash something with ID there. So once you work with a arrest API, the working with any other arrest API should be easy and guessable. And backward compatible. What does it mean? Let's say AWS, you are using AWS for your project and you have gone live. Things are fantastic and your website is getting lots of fits. You're storing all your assets and images or videos in Amazon web services. And suddenly, let's say Amazon changes its API. What happens? Your product will be, I mean your website will be down because they have changed their API. The next time you won't rely on AWS anymore. You won't use them, right? Why? Because it'll cause you as a user the downtime. So APIs, whenever one designs an API, a public API especially, it should the designers or creators of API should think of backward compatibility in mind. So if I'm going to add a new feature to my API, how should it be in the future? How can it support few other things in future? And if something is going to change at the core, how am I going to take care of it in future? So that four sides should be there in designing, while designing the API. And actually maintaining a version also makes it easy for maintaining the backward compatibility. We'll see how versions are there. And HATUs, we'll get into HATUs also a bit later. So we'll see example of orders API. So in the starting, my database is empty and I don't have any customers, right? I'm assuming curl, most of you might have used. Curl is a HTTP client, which you can use on Unix. And it takes an argument, the method, HTTP method with minus X option. And I'm assuming that my API server is sitting here. API.x.com is my API server, right? And slash customers, slash v1 is to maintain the backward compatibility, the version has come into picture. Now let's say if I want to change the API later, then I'll put v2 there so that the users that are reliant on me wouldn't drop their, I mean their websites will not be down because my API has changed. So when I request for it for this URL, what am I getting? I'm getting HTTP 1.1, these are all HTTP headers with a status 200, status code, okay. What does it mean? A status code 200 means everything went well and here is the response that you have asked for. Because I don't have anything in my database yet. What is the response that I've got? It is an empty list. I don't have any customers. The server has replied that I don't have any customers. What is the content type here? It's JSON. So the client is asking the server through this URL slash customers and the server has responded with a status code and a content type and this is the response. This dictionary that you can see is the response. Now let's create a customer to see, right? What do we, which HTTP verb would we use for creating a resource? It's post, right? So in the documentation, in the documentation, we have seen that it expects a name to create a customer. What is the name type? It's a string of length 64, of max length 64. So let's use post. My endpoint is still slash customers because I am trying to create a customer, right? Customer resource, so with the name Alice. So the client is just, the client is knowledgeable only about the API URLs, nothing else. How the server is adding the customers where the server is adding the data, whether it is maintaining in the RDBMS or no SQL database or in a flat file system, I don't care. The client doesn't care. It just knows the URL, nothing else. So what did it respond with? The server has responded with, I mean, I have omitted a few headers here, which are redundant in this context. The server has responded with 201 as a status code and created as a status. So the server says that, okay, I have created the resource that you have asked for and where did I create, how you can access it? It is a location header. So I have created a customer with ID one with the data that you have given. That is the server response. So if I want to see the customer one, I can go and see now. How do I see? I have to ask for the server, ask the server again. What will the HTTP verb be? It is again get, right? Sorry. Here, get is always used for retrieving a resource. So I want to get and see how the customers slash one is. How is the first resource? So what do I get? I get back from the server, name Alice and few links. We'll see what those links mean in a bit. So we started with an empty database where we didn't have any customers and we have created a customer with a particular name. And we could see the details. We could get back the resource. Now let's say I want to change the name of the customer. How do I change something? Once I have created a resource, many a times it's not just the same, right? We want to change it or update it. How do we do it in the APIs? It's through put method. Put method of HTTP. So it was name Alice before and I'm changing the name as Bob. So server again responds with 200, okay? Meaning what you have requested for is done. That the name has changed. Now if we request for slash customer slash one, it would give back the response as name Bob, all right? How do we delete? Sorry, I should have kept here curl minus six. It was a typo. So curl minus six delete. I can delete it like this. So any resource you can create it, you can update it. You can delete it. What else you can do? I mean, any resource or any object, the actions that you can do is adding, editing, deleting, nothing else, right? So rest is based upon HTTP to take these actions. So let's go back and see what these links mean here. So in browser build web, when we open our browser and open a website, we don't need to be taught where to go on, how to go on. We just see a link which is interesting to us and we see all the available links where we can go and just click the link, right? And it'll take us further. That's the way we navigate as human beings. In APIs, how does the program know what is available as a next step? Or how does the program know what it can do at this particular state? It is through Hatios. So let's say you open the Facebook site. You can see sign in button or sign in link. Or if you have signed in already, you'd say post message, something like that, right? So if it is through REST APIs and the state is that, and if I'm requesting for the home page through REST, then I should have the links of what all can I do? How do I post a message through API? How do I add a friend through API, et cetera, et cetera. So that is called, that is what is meant by Hatios. It is how you navigate through the API programmatically. So here, when we ask for customers, when we requested for customers with an ID one, we have got the orders link also. See, you have asked for customer right now, and you could be interested in the orders that he has placed already. So you have the orders information, you can get the orders information here by requesting that URL. That's what is meant by Hatios. We'll see examples of Hatios as the orders link. So from the customers, from the first customer, we have got all the orders. Let's see, let's request for this URL. What do I get? What do I get? I'm requesting for a particular order. I want to see all the details. Then I've got a self URL, which means that it has just the same, sorry, here there is a typo, self is wrong here. Sorry. The customer, the customer for this order is customer slash one. And what is the order actually? It has got some date and items. What are the products that were in the order? So Hatios actually adds so much of richness through REST, through REST because it lets the program intelligently handle on how to navigate or what are all the possible actions as a next step rather than asking the server every time, what is the possible action right now? Yes. Now that we know a bit of REST APIs, how they should be. Let's see how to choose a library. So let's say you are using Django or Flask to build your websites. How do you choose a library? You won't start with scratch, right? Start from scratch. When a data comes in here, here how did we add a customer by posting this data, right? Name is equal to Alice. The library should make sure that the name is a string type and below length 64 as the API documentations is. Here we have documented that the name is a string type and it should not be more than 64. Who takes care of that? So there should be a data validation layer before the API in the implementation of the API and authentication authorization. It's a most important thing that you don't want everybody to create customers in your website, right? Not anybody can post any data to your API and create customers in your database. It's not what you intend to do. So there has to be an authorization and authentication layer for your API so that not everybody can do whatever they want, whether the user is your service user or not. And so there are n number of libraries, even for Django or Flask or anything, actually, any framework that you can choose. But almost all of them are very tightly coupled with the ORMs of the databases. So what's wrong with that? Once you have designed your database, the libraries which are tightly coupled with the ORMs will generate the APIs on the fly. You don't need to do anything as a developer. You just put in the library and maybe do a few meta stuff and it generates the whole API for you without you taking care of it. But what is the problem with that? Whenever you change your database structure or do a migration, you can do a migration inside. But what happens to the API and what happens to the clients which are relying on the API, they're all lost, right? They won't work anymore. So it's not a good idea to have a very tight coupling between your databases and the APIs. Because the databases are inner stuff. You can take care of them through migrations or anything else. But APIs are public facing. They should never be non-backward compatible. And the libraries should support pagination. Like you don't want to show all the customers on one single page, right? I just want 50 of them to show on my first page or et cetera, et cetera. So the API library should be able to support pagination. What do you mean by rate limits there? You might have seen with Twitter or Google APIs or any good APIs. They'll have rate limit that you as a user cannot request more than 300, request per, maybe R let's say. So that their servers are not too overloaded. They will decide that number. The API writer or the API creator or the designer will decide that number based on their architecture and the servers. What, how much load can they take? Based on that number, they'll give you a rate limit to use. So if it crosses that, they'll say, sorry, we can't handle the request because they need to have a resource estimation. So rate limits are also one thing to be taken care of and obviously filters. Yeah, so I'd like to talk about a bit more about authentication authorization here. So filters is here in the example. See, I want to see all the completed orders, right? I don't want, I'm not interested in all the orders. At the same time, I'm not in, I don't have the ID, specific ID. I'm not interested in just viewing a particular order. I just want to see all the completed orders. So the API library that you're relying on should have the support of filtering, right? Completed equal to true or all the orders of a particular user, something like that. So filters is an important thing to be taken care of. So authentication and authorization. Authentication and authorization, there are different levels of doing them. One thing, one simple thing is that do a global authentication. What do you mean by a global authentication? Let's say I have a service to offer as an API. I'll just give you a secret token. Whoever has that secret token in their request, I'll allow them to use my API, nothing else. I'm done with that. But how do you manage authorization or what are all the resources that the user has access to and what are all the resources that the user doesn't have access to with it, et cetera, have to be figured out? Most of you might have used Amazon's S3 or any APIs. What do they do? How do they give a user token? They'll have a public token and a secret key, a public key and a private key through which the API talks. So I'll not go into the details of token based and HitchMack based, it's just to give you a glimpse of what are all the available options of authentication that you can do. Actually, each one of them is suitable in different contexts because of the time I'm not going to go into the details. That's it, I wanted to just share that information with you. If you have any questions, are you in time? Yeah, questions, yeah, I'll take questions. Yeah, if you have any questions. Yeah, hi, my name is Krasanjit. I was just wondering since you are, all these data validations and all these things, would you recommend having a service layer between your ORM and your controllers in this case? Yes, absolutely. So the libraries which are out there, which does the automatic job of handling things, they don't have a layer, I mean they'll handle the layer but it's not under totally under your control. So I suggest to have a layer in between, yeah. Just a follow up question, with a lot of products right now, there are a lot of them do dashboarding. So how would you handle that with APIs, would you? Dashboarding, yeah, would you again? The analytics type of, I mean use the dashboarding also. So for example, you have an e-commerce website. Yep. Or someone's coming using your e-commerce website. Now the thing is that users, typically even if you use a mobile app or a web app, the first phase that you usually show the user is a dashboard saying that okay, you place these many orders. Yeah, yeah, yeah. These are the products that we recommend to you. Yes. And all that information gets aggregated in one space. Yes. While although you have different models for them. Yes, yes, yes. So it's a way of jacks. So you can show different pieces of the same page via different requests, right? So you can show the aggregations of, I mean just the orders via one request, one URL and other to show the aggregations, other to show their recommendations. Actually it's one page, but sending n requests. It's not just one single request. So you would not recommend having one single end point for the reaction. One single end point should do one single thing. Ma'am, what will be preferable way for retrieving multiple customer specified ID with specific ID? For example, I want to go five customer with specific ID. Five customers. Yeah. So the ID is not a primary key there. It's not a unique ID. No, but I want a specific ID. For example, customer one, customer two, customer three, and customer four. Yeah, so that is bulk, bulk getting or bulk posting. I didn't go into that, but you should be able to handle that with, maybe you can say customers ID is equal to one, two, three, four, five. It depends on your usage. So the API designer should take a call on whether it should be a list or comma separated values or how it should be. What will be preferable way, easy to comma or any separator like? Yeah, it is up to the designer, API designer to be using. I mean, commas are fine. Oh, yeah. Thanks. Mike. Hi. Yeah. Like, I'm a little bit confused, like get post and here I am. Where? Here, here, here. So I'm using get post and all these methods, like when I did a new REST API also. So when I'm designing an application, that means that REST was existing that time also, like everything that we do, like whenever we are creating a web app, get post and I have never used port. Yeah, that is HTTP, yes. So that is HTTP, get post, port and patch, they are all HTTP verbs that existed before you knew what is REST, right? But the point here is the URLs would have order slash edit, order slash add or something like that. So the action what you want to do is going through the URL rather than using the HTTP verb. HTTP is so rich that it offers the verbs to do different actions. So REST is the part of HTTP, it comes under that. No, no, no. HTTP has the verbs and the status codes, which are sophisticated enough to a web service to be offered as an API, meaning REST is sitting on top of the HTTP. So like the example you showed, like the filter you were using. Completed equal to. Anything we can use like page id equal to one. Yeah, yeah. To extract that. Yes. So that is a simple get request. Yes. Can also retrieve the same thing. So what is the difference between a get request and HTTP get request and REST API. So we are talking about web services, right? And exposing them as REST APIs. Finally, HTTP layer is there there. We are not, REST is not an alternative to HTTP. REST is built on top of HTTP. It uses the protocol. HTTP verbs, it uses the HTTP status codes. Nothing else. So we are using HTTP. The difference is the architecture. When you think of the API, you think of the resources first, the actions second, and model your application in terms of that. So it's not the actions which take over in the first place. The resources, what are all the resources you have in your application? That's the primary focus. And once you identify the resources, then you'll go on to actions. What can I do for each order? Or what can I do with the customer resource? What all he can do, et cetera, right? Yes? We have a question in the back. Suppose we are having two platforms. One is REST and another is non-REST. Now if, suppose we are working in a distributed environment or whatever, if the format is different, suppose one format is JSON and one is in XML, now the two applications should have something common. Suppose one wants to share a common API between REST and non-REST. Yeah. So there would be a layer sitting in between which can compare JSON and XML, maybe. What is that layer? One should develop that layer. Maybe if you are writing in Python, you would use JSON library and something to pass the XMLs and club together. If they are both REST, then even then, actually as a user, you'd have a layer, right? You won't directly consume the JSONs. You'd have a UI or something like that which will consume that. So there would be a layer finally. Then the OAM and everything should be same or it would be different. No, it could be different. It could be different. One may be MongoDB or one may be something else. Yes, that's the best part of having APIs. It'll let the client and server evolve independently of all the architectural decisions. This can be totally in CE or legacy or whatever it is and this could be in very, very new technology. This could be using new technology. Thanks a lot. So the links will let you navigate for the self. Self is identifying which resource that you have asked for. You have all the information. Let's say you're asking for orders, a particular order. You've got all the information, the data, which product that you have ordered and with which quantity and what date you have ordered. But in the UI, you might need the order ID also. Where would you get it from? It's better to have it, right? So the self is by default included in the links in Python. It depends on what framework that you are using. So Django has a set of libraries and Plask has its own set of libraries. There are libraries which are independent of the frameworks like Python Eve. There are many actually. But it depends primarily on your use case, whether it is a public API or a private API. Actually, when you are thinking of a public API, then the design is a bit different than a private or an in-house API. You would not care for authentication and et cetera, et cetera, but it's a private API, right? Hello, ma'am. Hello. Yeah. This is Django. Yes. Ma'am, recently, I was in the condition that where I have to use almost too much filter for getting the list. At that time, we have to use the post request for getting the list of the thing. So is it the... Because the URL is too long. Yes. So it is the rest friendly thing, or is there any solution for... Let it be. It's readable. Unless it's not readable. We're using the post request because we have to use post request. So is it... No, post is not meant for getting the resources. You should not use post unless you're modifying something at the server state. And is there any particular standard for the rest, or is it the common type of practice to follow? Sadly, sadly, there's no standard for rest. So there is something, a very good site that we have come across, jsonapi.org, which is trying to lay down a few set of standards, but it's not actually standardized. It's an ongoing process that suggests the URLs could be like this, the error handling could be like this, and stuff. It's better for you to just go through jsonapi.org. One last thing. Yep. For updating the record, you have suggested, but post is, I think, also can be used. Is it a good practice to use the post for updating the record? So it depends on how you're creating, actually. So you can see here, right? Here, we are not keeping slash one here. It is, the endpoint is slash customers. Yes. And if I do the same thing again, slash customers slash two will be created. So post. For a specific customized, the ID we provide. If you provide slash ID, it's going to be item potent. Yes. Then you should use put. Thank you. So if you don't, it should be post. Yep. A quick announcement. Then we'll take two last questions. So anyone who hasn't gotten the feedback form can take it from our volunteers. So, okay. Yeah, ma'am. One last question. I want to, yeah, yeah. Yeah. We'll do a lot of request and we'll get the response. So my question is like, are there any package we'll be able to advise? Because I'm basically looking for automation perspective. For example, if I do request for VM in place of cloud, 100 or 1000 VMs will be there. I'm interested for particular kind of VM. As you've shown customer name or whatever it is. For automation perspective, automation framework also, others have become big. Are there any package in case of Python? What do you suggest? The libraries. Exactly. That will help for automation perspective. That's what I mean to say. You're getting right. Yeah, yeah, yeah. Can we take this offline? Yeah, exactly. So that it can be described. I will attend that auditor in two sessions. Then I'll come. Sure. Yep. Others I'll mail you also. Thanks. Thank you. I would like to give ma'am a small gift from Python India for giving a wonderful talk.