 Please welcome Nicola for his talk if REST APIs for humans Good morning. Thank you so first leader story two two years ago in 2012 it was at Europe I saw in Florence and there was a gave a talk about building REST APIs with flask that was a kind of a training a very long talk and There was a lot of interest about that project and the code that I showed it back then and people were asking if We were thinking about releasing that kind of application as an open source project and so if Which is the project of which that I'm going to show you today is basically the Offspring of that talk and so it's very cool for me to be here again in your python and presenting the result of that event so REST API for humans, I guess 100% of you knows that I stole this tagline from Kinect rights request, which is basically the client side of any python REST API and the reason why I'm doing this is because basically The idea behind this REST API framework is the same as request which is make things as simple as possible and so here it is Well, I will just keep about on these ones since you already heard from my chair speaker What I do for work for job? And what is the philosophy of even basically you have some data stored somewhere some data And you need the REST API to expose your data to some I don't know mobile client maybe or web Website or whether what have you and what you do is just a install leave and In a few hopeful hopefully in a few minutes you get working API For you it is powered by flask as I told you already MongoDB Redis for a few features and a few other things, but these are the big three Guys in town I say so And a very quick quick start so you get an idea of what working with this framework means How many of you are working with flask already or have an idea? Oh great So if you work with flask you can recognize this code is basically the quick start from the flask website The only difference is that you are Using even instead of flask disease because it basically is just a subclass of flask So everything you can do with flask you can do with you and this is Probably a good idea because I see people using Eve as a Yes as a REST API, but also as flask so they are using blueprints for example for adding new features to the API and stuff like that Then the other thing you need to do is you you need their launch script that we just saw and you need a settings file where you basically design your API and The idea here is like Django and other framework. You just have a text file and in this case We are giving a two endpoints to our APIs people and books as you see We aren't defining anything for these endpoints. So we are basically just saying hey I want two endpoints on my REST PIs and these endpoints are named people and books and then you just launch the API and your API is up and running and ready to to work for you For example, you can access the people endpoint you see that even if we didn't define anything for the endpoint and that we Did you notice we didn't define any any kind of database connection actually, but The API is working. Anyway, what you get here is a few meter data meter fields So items which is supposed to be the list of items from the people collection, which is empty, of course and the links is a Different meter field and we will cover it in a few minutes, but you can already guess what's up going on there Yeah, it's this age 80 a Yes, I don't know how to speak in English, but basically they are just the links to the API endpoints and you can if you want to you can write your client in a way that it can navigate these links and build the client API and The client UI based on these links you can turn this feature off if you want to Mm-hmm Okay, I'm just keeping it and let's Connect the database now Very simple, of course and then why we are here. Let's also define some Schema for our endpoint. So here what we are doing is Defining a few fields and we are using and we are defining some data types and some Validation rules so the name fields is a string it has a max length it is unique and Email field if we can it is a string of course, but we can also set a reg X for validation of this field don't use these are the exact same production because it sucks but just to give you an idea of What you can do and you can even if you look at the schema Key word down there You can even nest sub dictionary with it within dictionaries list within dictionaries and least whatever you have you and Then we can by default an API is read-only, but of course you can change that in this case we are Enabling writing to the API endpoint. We are also allowing Edit of the items replacing the items and deleting the items so you have to do this explicitly otherwise the API then point will be read-only this is of course for safety reasons and Yeah, a few more Toys just to show you what you can do you can set cache control on that point additional lookups a lot of stuff and so We defined our API endpoint We We brought that launch script with a few lines and what do we get from this for this? Well, first of all you have filters for example your clients can query the endpoints. They can do that using a mongo Syntax of source so here you have an example where we are querying the people endpoint for the last name though But you can also use a Python syntax if you prefer This is because maybe if your clients is being writing right wrote by you You can use mongo and there is no big deal But for example if you expose your API to a website or to people you actually using it They don't know anything about mongo. Maybe you prefer to use a different syntax so you can do that You can have a sorting on your endpoint in this case. We are starting when the sending order You can use pagination. It is enabled by default. So for example here We are asking give me page 2 and only 20 result at maximum Projection and this is very nice. So you can say you have a document win with I don't know 50 fears you can say don't give me these fears in this request Back to the client because I want to save on bandwidth or on performance for example in this case. We are Telling the API don't send me the pictures for example don't send me the avatar because I don't need it And here we are doing the contrary only return me last name for example Which is very handy if you are using you're writing a mobile application for example You want to optimize the traffic and the the data being sent on the wire Another very cool feature is embedded resources. So basically here we will see an example here We have we are asking to embed the author field. Let's see what does it mean by default when you Get a document you will get for the author field. It's foreign key for maybe another endpoint This is what you would get by default But if you send a request with the embedded keyword What you get is an embedded document with the the fuel author. This is again To avoid sending true requests For the for the data that you need on your client By default your API will support both G is on an XML and Here you have an example of Resources with G is on you know that very well by the way all the field names For the meter field names are configurable by you so you can change whatever you see here to suit your needs And this is the same resource in XML We already saw hyper media of the engine of application state at work Let's quick look here just to have an idea You get the link to the same item to the parent item next page if the pagination is enabled and you have more pages you get a link to the next page and Even a link to the last page of course Again all these features are enabled by default, but you can switch them off For example, you don't want us to support a XML switch it off. You don't want a hose You you turn it off, etc. You can customize the API however you want document version is something that we just released with The last release and it is basically Git for the documents if you allow me and what you can you do when you switch this on basically you get versioning for your documents This feature has been contributed by a space x engineer actually, so I am very proud of that and you see here we are asking for version three of you over one document or Give me all the versions of the documents or you can even ask for the differ the diffs or the between the documents Maybe this is not something that everybody needs, but it's very cool to have it at hand file storage you can store files in the API since by default eve is Supported by mong is using mongo. We are storing in grid fs, which is basically Optimized storage for files in MongoDB. How many of you are using mongo or think about think about using mongo in the future? Okay, not okay quite a good number. We will see that there is also a secret alchemy branch forever later, so keep your hopes high and Here an example of how you do storage of file when you define define your endpoint We saw earlier that you can define define a string type, but you can also define a media type and Then when you send your data, what do you do is just use a multi-part data format? data format sorry post and you send your pick along with the other fields of your document your picture and When you get that document back you get the picture as a basis before string And you can also Enable these extended media info setting which basically is going to give you not only the file itself But also the extended the media the meta data about this field So for example content type and name of the file size, etc Again, you can disable enable first orders however you wish Rate limiting this is powered by radius what you can do here is Set the number of requests that a single client is allowed to perform on your single endpoint per minute or I shouldn't say per minute, but Per time window here. We have an example where we are setting the get method method limit at one request per minute window so you can have different limits per method and Different time windows as well for every single endpoint so the first get gets back is is Answered by the API and in the header section. So you get information about rate limiting. So you only have You did there are there is one request allowed on this endpoint per minute You have zero remaining and the next reset of the time window it is at that time point in time The second request within the same minute will get a two to nine too many requests This is just to give you an example of how this works. This is supposed to work This is of course useful if you want if you have Performance issues or if you want to avoid the your API getting hammered by some client Maybe a buggy client or somebody trying to do some kind of weird thing or attack on your API conditional request So the client can send a request using the if modify since header and for example Say, please return Return me the data from this endpoint only if it has changed since so We don't get back always the same the same I don't know in the people endpoint example as we saw earlier. I When I get back the first result set this the next request I can get back new data only and on all the the data load If none match is similar But we are using it tags here. So this is mostly used on the item endpoints So not on the people endpoint, but on the single person and point and for what it does is the same things Basically, give me the personal if it is something has changed on it There is also support for bug inserts So you can send multiple documents on the API endpoint With a single request here We are sending two ready documents for example and when you get back as a response is an array of responses actually because validation is performed on every single document and There are two me two Options here you can switch a coerence model off which is the default in this case You are only going to back the meta data back with they will be useful for Sending a subsequent recast later and stuff like that or you can Basically say every time send me back the world document Included the meta fields. This is something new. We that was just added to the APS to the feature set quickly on that integrity concurrency control you basically we are basically using it acts for That integrity checks So when you try to modify the document if you don't provide that if match header you aren't going to get that patch in In fact, you are going to better to get a four or three back If you send any tag, but this e-tag is not matching the e-tag on the server You are going to get a precondition failed error if you give me the E-tag which matches matches the document on the server then the edit will go in Why this is it because we want to avoid a client with an old version of the document Overwriting a newer version of the document on the server. So only the client who already Knows about the latest release of the document can update it validation of course in this we have here we have a response example of Bulk insert where the first document got an error and Clinton is not unique, but the second document was accepted There is support for authentication Authorization so basic token authentication age Mac, which is basically what Amazon S3 is using and on their platform Ever answer on all kind of Python sir 3.4 and pi pi included and There is a lot of more stuff that we don't have time to go over that Versioning is here versioning it means API versioning and non-document versioning so you can have basically your end point Version 1 version 2 version 3 of the your API It's BS delay censored open source. You can do whatever you want with it. You don't mean any money and Okay, what we saw So far is what you get for free without any line of coding you just have to Pull the switches on and off you Have this feature as we turning on and off, but what about developers if what how can I customize my API here I have a few example for example, you can have custom data layers so in this case we what we see is the code from the SQL alchemy branch, which is a working project in progress What do you do is basically a subclass the base data layer? And and then you go off and write your own data layer for example we have an extension which is called if elastic and is using elastic search and Here we have a sequel alchemy under whatever you want to use This is an example of the sequel alchemy by the way here you see that in this approach what you do is using sequel alchemy classes and you just register the schema so you don't have to write it the Resource schema in the settings file because a sequel alchemy is already providing you the the classes with the Basically the implementation of the your endpoint Here is an example of the elastic search data layer and the MongoDB is doing exactly the same. It's just a subclass in the basic data layer Authentication this is where You actually have to do some work because you need to Subclass the base class here and provide the authentication logic by yourself. This is of course because This is something you want to be in total control of You can do a lot of stuff with authentication. You can lock the wall API You can lock only certain endpoints and leave other endpoints open to The public or read only write only read and write whatever role-based access control There is a lot of stuff here, but we have to look at it. Just three steps Host tutorial to give you an idea of how you do this you basically just import your class basic oath class you Overwrite this check-off method here. We are basically saying hey Whatever Request comes to this endpoint with username and mean then password secret let it go if it is good to go Otherwise, we will send back a not allow response and then what you do is when you Create your instance you just pass your custom class to the to evil and that's it your API is now protected Of course in this case. We are just setting this Protection for the wall API all the API endpoints, but you can actually change your class forever for Every single endpoint or as I said before you can even leave some endpoints without protection and other to with and That's all you that you need Custom validation you can add the custom data types custom validation logic if you need to and That is very nice because for example in the next release We will add the support for geog zone and so we will have a point a multi-point polygon polygons and all this kind of stuff And then we have even hooks. This is Sorry if I'm going very quickly on this but with the time is short on us So this is very nice because when something is going to happen on your API What you can hook callback function on basically every event so here for you see that you can Set add a callback function every time an item is it being inserted for example Or after it is being inserted the same happens with get patch put delete and whatever have you there is a Simple example here what we are doing in this example is updates the documents that the client is sending us with a new field So you just define your callback function here We are and you see that the functions is getting the resource and basically resources the endpoint and documents is the collection of documents beans that are going to be inserted in the MongoDB and what I'm doing here is Just adding a new field or over writing this field if it if it exists I gave this presentation and force them in brucell in fiber disease why there is for them and I didn't have the time to update it and And then when you are about to launch your API you just hook your function to the callback as you can see down below and Then you have custom file storage as I said before we store on grid FS by default But you can change it to to whatever you want. There is a guy who did an s3 class for example, so he's a Instance is storing the data on s3 on Amazon s3 or you can store on file systems Whatever you want and then there is the community just a few words on it There are a few extensions available already released by the community for example. If dogs is a this is a very cool project Actually, it generates documentation for your API's and what it does is this you get a Docs endpoint and when people access that endpoint instead of getting a json or XML they are getting an HTML page with the documentation of the API and It is actively maintained and there are a lot of contributors on this project And what basically gives you an automated documentation for the API? if mongo engine this is a Basically a connector between Eve and mongo engine if anybody you're using a mongo engine You can do what we have seen with SQL alchemy with this ORM for mongo You elastic we told about we saw it already if mocker is a mocking tool for Eve and Then I the the thing that matters to me is that the community about Eve is quite He's starting to grow quite a bit At the moment we have about 50 contributors to the project But what I really looking for is more contributors joining the project So if you are interesting in this kind of stuff you do know that you can actually contribute to the project as it is on GitHub of course and that we have a few tickets open and what I'm specifically looking for is People willing to work on the SQL alchemy Branch because it is now feature complete it is but before merging it what I need these people Actually wanting to work on the SQL alchemy branch even after it is it's been released I don't want to merge the the branch and then I have people complaining because something doesn't work I don't I don't want to look at it personally because it's not my kind of job. I'm doing something else So if you want to join a nice project and you're interested, please do so We are working on give Jason for the next release. So you you will you will be able to define a Geo data point or geo data point and Validation will work. You can do queries on these kind of stuff Etc. G is on P and if you a custom render classes a lot of stuff is coming up in the Is on the pipeline basically This is the URL for the project. So you can go there read the documentation Get to the GitHub repository and see the changelog Get in touch with me or or my Twitter account, of course You can get in touch with me even at this account if you go to get up at Nicolai a github dot slash Nicolai Roger you find the source code for the project and if you go on Twitter and You can follow me and I usually use Twitter to update on the mergers and the newcomers coming in and commenting and sometimes even I Don't know like Complaining about stuff and like the stuff like that. But well, if you want you can follow me on Twitter I'm basically that's it. I wanted to give you a little demo, but I don't think we have time for Yeah, I figure so if anybody wants to see something working just one thing There is there is actually a I'll cheat here There is basically an online demo of an API. It's With you can consume with your clients or with even with just chrome for example this is postman, but if you go to If demo rockwap.com and the slash people what you get back is a mix ML because Chrome is either casting is ML data, but basically you can consume a API if you play with it to send the get put request and stuff like that and play and see whatever what for example here, I Asking for the people and point with Where people has have a white last name and what I'm doing here is using the actually the hero cool application in remote so you can play with it basically and and get First-hand experience of the API. Thank you very much Okay, thank you. Nicola. We have a little bit time left for questions. So please raise your hand and I come with the microphone Thanks for the doc looks really interesting. Thank you. Do you have any support for testing your STP? Testing. Yeah What do you mean by testing? I mean there is a huge test suite on the on the repo So every feature is being tested when every on every committee you submit today to the To the source code actually So what I mean is? It's easy to create the rest API, but what if I want to have some tests for it For my for my API that they built with your framework like something for that Yeah, it is easy to do because you you can basically even again is a flask application It's flask so whatever you can do with flask Which means that you can do test very easily on your own API actually you can also use our tests To see how it's done is done and then implement your own test on your own API very easy to do The test suite is something of which I'm kind of you know I miss it feelings about it. It's I'm kind of proud because there are I think 500 tests and I'm testing everything but it is in need of some refactoring actually So if you that's an ultra another area when you could think about joining the project. Thank you Go ahead Thanks for your project. Do you also support URLs instead of IDs? For example, you showed us the book and the author of the ID. We have a URL on 790. Yes What you can do it didn't show here is you can actually have nested the URLs. So for example, you can have cities city ID slash People slash people ID for example, so the person in the in the city This is not a very good example actually, but you get the idea and that you can design your own a URLs Since again, it is a flask application. So you can you can even define An additional URL for the same endpoint for example, if you don't like the ID Which is a row number of course you can define a new URL based on the last name for example, so API comm slash Smith will get your Person with the last name of Smith. So yes, you can play with the URL quite a bit quite a bit Okay, thanks a very thank you again Nicola. Thank you guys