 Hi everyone, my name is Adriana Vassil. I work for British Sky Broadcasting in London and I'm here today to talk about hypermedia APIs, how we use them in Sky and what we've learned. So if you have any questions, I'm going to ask you to write them down and keep them until the end. I'm going to do my best to allow a few minutes for questions and also if you have any further questions, we can take it offline, feel free to approach me after the talk. So I'm going to start with introductions. I work on the Sky Sales platform department. We are a distributed platform that uses loads of components agnostic in terms of languages. So we have a few components built in Python and we also use Ruby, JavaScript, Java. So we interact with loads of services that are not in our control and we found it challenging to create a uniform interface for our clients. So we decided to implement hypermedia APIs at the orchestration there. We found it really useful to sort of tie together all the agnostic components and when you have loads of clients for your backend services and the workflow for the clients can be different, it's challenging if you don't have a solid platform. So we decided after using hypermedia APIs at the orchestration layer to extend it and using for shopping basket services. So basically when clients go on sky.com or they want to buy something from Sky, any of the talk, broadband or TV products and they call the call center, they interact with shopping basket. So there are loads of interactions in terms of adding, removing products, adding offers and so on. And we found it really useful to use hypermedia APIs at that level and also it allows us to expose all the products in the sky product catalog basically. Just to give you an idea of the challenges that we have, the sales platform was started more than two years ago and we're just celebrating two million orders last week. So there's loads of interactions, loads of traffic on our backend services especially and we have loads of challenges. So that's why I'm here today to basically share with you some of the challenges and some of the things that we've learned while using hypermedia APIs. How many of you are using at the moment hypermedia APIs? A few people. That's good because this talk is going to start with the beginning. So basically I'm going to say what hypermedia APIs are and then we can go into a practical example and show a very simple client-server interaction and hopefully after that you're going to have an idea of what the design is for this kind of APIs. So first thing is first, hypermedia APIs are based on REST, RESTful interfaces. I'm not going to insist on REST because this is not a talk about REST but it's very important to get across the fact that hypermedia APIs are supposed to be simple, uniform, are supposed to make full use of HTTP verbs, use mechanism like caching or compression and also use the concept of resources and updating resources in a uniform way. I'm not going to insist so much on REST because I assume that most of us already have used or tried to use some RESTful interfaces on our services. I'm going to take this farther and having in mind that hypermedia APIs are based on REST principles, I'm wondering what if an API would be as easy as a Google search. When you go on Google and you search for something as a human consumer of the web, all you have to do is know the Google domain. After knowing the Google domain and you do your search, you're presented with a list of options or links but you don't care where the websites or the applications that you're trying to look for are hosted. They can be on a server in Germany or they can be in the US. The URL itself is not very interesting from the human consumer perspective. So hypermedia API is based on the same principles that the consumers of your API, most probably client machines rather than human consumers are not going to care about the underlying implementation or the URL. I'm just going to care about the name of the links that they're trying to access. So we've seen that hypermedia API is based on REST and is trying to simulate a sort of Google search, let's say, or search on web. So saying that, hypermedia APIs add to the table the following. Hypermedia controls or what we call links, which is actually, hypermedia controls are the interactions that clients can have with your application. And a very important thing is in the RESTful world, we update resources, delete, create. We do all the crowd operation at a very granular level. So we interact with one or maximum two resources at a time. Hypermedia is about interacting with the whole application state. Just think about the users that go on the web and they want to change something. They really don't care what they're changing underneath. For them, they see the web as a whole. Or if something changes on the web, they continue to navigate us before. So hypermedia is all about hypermedia controls and changing the whole application state. And what I've mentioned earlier, when I used the Google example, hypermedia is all about giving meaning to your application, giving a semantic to your links. So based on the things that hypermedia adds to the table, you can pretty much see straight away some of the advantages, like loose coupling. Well, if your clients don't need to know all the URLs that you use in your implementation, that makes them loosely coupled straight away. So imagine the advantage of having mobile clients that actually tie to your applications. You don't need to deploy them all the time. So because it's out of your control, how user download the new versions from any repository, that's great. Another great advantage is the fact that if you think of the application state as a whole, you can model your business interaction as you see fit and give the clients the options to navigate towards their goal. So those are some of the main advantages. Obviously, it's designed to be simple and uniform because it's inspired by the web. It doesn't all have to be about remote procedure calls and message queues. It can be as easy as HTTP interactions. So now we've seen kind of where hypermedia APIs design goes. So I guess the best thing is to actually see an example and see a client-server interaction just so we can remember more about it. The example that you're going to see today is a Pet Shop. So as you can imagine, the application domain, it's pretty simple. You have a basket and you can add or remove pets to the basket. It's designed to be very simple just so you can sort of understand the client-server interaction and it's at a very, very tiny level, it's similar to something that we do in sky with the shopping basket. So hopefully you'll enjoy it. The technologies that we're going to use today are Flask, which you probably know, it's just a web framework that allows you to do routing in a very simple way, something lightweight. And then Dolgrain and REST Navigator. Dolgrain from the server perspective and REST Navigator for the client perspective are the modules that enable you to work with Hal, Jason, Hypermedia type. So you're probably wondering, what is Hal, Jason, Hypermedia type? Well, in Hypermedia, we said that we changed the whole application state. So when I send the response back to my clients, I'm going to send not only a data structure that represents my resource, which could be Jason, I chose Hal and Jason because this is what we use in sky at the moment, but besides your data format, you need to communicate to your clients where your links or your Hypermedia control are going to be and how the structure is going to be. So Hal, it's a very simplistic Hypermedia type that gives you that structure. And you're going to see in the example how it looks like and I'm going to explain. Let's have a look in the Pet Shop RESTful API. Let's start with a simple comparison. What do clients need to know in this scenario? Well, clients need to know quite a lot. They need to know the host name, obviously, but they also need to know the URL to create the basket, the URL to get a catalog of pets, a URL to add a pet or remove a pet to the basket or even the pet type so they know what to render in their front ends. So they need to know quite a lot and they need to hard code all those URLs that we advertise them while we develop our services. Maybe there is another way and this is what Hypermedia brings to the table. Let's just assume that this is what clients can see when you access the root URL of your application. They can see a Hal JSON response. So this is what I was talking about. Hal means Hypertext Application Language. So you can see a few properties which are completely arbitrary, name and description. So that would be the data that you want to communicate to the clients. And you can see the Hypermedia controls or the interactions under underscore links property. So as long as your clients understand how to parse JSON and they also understand how to relate to the Hal format, they know how to interpret the response. But it's great because all they need to know is the host name. After that, you can see the basket, the basket name. That's the semantic of the link or the name of the relationship. That is what your clients are going to know. In reality, even if your clients try not to couple to your API, they still need to know something about your domain. So if they work with a shopping basket application, they're pretty much going to know that you have a basket and you can add and remove items from it. So they can know only the basket name and they can start to consume the API. What happens in reality is if you use something like REST Navigator, which is a very simple client in Python for Hypermedia APIs, you can use the Hal Navigator class to basically you only need to know the host name of your API. You can create a Hal Navigator instance, which kind of gives you a view of the whole application. And then you can start and get the data, like the properties, name and description that you've seen in the Hal representation and then go farther and get the links. In this scenario, it's just the basket link. But if you go back just a second, the method on the basket, ahref is POST. So you can have a list of methods if applied, but in this scenario, you're just trying to create a basket. So developers, what would do? They would use a tool like Hal Browser, inspect your API, know that they need to do a POST, grab that link and then try to create the basket using the name basket. So if we go farther, how do you create the basket using the navigator? Well, we already grabbed the link basket and we apply create. It's a very simple wrapper in REST Navigator, which says make a POST to the link called basket. And after you create this basket, you fetch the data. In this scenario, I'm just returning a random ID, something that you would see from a MongoDB database. And then you can see the next interactions. Like, suddenly you have a new link, a pet link this time. So probably you can explore the catalog. So what client developers are going to do? They're going to use, let's say, Hal Browser, explore the API and says, yes, I've got an ID of the basket and I've got a pet link with a get on it. So I'm going to continue and grab the data on the pet. So it goes pretty much iteratively through the API. But so far, you have seen that the only data that the client needs to know is the entry point and then named relationships, the name of the links. If suddenly I decide that one of my links is suddenly not going to be served by one component, but it's going to move on a different host, my client won't care. They're going to blindly follow the link with the method that I give them. They're going to fetch the data and grab the next interaction links. That's pretty much how it happens. You can see that when you get the pet information, pet in this scenario actually represents the whole state of your application. The way that navigators go, they see application as a whole. They give you the data and the next interactions. You can see, for example, quantity zero. For the purposes of this demo, I created a very simple logic saying if the quantity is zero, you get an odd link. When you follow the odd link and you add the pet to the basket, then you get a remove link to decrease the quantity. So that's very simple logic just for the purposes of this example. But that represents the state of your basket. So clients can see that there are further links, so they can further explore the API and say, great, once I follow the pet link, I get the next interaction, odd. Well, this is great. Even if you want to add new relationships at runtime, you can because as long as your client is not going to use the name of your relationship, it's fine. For example, I can add a new link and say, add multiple pets or something like that. When the clients are ready, they're going to deploy and use the link. So that's the only time when they actually need to change. So with the odd link, they've seen it's a post. So it's kind of creating a pet instance resource and assign it to the basket. So REST Navigator in this scenario, it's again very simple. You grab the pet link. In this scenario, I have multiple pet links. But the great thing about it is that you can advertise new types of pets or products in your basket, and your clients would automatically know how to render them if they are data-driven. For example, you can just tell them the pet type, it can be cat, dog, whatever, is in the age ref or give them additional information and then can render it straight away and follow all the links. In this scenario, I just grabbed the odd link, add a pet to the basket, and increase the quantity of the basket with one. This means that my whole application changes, my whole state of the application. So it's very important how you define that. And obviously, you see that the next interaction that they can do is removed, but that's why I said it's so easy to define business-to-business interaction in your application because you can just guide your clients through the API. This is how the HAL JSON response is going to look like, and this is what the client developers are going to look at and understand how to discover the API. Obviously, you can use documentation, HAL supports something like content URIs if you want to discover documentation about links, but that's okay as long as you use a tool like HAL Browser and you inspect the API, it doesn't have to be very complicated. So this is the client perspective, so you're probably wondering how difficult it is to use Dograin and create the server-side interaction. So resources are used, resources are built using the class builder, builder on Dograin. It's very easy to set properties using set property. So for example, this is my basket resource and I just return a property, an ID, and I grab all the pets and just render, add some links to them, so it's pretty straightforward. You're probably wondering, this is getting really chatty. Like you have to go from one link to another, how do I manage that? Well, it depends on the interaction. If the client only needs specific information that you can make it as chatty as you want, it's gonna be really fast, but if they want everything all the time, then you can actually, HAL allows you and the Dograin module to embed the whole resource in the response of your resource, basically. So I could embed all the pets into the basket resource without adding links to them, but that depends about your client-server interaction. But as you've seen, even the client and server-side interaction is pretty much simple. And to be honest, in a large application, these things are going to repeat so much, it's gonna be easy to build your entire API, add constraints, and so on. The root on the server-side is very simply used, is very simple, created in Flask. You're probably wondering, what about four of fours and all the normal HTTP responses? How are you going to enforce the users to use your API as you want? Because even if you basically allow them to do different interactions, you still need to enforce. Well, the answer is you don't need to forget that you have to stick to the same REST principles as you would always do and enforce all the negative scenarios, just so users, even if they access the links as they want, they still get the appropriate response. Just like on the web, users can type pretty much anything. So that concludes the example, a very simple interaction that emphasizes how easy it is to build your APIs, both from client and server perspective. Obviously, most of the clients would be in front-end technologies, like our clients in Sky would be building AngularJS or Hyperagent. There is a hypermedia client, which is great. It's just the Python client was used for the purposes of this demo, and it's not yet complete. It's very new, but I think you get the point. So after going through a simple example, I'm just gonna do a short recap before we go into questions. Well, we talked about advantages and we've talked about challenges. It's basically quite difficult to design your API, not because you need a different infrastructure, not because you need something special. It's just a different frame of mind. We are all used to restful interfaces, interact with one resource at a time. Now you need to see the whole picture all the time, and that is quite challenging. But for us, it paid off really well because imagine if you have loads of components, your clients would need to know the endpoints for all these components to interact with them. Imagine going from QA to stage to production and clients need to change all their mappings. Well, they only need to know one entry point for our backend services and have a mapping for each of the environments and we deal with the rest because we create the links dynamically. So that paid off really well for us. In conclusion, we are still improving our APIs. We are applying new design techniques. There is a lot of documentation on the web that helped us and there are loads of advanced topics which you can research like profiling, adding metadata to your links, API forms, there are different kinds of design. So it's pretty much, it's great to do research on something so new basically. So before we go into questions, I'm just gonna leave you with a list of further reading. There are loads of books out there, Mike Admus, and I'm just gonna mention he's great. He works for Layer 7 API and he's done so much research on restful services and hypermedia APIs. It really helped me a lot for research and it helped us in sky. So, and also you have my contact details if you want to ask any further questions after this talk because we don't have too much time for questions. So I can take questions now. Thank you very much. Okay, first question. Thank you for that overview. Very interesting. Do you have a client for iOS and a client for Android that supports that just like hyper-agent or Angular? To be honest, I don't know about a client in iOS. I know that there are loads of clients in Ruby, Java and all the front end technologies, but I'm not sure if there is one. Yet, the one that I'm using is also very young client for the HAL interface. It's only like a few months old and because hyper-media starts to be used in loads of companies, probably it's gonna be one pretty soon but I don't know about one. And if I may, how do you communicate a schema for your basket to a client? Isn't that something that they still need to know? Yeah, so the schema for those messages, the idea for hyper-media APIs is to keep them as schema-less as possible to be quite forgiving just like on the web. The only schema that you actually communicate is the HAL response. As long as your clients accept the content type, HAL and JSON, they understand that they will find the interactions in the underscore links or they can find embedded resources or they know how to grab the properties. But hyper-media is not about defining a strict schema. So, but you must have a product ID when you buy something, right? Sorry? You must have, like, in your basket. Yeah. You gotta have a product name and other things. Yeah. How do clients discover these things? So, as long as they, as long as, for example, in the example, following the pet link would give you more information about what you actually bought. I haven't put loads of information because of the purposes of the demo but you would just send the data as JSON and most probably they would only need to know the type of the pet, let's say, in this scenario to render in the front end. But you send the data as properties on your HAL response. Feel free to chat after if you need more. Okay. On your last slide you had challenges and advanced topics and especially the advanced topics are really interesting. I just wonder if they are not challenges anymore, does that mean that you have a satisfactory solution for all of these? You just didn't have the time to talk about it or are there still challenges? No, it's just because this talk was supposed to be an introduction for what hypermedia APIs are and didn't have enough time to talk about them but would be really interesting to prepare some talks on the advanced topics, which to be honest, it's quite a lot to talk about. So, I just mentioned it in terms of pointers if you want to find out more about it. Yeah, two more questions. Okay. Sure. So, over here. Over here, on the left side, right side, whatever. So, how do you do filtering with this approach? So, we tried using hypermedia APIs, I'm here. Sorry, I was still looking there, sorry. We tried using a hypermedia API without using IDs and static route and so on. So, just exploring some things but we realized we still need IDs for something like filtering. Let's say you have containers and you have items in it and you want a query, give me items of container A, B and C. So, you would need the IDs of those containers, for example, right? Yeah. So, the way we work like in Sky, we had some sort of similar challenges. So, we sort of wanted to organize our containers or items in a sort of logical structure so that the clients say if those are products and those are offers, then I only need to know the IDs for those containers to be able to render the whole page. In reality, clients are still coupled to your domain. So, if you have business sort of resources as basket product offers, they will still need to know or if you have like in Sky, we have TV products and broadband products, they still need to know the high level knowledge about those containers but it's restructuring the API sometimes helps even though clients need to couple on certain things. You have to aim for clients to couple on the things that are not likely to change in your domain for a really long time. There, thanks for your talk, I really enjoyed it. Thank you. So, I'd be interested to know if you think that there are, if you found any limitations when using HAL or if there are many cases where your developers are creating clients that ignore the hypermedia approach and step outside of that in order to interact with the server. Yeah, that's a very good question. So, limitations of HAL. To be honest, as I mentioned in the beginning, HAL is one of the simple sort of hypermedia representations. There are others more complex like Collection JSON and ATOM. So, the limitations are that HAL doesn't support API forms. On the web, you have forms, for example, to submit information about something, about your application. So, HAL doesn't support that. Not even ATOM, which is a more complex media type, doesn't support API forms. Collection JSON started to implement it. So, those were one of the limitations. The other limitations were that there isn't a way to express metadata about your links. There is a thing called profiles, which I mentioned on the advanced section that allows you to say something about your content that is outside of your content. And that is one of the limitations of HAL. It doesn't have that. It does have documentation, though, in terms of you can expose content to your eyes to document your links, which is great. Those are two of the limitations. Thank you. Okay, thank you. Thank you, Adriana.