 So hello everyone. My name is Jakub Różański and I came from Poland. I am currently hired at software engineer position at Intel Corporation in Gdańsk. Today I'm going to talk about application broker, which is a project aimed to handle custom applications in a service like manner. I have to admit that I am a little bit nervous because it is my first serious speech given in English. I will appreciate your understanding in case of some problems with my pronunciation or grammar. Ok, but it's not about me. It's about you getting some information on how to solve problems and what kind of problems. Let me just talk about it with providing you introductory story. Ok, so we have Steve, who is a developer for some company, in some company. And this company is aimed to provide product on top of multi-tenant cloud funder installation. And as you can see on the slide, both told Steve that he needs to place existing demo application in the CF marketplace. This is multi-tenant CF installation, so we are assuming that every single company that will finally use this cloud funder environment will need separate organization to be created. And another assumption we are going to make here is that end users are not so technical. So natural selection to choose is to use marketplace for end users just to go to the UI, click on tile and end up with new instance of application created. Ok, so Steve did simple research and he found out that he needs to implement separate service broker handling demo application. Are you guys familiar with notion of service broker already? Yeah, great. So what I have to admit, it is not initial intention of CF architects to mix notions of services and applications, but here the need is a father of thought. And we want to have service instance with backing up application in the background underneath. So we have here endpoints that we are going to implement to introduce new broker. So Steve is going to implement these endpoints. Endpoints are of course imposed by CF documentation. First endpoint is just to introduce broker offering. Next three endpoints are for managing instances and last two to create and delete bindings. Connecting together services and applications. Ok, finally Steve managed to come up with simplest solution. What he did, he just simply stitched together broker itself and application, demo application binary that will be managed. And assumption here is that they are going to be pushed as a single component to the CloudFunday instance. On the provisioning operation, our binary will be pushed again as a separate application in chosen organization and space. Ok, so we have this simple solution. It looks like we are done and Steve is happy with demo broker he implemented. But of course this is not end of our story. It's time that in which boss comes again and asks Steve for some other things to be implemented. And here we have new requirements. Steve needs to introduce some other application to CF marketplace. And what is important, list of them will change in time. To delete offerings and also introduce new ones in time. Of course you can also think about some other more complex requirements. For example pushing application with some dependencies. In opposite to this demo application, which was just one row application. Or we can also think even about pushing and wiring multiple applications together as a group and place it as a one offering in the CF marketplace. Ok, so in the second part we are going to talk about solution which is called application broker and was built at Intel. And it solves all problems Steve was facing in the introductory part. Ok, so we introduced extended API. Besides this standard broker API imposed by Cloud Foundry documentation. We are introducing three other endpoints to introduce, update and delete offerings. In the background we have MongoDB instance to hold to persist this information related to all offerings we are going to have. And now we are probably thinking about how to tell broker about handling this particular application we are going to handle. Are we intended just to mix together broker and binary of application to be handled? No. Are we intended to post whole binary in this first and second endpoint? No. We are just pushed separately broker and application to be handled. And then just register it using identifier of reference, so called reference application that are going to be handled. And application broker is going to care about it. Ok, on this slide what I am going to talk about it is that in application broker we come up with more complex scenarios covering also grouping of multiple applications. And as you can see here we have two applications wired with user provided service. And also other dependencies in form of service instances, MongoDB and ZooKeeper. Ok, reference applications that are going to be registered within Cloud Foundry Marketplace can be not even started to not use up resources. And of course as was told on the previous slide to append whole group of applications what you need to do is to just put identifier of application. In this case what we need to do it is to just place root application and all components will be concluded. So how to conclude all application within particular group? We found out that we need to separate concerns and introduce other microservices called discoverer and it is intended to conclude all components particular group is made of. Ok, discoverer is conducted from top to bottom with traverse bindings between all parts of the group. And also what we do we are assuring that no cycles within this particular graph occurs. Finally discoverer is able to return list of components particular group consists of. Ok, so now let's analyze flow diagram that shows what happens during simple service creation. So end user uses CLI to issue create service and then cloud controller is contacted. And finally application broker is informed that we are going to create new instance of service and of course application underneath. Discoverer concludes all components that we are going to create and informs application broker about it. And then we are just creating reference application, creating all dependencies for this reference application or bindings. And then application broker is informed that we are done and pass this information to cloud controller and finally user gets this information. Ok, we have this clone reference application operation. What might be interesting for you? It is that underneath we have copy bits operation. Fortunately cloud controller provides such operation for us. And what it did, what it do, it is just cloning bits binary of existing application into chosen one. Initially we just create empty application which is possible and then just put bits of reference application into this empty one. Ok, so this flow solves all problems Steve was facing. And now let me just talk about some other extra features we also implemented in application broker. Recently we implemented possibility to configure chosen parts of our complex offering. So when having this group of components on the provisioning operation we can just pass JSON object and choose which component will be populated with which value. We use namespaces to indicate what value goes to which component. As application broker is part of larger solution called trusted analytics platform we are also thinking about security. And communication going to application broker is secured just by basic authentication at this is proposed by cloud funder documentation. And in the opposite direction communication is protected by all in the second version. Typically we introduce just separate UA client with high privileges. We do so because we must clone, we must discover and also delete applications within all organization and spaces in the cloud funder installation. So admin rights are crucial here. Ok, so let me now proceed to demonstration. I prepared video to make it easier for me to go through practical examples. So let me just switch to it. So here we have UI of trusted analytics platform. I decided to show it on the UI because it will be easier. UI just covers all the functionalities that regular cloud funder has with some new features. And of course we can choose from the list of organizations to operate on. And previously I just prepared separate organization and space with application broker and discoverer. So here we have panel describing application broker application. And we have services bound. Updependency discoverer meaning in form of user provided service. And also instance of MongoDB I was talking about which holds information about service offerings this particular broker have. Now what I'm going to show is that we have also sample application pushed in source organization. And we are going to register this application, this simple application as a separate service within CF marketplace. It has some environment variables that will be also cloned on the provisioning operation. It has also fake instance dependency bound just to show you that underneath during provisioning also all bound services will be copied. This is just the instance of this MongoDB. And as you can see sample application is as simple as that. It just prints hello world text. It can be also configured to print hello and the name of somebody. And I will show you that while provisioning new instance we will be free to choose what the value of this text will be. Ok. Inicially we have also the destination organization and we are going to provision new instances of service and application underneath within this destination organization and space. As you can see we are going to request catalog endpoint and initially it is just empty response. And now we are going to extend it. I'm turning on logging and just by using UI we are going to register sample application within the marketplace. So I'm just typing name of the service, display name within the UI and also a simple description. And registration just happened. You can see in the logs that we have metadata and identifier of this particular application we are registering. And request is completed. And as you can see we have in the marketplace just new offering a sample app service. Inicially in destination organization it is not visible because we have to enable it within all organization in the cloud funder. So let me do so. And here we are. Sample app service is also available in destination org. Of course underneath in the database we have now sample service entry. And now what I'm going to do is to instantiate new service instance. And finally in the background new application will be created. I'm typing just username to show you that this new instance will come with also environment variable provided. And underneath magic happens. Discover discovered that this particular group consists of sample app and fake instance dependency. It informs application broker about it. And application broker is going to create clones. So here it requests copy bits. Also binding are created. And finally we are starting new application underneath. As you can see in the destination organization we have demo instance with some random prefix to avoid collision. And we are going to wait for this application to get up. So request is completed. And as you can see new instance of sample app prints on our hello drive app. Of course there is demo instance on the listing of services and also clone of fake instance dependency was created. And it is bound to our new instance of application. Ok. And what I'm going to do now is to delete this instance of service and underneath application will be also destroyed. Right. And you can see here that we are deleting application, deleting route, also unbinding and deleting application routes. So now instance of sample app service is gone, application also is gone and fake instance clone also is gone. So this is all for my sample demo. And let me just go back to slides. Ok. Let me just talk about possible enhancements. We have some limitation in the application broker. First to talk about this lack of asynchronous API implementation. So when coping with some large applications to be handled, we may end up with some timeouts. So to work around it, we can of course increase timeouts on the HA instances, HA proxy instances. But of course the final solution maybe to just implement asynchronous API. And the second thing to implement will be just extending list of credentials that will be provided by application broker on binding operation. Because now it just returns URL to application that is served underneath. You can of course think about any other opportunities to develop application broker. It was open source last year and I am happy to encourage you to take application broker and just use it. If you discover any places with some limitations, feel free to contribute or just place if you own GitHub. We built application broker using Go language intentionally because CFCLI is also written in Go. So possibly Cloud Foundry community will be able to also implement some features in the application broker. Ok. So let me just summarize all points I want you to remember. First of all we are able to make catalogue of broker offerings dynamic. Almost always brokers come up with static list of offerings that are hard to extend. But just by extending existing service broker API you can end up with dynamic version of catalogue. And what I want you to remember is that broker that managed application already exist. It is called application broker built by Intel developers and you are invited to implement new features and use it. Ok. Feel free to contact me in various ways. I will be pleased to hear from you. You can ask me any questions about application broker and about things covered in this presentation. And all feedback from you will be precious for me. Some acknowledgements related to the graphics I used here. And this is all I have for this presentation. So any questions? Ok. When you updated reference applications all new instances of services will be just clone of this new version. So you are able to update offering easily. But we are not touching already existing ones. I don't hear you. Are you considering mechanism to upgrade the existing ones? Say the existing one has a vulnerability but it's exposed as service instances. Can you think of ways to purchase that a requirement you've received already? So are you thinking about making purchase to existing instances, right? So I think right now in this current version of application broker you will just end up with doing this manually by hand. Because we have these applications running in the background which are available to also see within the CF installation. And you can of course push new binary on this particular instance of application. So currently it is just this option to choose. Any other questions? Ok. So that's all from me. Thank you very much for listening and for your time.