 Welcome. I'm Micah Silverman and I'm one of Okta's senior developer advocates. Today I'm going to be talking to you about how to build a Heroku add-on. Now if you're not familiar with Heroku, Heroku is a hosted application service that supports various languages and various frameworks including Java and Spring Boot, which is where I'm going to be focusing today. And Heroku add-on is an additional service that you can add to your applications. Now if you're a service provider you may want to build your very own add-on and Heroku makes it easy to go about doing that. Now just off the bat I'll show you what I mean by this. I'm going to create a new app and I'll call it a fit nerd add-on test. So I can create an app in Heroku. Now there's no code yet, there's nothing to deploy. But Heroku has this notion of add-ons. I can come over here and click on configure add-ons and probably the most common one is MySQL or Postgres. Adding a database to your application is pretty common. And once you select one you can then choose a plan name in this case free and provision it. And so now this application of fit nerd add-on test now has MySQL capabilities. And if we go over here to settings you'll see that it's got this JAWSDB URL set and that wasn't there before that was configured by virtue of adding this add-on. Now you can have any number of add-ons and we're going to work with this a fit nerd add-on test application. And we're going to build our own add-on and see what that looks like in practice. So where to start is devcenter.heroku.com articles building an add-on and they have a great set of documentation here. And the important first step is going to add-ons-next-heroku.com to accept the partner terms of service. That page looks like this. You'll click join provider program. And once you do that and accept their terms of service you'll be able to start creating and adding add-ons. Now this is what my partner portal page looks like. You can see that I've created some add-ons. I have octa and octa staging. That's the octa add-on. I'm going to get started in creating a brand new add-on and we'll build the code that's required to work with it. But first we need to do a little bit of housekeeping. There's a Heroku admin plugin that you need to add to the Heroku command line interface. Their documentation goes through setting that up. It's pretty straightforward. But once you do that you're going to use this add-ons command with sub-commands. And the first thing we need to do is to generate a manifest. And I'm going to give it a name here and I'll call it a fit nerd Heroku add-on staging. Now it's going to ask me some additional information. I'm going to give it a slug name. I'll keep that same name and it wants a display name. So we'll call this the add-on staging. And then the regions that we want to support will leave it at the default with us. And now we can generate this manifest file that includes some secrets in it. So now we have the manifest and it has these elements to it. And this manifest file needs to be carefully managed because it includes some secrets. It includes a password and it includes assault value that's used for SSO, which I'll talk about in another video. So that's what gets us started in creating a Heroku add-on. The next step is we're going to look at the initial requirements of our add-on from the perspective of code. And that is we need to implement three endpoints and to support the ability to receive a Heroku request and exchange an authorization code for access tokens so that we can interact with the Heroku API. Setting up our own API and securing it to be able to interact with Heroku lends itself pretty naturally to a Spring Boot application. And so that's what I've set up here and you can see that there are three primary dependencies. We have Spring Boot Security, Spring Boot Web, and we have here this Apache HTTP Components Fluent Library. And that's just a layer on top of the Apache HTTP client that makes it nice and easy and modern to interact with HTTP calls. We'll see that in action in just a little bit. Now what I did to organize this project is I set up a number of branches here and we're going to go through them to iterate to the point that we have a fully functioning Heroku add-on. So we want to get started as quick as possible and we want to do some rapid prototyping and so this add-on initial brings us to the point of being able to have Heroku interact with our application even though it's not doing anything yet. So I'm going to check out that branch and now if we jump back to the code, it's pretty simple right now. Let's take a look at what's going on here. First of all by default with Spring Security it automatically locks down every endpoint. Now initially we don't want to lock down our endpoints because we're just prototyping here and so basically I'm telling it to disable CSRF and to permit any request. So we're effectively disabling Spring Security but we're going to come back here as we iterate and update this configuration. Now the other file that we start with is this Heroku controller and I've defined this top level endpoint slash Heroku API so all of our endpoints are going to start with Heroku API and then I have this resources URI called slash resources. That's what Heroku is going to be looking for from our manifest file and so I have a post mapping, a put mapping and a delete mapping to handle the different operations that we have to implement for the Heroku add-on. So post mapping is the provisioning step. Put mapping is used to change plans from like a free to a paid plan and you can see that initially the response that we get is that plan change is not supported yet and then deleting when we get the delete mapping that is to deprovision our application and all it's doing right now is saying that delete isn't supported yet. What it is doing is first of all it's logging the incoming request from Heroku and initially we're just setting up both our requests and our responses as maps. It's a quick and dirty way to map JSON to Java objects and back and forth. They're always going to be maps. It's a key is the string and the value can be any type of object. Eventually we're going to layer in more formal models as plain old Java objects to make this a little more civilized to work with but starting with maps is a great way to rapidly prototype. So we expect to get a map as our request and we will return a map as our response and the built-in Jackson Mapper for Spring Boot will do all the heavy lifting of serializing and deserializing for us. So the first thing we do is we're going to log out that incoming request and I'm using the Jackson Mapper here to pretty print it so we'll see it in the log and then sending back a response that Heroku expects. I'm spitting back the request UID that Heroku sent to us. I'm spitting back a message saying that our add-on is created and I'm setting some configuration and this is similar to the MySQL configuration that we saw earlier. We can have any number of key value pairs that will end up becoming environment variables for this add-on in the application that the add-on is added to. Now I'm going to fire this up. This works like a regular old Spring Boot app. I'm going to start it up in debug mode. That'll take just a moment. Now we're listening on port 8080 locally on my machine and we want to provide a way for Heroku to access this locally running instance. And one of the easiest ways to do that is to use Engrock. Now if you're not familiar with Engrock, it's super easy to install as a command line tool but what that does is it sets up a proxy for us and I now have this routable Engrock endpoint and what I can do here is update my manifest to use that Engrock endpoint. So I've updated the manifest. I now need to issue a command to tell Heroku to use this Engrock endpoint that's in the manifest. That's the base URL and so I'm going to do Heroku add-ons, admin, push, manifest push. So Heroku add-ons, admin, manifest, push. This is going to push the latest version of the add-on manifest. And so now Heroku is configured to hit this endpoint once we start provisioning, once we start working with this add-on. So let's do something simple. Let's add the add-on to this application. To do that we're going to say Heroku add-ons create and we'll give it our name. And so that is a fit nerd staging. And so now we're going to add this add-on to this application. And I forgot to give it an app, so let me do that. Okay, the first time I ran this I forgot to give it an app name to add this add-on to. So we're going to give it the a fit nerd add-on test app. So we're telling it that we want the a fit nerd add-on, oops, got to do a double dash there for the app. We're telling it that we want the a fit nerd add-on test to have the a fit nerd staging add-on set up for it. Now here's the feedback that we get. It says that it's created it. It tells us it gives us the message. This is what I sent back. The add-on has been created and is available. And also it tells us that it sets some variables in a fit nerd underscore var one. And if I now look at this app, the setting for this app, a fit nerd add-on test. I can see that it has this a fit nerd underscore var one with the value val one. Now let's unpack what happened here behind the scenes. If we go back here, first of all, we can look at our output and we can see this is the incoming Heroku request. So this is the JSON that Heroku sent over to us. And it has things like a callback URL. We're going to use that later. That's for when you have an add-on that takes a while to provision. You may need to set that up async and then use the callback URL to indicate to Heroku that you're done. It has the plan. It has region information. Importantly, it has this OAuth authorization code. And in just a little bit, we're going to exchange that code for a token as part of a standard OAuth flow. And that'll allow us to interact more deeply with the Heroku API. All right, so we get all that. And if you notice what I'm returning for the config is var one. So you might be curious to know why it comes out as a fit nerd underscore var one. I just said it as var one. Well, if we look at the add-on manifest. It includes a config vars prefix that it's going to put in front of any variables that we set. So that's where the a fit nerd underscore comes from. So what we've accomplished so far is getting Heroku to talk to our spring boot application and to actually do some activity and actions to set up our add-on. And in fact, if I go to add-on test, I can see that a fit nerd staging is set up for this app. And if I go to this app's settings, I can see that our a fit nerd var one value is set. So not a bad accomplishment for a first pass at this application. All right, I'm going to kill the application and let's go check out another branch. And I'm going to say get check out basic off. Okay, and what I want to do is I'm going to remove this add-on from the application. Now I need to have my app running for this. So I'm going to remove the add-on from the application. And as long as we send back the 204 no content response, then Heroku will consider that to be successful. Even though we're logging here, that delete is not yet supported. Now as we iterate here, I want to be able to add and remove this add-on so I can test out new features and functionality. So before we can continue, we're going to remove this add-on. So we're going to say Heroku destroy add-ons destroy. And we want to give it the add-on that we're getting rid of. This is a fit nerd staging. And we need to give it the app. So this is a fit nerd add-on test. And we're going to confirm it on the command line so that we don't have to input it again. Add-on test. Okay, so now as far as Heroku is concerned, we've removed the add-on from this app. And if we go back and look at our log here, we can see the delete request. And it gives us the UUID. And so we say the delete function isn't supported yet. Now in real life, when we provision this add-on, we would save this UUID probably to a database with whatever information we needed to. And then when the delete request comes in, we can look up the information from the database. And maybe we remove that record from the database. That's how we go about deleting it. It's up to your use case and what functionality your add-on does. Now let's take a look at the difference between these two versions, between these two branches that I just checked out here. I'm going to do a get diff. And we'll say the difference we want is between the add-on initial and basic. Now we can see what we've added here. We'll look at this code in a minute, but everything in green is what we've added. We've added some environment variables for Heroku ID and Heroku password. And we've configured our spring security to use this Heroku ID and Heroku password to set that internally as our in-memory authentication. And now we get rid of permit all. And now we're saying that all requests must be authenticated and use HTTP basic. Now this is handy because Heroku automatically sends in basic authentication when it makes its request. And so now we're requiring authentication. And so if we go back and look at our code, really the only thing that's changed here is our security config. And now we're enforcing basic authentication. So for instance, if I look here, all this endpoint requires is a resource ID. So now if I try to hit that resource directly, I'm going to use HTTP. It's a handy command line modern replacement for curl. It works well with JSON. So I'm going to say delete and I'm going to call 8080. That's our locally running server. And I have Heroku resources and I need to give it some sort of identifier. That's what that endpoint is defined as. Now you'll notice we get back a 401 unauthorized because it must include authorization in order to work properly because of this configuration change that we made. Now this is kind of an ice little tricky bit of code here. But what we're doing is we're taking the Heroku ID and Heroku password, which Heroku gave us in the manifest. And we're setting that as our user and we're saying this is a global configuration. So we're identifying we're creating this global user. And then we're saying that all requests must be must be authenticated. So if we pass in the proper HTTP basic header with this quote unquote user Heroku ID and Heroku password, then it'll allow us to hit the endpoint. Let's take a look at that in action. So if I look at the manifest, remember I said earlier that the manifest has secrets. That's why we don't want to leak this. I have the ID as a fit nerd staging and I have the password is right there in the manifest. So let me go back and try and make this request only this time. I'm going to give it some basic authentic authentication of a fit nerd staging and our password. Now this time we get back the 204 response. And so as far as Heroku is concerned, this is all going to work now. Now I still have Ngrok running so I don't need to update my manifest or make any of those other changes. But I do want to verify that my Heroku request for creating the add-on and for destroying the add-on still works. So let's go back to Heroku add-ons create. I'm going to give it the same add-on a fit nerd staging and I'm going to give it the same app a fit nerd add-on test. And that completed successfully and we can go back and look at the output here and we can see that this is the request that came in from Heroku. So Heroku is still speaking to our app and now I can delete it just like before to get us set up for the next iteration of the code. So now it's deleted and we'll get the response saying that this request was received for this resource and delete isn't supported yet. Let's take a look at the next iteration of our code which is to exchange Heroku's authorization code for an access token. And we need that in order to interact further with Heroku's API. So I'm going to stop my server once again and now this time I'm going to check out the code exchange branch. And let's do a diff between basic auth and code exchange to see what changed. So first thing to notice here is that I've added a service called Heroku communicator service. That's going to be our central service for any time we need to make an API call to Heroku. And I've also auto-wired it into my controller. Then we have the code itself up here which is where we've added additional code here and it's going to extract the authorization code from our request. And then it's going to use the communicator service to exchange that code for tokens. Now one thing I want to highlight here is that while working with maps is a quick and dirty way to deal with JSON, it gets out of control pretty quickly when you see a line of code like this. We're getting information out of the map, we're casting it. Then we're getting more information out of the map and casting that. So pretty soon we're going to need to start working with models. In our next revision we're going to do that. But here's the core of the code. We're getting a token response which again initially is just going to be a map. And we're calling the exchange code for tokens. That's the net of what we're doing here. And then we're pretty printing out that token response. So let's go take a look at the code now. And if we jump over to our controller, so now we have security that Heroku provides. And so our service, our endpoint is now locked down. Now, if I look over here, these lines of code are the new lines of code where we're exchanging the code that we get from Heroku, the authorization code for tokens. Let's take a look at this new Heroku communicator service. Now it gets configured with a Heroku client secret. We need that in order to do the exchange. And then we're using the Apache Fluent HTTP client, which I like because I think it's very readable. We're posting to an endpoint called the Heroku token URL endpoint. We're sending it a form and this form has some elements to it. It's got a grant type, authorization code. It's got the code itself that we received from the initial request and it's got the client secret. We execute that HTTP call. We get the response and then we use the Jackson mapper to convert that content that we get back into a map, a Java object. In this case, a map. Now, where does the client secret come from? We need to configure that in our add on application and we can see that over here. If we go and look at the our partner portal and we go to a fit nerd staging, which we created with the manifest. And we go to look at the settings for that. We scroll down a bit. This is the client secret that's provided to us from Heroku. And I can regenerate that client secret, which I will do at the end of this video so that you don't have that information because it's meant to be secret. So we can figure the app with this secret so that it can make the call. And all we're doing right now, we're not really doing anything yet with the token. We'll do that when we get to converting this to being async will actually make use of the token, but we're developing iteratively here. So now we have a way to exchange the code for the token. And if you're not familiar with this OAuth authorization code flow, that's kind of outside of the scope of this video. But I'll make sure that we provide lots of links and references for that. All right, let's fire up our app again now with this Heroku communicator service ready to be used. All right, and once again, I'm going to use my add ons create command to add the add on to the application. Okay, now nothing different here just like before we get the same output. Now I can see that I got this token response and you might notice all this invalid cookie header. Heroku is sending over a bunch of cookies, which we are essentially ignoring. I'm not sure why that is invalid exactly, but it doesn't impact our application. But we can see that we were successfully able to exchange the code for tokens because now we have an access token, and we have a refresh token that we can use to interact further with the Heroku API. Okay, in our next iteration of the code, we're going to convert our maps to models so that we no longer have ugly gnarly code like this where we have to do a ton of casting on our maps. So before I kill the server and start over, I'm going to first remove our add on. And now I'm going to stop this application. And I'm going to check out our next branch, which is the models branch continuing on in our iterative approach here. Now if I do a diff on the difference between code exchange and models, what I can see here is that this big ugly gnarly code now turns into we no longer have a map for our request. We have this thing called a Heroku provision request, and we no longer have this map for our token response. We have a Heroku token response. And so now we get rid of all of this casting, and we simply call the get oauth grant get code method. And now we have an object of pojo model to use instead of the map. So this iteration, this branch doesn't really have anything to do with the Heroku add on system. This is more of a pure Java concern in our iterative approach here, where we're going to make our code better by adding in these models. If we look at the Heroku provision request, it takes advantage of some of our JSON property annotations. So we have callback underscore URL, and we want to use regular old Java naming conventions. And so the JSON property annotation allows us to do that. We can see that within it, it's got additional embedded objects. This one's called oauth grant. And the definition for oauth grant is just an embedded class within this model. Now that alters our code to make it a lot easier and readable to work with. So now instead of all of that casting, I have this simple call to request dot get oauth grant dot get code. Let's start up the application again, and we'll just validate that the models version of our application is now working. And just to be clear, this isn't a full conversion yet. You'll notice from our provision method we're still returning a map, but it's an iterative approach here where we've now refined the code to make it a little more readable, and we'll continue to do that. So I'm going to use add-ons create at each step of the way. We're just making sure that our code is still behaving the way we expect it to. I can see that I still get the output that I expect. I have my token output, and I have my request output. But now we're dealing with these objects. We have our Heroku provision request, and we have our Heroku token response. The last iteration of our code refactoring that we're going to do is to make this async. Now this is very common when dealing with Heroku add-ons. It may require some time for whatever provisioning you need to do. And after we make this code change, I'll show you a real-world example with the Okta Heroku add-on. And because we're provisioning an Okta org, it can take some time, and therefore we want our application to return immediately when it receives that request from Heroku. But then we want it to continue doing the provisioning in the background to communicate with Heroku to indicate that it's completed its task. And the way that it's going to do that is by using that token that we set up two iterations ago. So just like before, I'm going to start by destroying this add-on. Do a little bit of housekeeping with Heroku. I'm going to stop the application. And now I'm going to check out our last branch, which is async. Now there is quite a bit of difference between our models branch and async because in addition to setting things up to be async, I completed the work of swapping out all of our maps for models. But let's just take a look at some of it. Now I added in a Heroku provision service, which is distinct from the Heroku communicator service. The Heroku communicator service's job is to make HTTP requests of Heroku, whereas the Heroku provision service's job is to manage the flow of the provisioning itself. So it's a subtle distinction. We'll see how that changes. The other thing that's important is now the response type, the response status by default in a Spring Boot application is going to be a 200 OK response. I'm now changing it to the accepted response, which is what indicates to Heroku that this is an async request. Now instead of interacting directly with the Heroku communicator service from the controller, we're now interacting with the Heroku provision service and it in turn is going to interact with the Heroku communicator service. We'll see that in just a little bit. You can see that I added on a Heroku addon config model that's going to replace our map. I also added in a Heroku provision response model that's also going to replace another map. Then I refactored or I added some additional methods to the Heroku communicator service. So let's go and take a look at that. Let's go and take a look at these changes. So now instead of the Heroku communicator service, I have the Heroku provision service. Now it may not be obvious yet why I made that change, but it does simplify our controller. We basically have three lines. We have our log line, our provision line, and we return a Heroku provision response that just has the UID that Heroku sent and a message. This response combined with the accepted status is what tells Heroku that we're now in the world of async. Now let's go look at our provision method of our Heroku provision service. And one thing that's really nice about Spring Boot is its ability to provide async services without us having to do a lot of heavy lifting. There are two changes that we had to make to support async. One is to add the async annotation on the method that we want to be async. And the other is on the application itself, I needed to add the enable async annotation. Once I do that, I don't have to do anything special here. I don't have to manage thread executors and the kind of stuff that we had to do in the past. I simply annotate this as async. And then I call the actual method that's doing the heavy lifting here from this async method. Now if we look at our do provision method here, now this is interacting with the Heroku communicator service rather than our controller doing that itself. Because this now is happening async. So we're exchanging the authorization code for tokens. And then I've introduced an artificial sleep here of 10 seconds. We'll see how that plays out with the Heroku provisioning in just a moment. And now I'm setting up a Heroku add on config. This is where we're going to set our var1 equals val1. So I set up this Heroku config with var1 and val1. And now I'm using the Heroku communicator service once again to update the config. This was something that I was returning directly. But now I can interact with the Heroku API using my access token. And I can update the configuration. And when the provisioning is completed, I can call the provision complete method also with the access token and the request. Let's take a look at what that looks like to complete. So what's going to happen is when we make a request to add to create the add on, it's going to return immediately and say that it's working on it. Once provisioning is complete, we hit this endpoint add on UUID actions provision. And that's going to signal to Heroku that the provisioning is complete. And it will indicate that to us both in the UI and the command line. So let's get set up to look at that in action. And so I want to go to a fit nerd add on test. And there are no configured add ons right now, but I'm going to refresh this page in just a moment once we start. So let me fire up the application. Looks like I broke something here. That should not be there. Let's try that again. All right, that's better. And now we're going to create the add on. And once I create the add on, you and I know that we have 10 seconds where it's going to be waiting to complete the provisioning. So I'm going to do add ons create. Now it's going to return immediately and it tells us that we can check the progress. We can do that from the command line, but we can also do it here. And if you notice, we have these three dots. And when it's complete, those three dots will change and now we can see that it's complete. Now that's because we called that completed endpoint to indicate that it should tell heroku that we're done provisioning. And so if we look at our output here, this is the same output before here's our incoming heroku request. And here's the configuration that I set var1 and val1 that I'm now doing asynchronously. Now most of what we've been doing up to this point, we've been interacting with from the command line, especially when we're creating and destroying the add on. And while the add on is in the alpha phase, that's what you have to do to manage it. So we get all this stuff set up and then in our next steps, we want to look at bringing the add on to market. And the life cycle is you create this as an alpha project. And I can see that its current state of fit nerd staging is in alpha. And in order to transition from alpha to beta, there are a couple of things, a couple of bits of housekeeping that you need to do with heroku. Most importantly, you need to have 10 official alpha testers that you can add in the heroku dashboard. And you need to complete the documentation and have it approved by heroku. Then you can ask to then you can request to have a transition to beta. Now the last thing I want to do in this recording is to show you an example of a heroku add on in action. And that is the octa heroku add on. And as you can see from this dashboard, it's now in beta. That means that it's publicly accessible. And it means that it can be used directly from the web interface and not just the command line client. When it's an alpha, your only choice is the command line. When it's in beta, you can now use it from the web interface as well. So let's jump back over to our application and let's configure another add on. And in this case, the add on that I want to add is octa. Before we saw or the beginning, we saw that I added my sequel. Now I'm adding octa. Well, what that's going to do is that is going to provision an octa org, which you'll be able to use for authentication and authorization in the cloud for your applications. You'll also be able to use the octa management API. Let's take a look at this in action. So I'm going to add octa to this a fit nerd add on test. I'm going to use the free plan. I'll click provision. And this also is async. And we can see that the purple dots over here, it's allocating. Now this is going to take about 20 seconds and now it's converted to this green check. So we've officially added octa to this application. Now what does that mean? What does that get us? Well, let's first look back at our command line. Let's take a look at the configuration of our app. I want to do a fit nerd add on test again. Got to be a double dash. Okay. So now we still have the variables that were set from our test add on. But now we have these official variables from octa. So we have an endpoint for our octa org. We have an API token. And then we have some open ID connect OAuth values, client ID, client secret and issuer. Let's see what just as an example, what this can do for us. So I'm going to use the octa API to take a look at the users that are in this org. Again, I'm going to use HTTP and I'm going to use the API endpoint API v1 users. And then I need to give it an authorization header. This is the octa authorization header. And there's the API token. And now it's going to return to me a list of all the users that were created in this org. And so far there's only one user that's been created and this is all the information about that user. Now the other benefit I have here is I've implemented single sign on for this octa add on. And what that means is that by clicking this octa link right here, it's going to bring me over to the octa dashboard in an authenticated state as my super user. And from here I can work with all of the aspects of octa and the features that it has to offer. So this is an actual real Heroku add on in action and this will enable you to hook your applications into octa right out of the gate and to take full advantage of all of the features of octa. So I hope you've learned something from this recording about building a Heroku add on. It's quite a journey. The Heroku documentation is really good. Check it out and I'll make sure that there's a list of links and references on the page that you find this video.