 Hello, everybody. Welcome to the last session of OpenStack. I'm glad you guys stuck around. All right, so this is Poppy. It is a CDN project for your cloud, for OpenStack. A little bit about myself. I work for Rackspace. My name is Amit Gandhi. And at Rackspace, I am a senior engineering manager. I've been working with web technologies for close to 15 years now. A lot of it's been on website development. I've worked in the .NET, Microsoft World, and for a few years in Python and OpenStack. So one of these slides is not moving. This is not working. OK, here we go. OK, so what is Poppy? So as I said, CDN for your cloud. This is our mission statement. We want to provide a generic modular and vendor neutral API that wraps provisioning instructions for CDN vendors that support it. So in a nutshell, it's like many of the other OpenStack projects that integrate with third-party vendors. We want to integrate with companies like Akamai, with Fastly. Amazon has CloudFront. And we want to kind of unify the APIs that they provide. So if you look at the world today, there are hundreds and hundreds of CDN vendors out there. You've got big players like Akamai, Amazon with CloudFront, Edgecast. You've got newer startups that came up in the last few years, like Fastly, like Mac CDN. Many of the other companies that are on here, and there's plenty more. This graphic I actually got from CDNfinder.com. So one of the struggles that a lot of developers have, a lot of cloud operators have, is vendor lock-in. I mean, that's the whole point of OpenStack, right? We want to break that vendor lock-in. Today, if a website developer starts implementing their applications against a CDN provider, a lot of times they have to sign up to one year to a three-year agreement. So it gets very expensive. Their code base gets tied to that vendor. And then if, for whatever reason, they want to change, because features aren't provided or contract negotiations go sour, it's very difficult for them to go to another vendor. With Poppy, we want to make this really easy to do. So I'm going to go through some benefits to the different groups that Poppy is attractive to. So the first one is OpenStack operators. So these are the people that operate on OpenStack Cloud. So some of the benefits is it sits above other OpenStack services. So it's not going to be invasive to the infrastructure that you're already running. It runs almost as a platform as a service on top of what you already have. CDN will reduce load on your infrastructure. So why have all the requests going back to your Nova instances saturating your networks when you can take a lot of the static assets and stuff that can be cached on the CDN and just put it on the edge? Then you can utilize your infrastructure for the real juicy stuff that customers want to put on the load that they want to actually put on those instances. So I mentioned breaking the CDN vendor lock-in. So with Poppy, I'll go through how this works. But you can, through a configuration change, change a flavor that you want to use for your CDN. And that'll point it to another vendor. And it becomes very trivial. Customers don't need to change the DNS settings, that kind of stuff. And then lastly, it abstracts away complex CDN APIs. So if you guys have used CDNs before, you'll know that everyone is different. You have vendors that use XML, others use JSON. Everyone uses different terminologies. Everyone has different types of endpoints. And so with Poppy, what we are aiming to do is have a single base API. And it'll basically talk to these other services and take away that complexity from you. So why would a CDN vendor be interested in Poppy? Like, why would they care when they already run a business and they have customers already? So by integrating an OpenStack Poppy, it increases ease of adoption, right? So if a customer is already using OpenStack, they can start using Poppy, and boom, those vendors have new market share. It exposes a provider to a larger customer base, right? So same reason, like, if an operator has customers and they add Poppy to their portfolio, then there's a whole new group of customers that otherwise may not have gone to that vendor. And then it allows providers to compete on pure performance and ability within their niches. So whether that's like, what is their scale? How many edge nodes do they have around the world? How fast does the edge respond to requests? So it's now a performance play. It's no longer about contracts and cost and all that kind of stuff. So what about the users, right? Like, we're building OpenStack for the users that are actually out there. So these are like standard CDN benefits, right? Like, they want to speed up their websites, their applications. The world is going increasingly mobile, a lot of static content out there. So CDN will bring content close to those users. They want to restrict who can access the content. It could be based on geo or IP or where the referral is coming from. There's various ways that they want to restrict who can access that content. They want to reduce infrastructure costs, right? Like, they don't want to spin up 100 or 1,000 different services to serve traffic, put it on the edge. It's cheaper. They want to handle traffic spikes, right? Like, Black Friday is coming soon in the US. That's going to be a huge e-commerce shopping day. So how do you prepare for that? The CDN will help you. And then they want to finally easily choose a CDN based on performance rather than the effort to change, right? Like, the decision should not be, oh, I have to spend three months or six months or a year to change to CDN A or CDN B. It should just be very limited technical challenges there and then let the business decide which one is better for them. So who does Poppy work with today? So we have drivers implemented for Akamai using their wholesale delivery API product. You can see that on the open APIs on Akamai. And they have many products, so wholesale delivery is what we're focused on right now. We also integrate with Fastly, with Mac CDN, and with Amazon's CloudFront. We've also been in talks with Hiwin CDN and Edgecast on building those drivers, so we'll see how that plays out over the next few months. So what can Poppy do? So I'm going to go through a lot of these features one by one. That's just a quick glance at it. So multiple domains. So we have the ability to configure multiple domains to have the same origin and settings. So you can have like js.example.com and css.example.com and set up the same, it's going to come from the same origin. You may have the same rules for those domains, but you can settle up with multiple domains to make your configuration a lot easier. We also support multiple origins, so let's say you have all of your images stored in a public Swift container. Then you can put your images there. You can set up rules to say, OK, images pull from my public Swift container, but everything else pull from my Nova instance. And then we have caching rules. So typical CDN feature, set the TTL on your assets, and those CDN vendors will honor that. One of the recent features we added was support for secure certificates. So you have the ability through the Poppy API to request a TLS certificate, and that will assign the certificate to that particular CDN vendor. For this feature, we've seen different ways, different CDN vendors implement this. You have CDN providers like Akamai, where they handle the provisioning of certificates on their own network, and you use their APIs to do that. You have other CDN providers like CloudFront, where it's bring your own certificate. So in those cases, we're going to integrate with Barber Can to issue the certificates and provide them. So restriction rules. So as I mentioned before, we do refer restrictions, IP restrictions using the CIDR standard. We also support geographic restrictions by country codes, by areas, and you can white list and black list any of these options. And then we have host headers. So a lot of times, if you want to pull from, say, a public S3 bucket or public Swift container, then you can override what the host header is when the CDN provider pulls from your origin. That way, you can have custom domains, and on the pull request from origin, it'll set the appropriate host header as you configure it. That's also used quite a lot when people want to work around engine X rules and Apache rules that they may have in place. Content refresh. So we support cache invalidation, which is expiration of your cache. Most providers, that happens almost immediately. And then we also support the ability to do hard purges at the edge, which, depending on the provider, could be instantaneous or near real time. Other providers take minutes to hours to do those purges. But we support that capability. We have log delivery. So a lot of customers want to see the raw logs of the traffic that has been happening. And so we have the ability to take the raw logs from those CDN providers. And we copy it over into the customers' Swift containers that they provide based on the Keystone service catalog that we read. And then we also support multiple flavors. So as an operator, you can configure one or more CDN vendors to one or more flavors. So you may have a flavor called North America, for example, and then assign one or more CDN providers to the North America flavor. And then you may have another one for Asia-Pacific or China, and assign other CDN providers to those instances. And then as a user, when you select one of those flavors, we'll distribute those configurations to any provider in that flavor. So very, very extensible. So some history of Poppy. The project started pretty soon after the Atlanta summit back in June of 2014, where some discussions during that summit. And basically about, I think it was two or three weeks later, we had our first commit, which started off the project. Fast forward to November to the Paris summit. We had a design session on Poppy and how certain features should work. In the meantime, we were building drivers for Fastly, Cloud Front, and Mac CDN. Shortly after the Paris summit, we also started building the Akamai driver as their wholesale delivery API was starting to become public. So fast forward to the Vancouver summit. At that point, we had a lot of the basic features of CDN ready. And Poppy was basically at a production ready level at that point. We also had a collaboration session in Vancouver around this. The last six months during the Liberty Cycle, team has been very, very busy. We added more features like the geo-IP restrictions of content refresh and the host headers. And just last week, the TLS support came in on there. And that's where we are today. So Poppy has actually been running in production at Rackspace since May of 2015. We've had over 2,000 services already created. So it works. It's production ready, and it works. So what is the architecture? So it's very similar to other OpenStack services. We use Keystone to do the authentication authorization stuff. And then you have a Poppy server. The Poppy server is basically the RESTful API. It pulls in the requests. In the cases of GETs, it can go straight to the database and pull back the records. In the case of creations or updates, it puts an event on a message queue. We're using OpenStack's Taskflow. That's related to the Mishul project if you follow that. And so the message goes onto the queue. And then we have a series of workers which can read from there. The workers will basically take action based on what the event is. Usually if it's a create, it'll go install something in the database about what's been created. It'll go to the CDN provider API, so the Akamai, or the Fastly, or the CloudFront, and create the appropriate services there based on what's in the event. And we also talk to DNS. So a lot of times, like an operator will want to create a intermediary DNS record in the CNAME chain. This gives the operator control where if they want to change who that provider in the back end is later, then the operator has full control of changing it. And customers don't need to update their own DNS records for that. And then you have the CDN Edge servers where that's all on that third party CDN. So just like other OpenStack projects, everything is very modular. It's driver-based. So the transport driver is pluggable. Today, we have one driver. It's based on Pekan. It's a whiskey framework. We have a manager driver. Really, this is where a lot of the business logic lies. I don't see anyone ever building a different driver for that. But we have that. We also have drivers that talk to the messaging bus. And so if you didn't want to use Taskflow and you wanted to implement some other kind of way for the messaging to work, you can always substitute that out. Storage driver, we use Cassandra today. We do have plans to add SQL Alchemy. But today, Cassandra has been our focus for the team. DNS drivers. So as I mentioned, that allows you to add an intermediary record in the CNAME chain. So it gives you more flexibility to change providers later on. And so today, we have implemented the Rackspace Cloud DNS API. We do have plans during this next cycle to add designate to that. So it's coming very soon. And then you've got provider extension. So this is where you can have one or more providers. And you just build a driver. It's got to implement the API that you have defined. So stuff like create, delete, updates. And it'll basically go and talk to those APIs to create it. So this is all interesting. But how do I use Poppy? Well, it's very simple. So how do I create a service? Well, you make a post call to the services endpoint. Give it a name. That's purely for the user to recognize what they're working with. And give it a flavor ID. So as an operator, you would have defined multiple flavors. There is a endpoint to get from the list of flavors. And then give it the domain. So in this example here, I want my domain to be www.demo.me. That's what customers would type into the URL bar on Chrome or Safari or wherever. And you define the origin. So the origin could be an IP address. It could be the IP address of your Nova instance. It could be the IP address of your load balancer. Or it could just be another domain that you have set up through DNS. This is an asynchronous call. So we're going to return you a 202 with a location header, which you can then poll to check the status for. Updating a service is similar. We follow JSON patch semantics. So you can call a patch to services and pass in that service that was returned on the create call. So if you're familiar with JSON patch, you'll see that you pass in an operation. It could be add, remove, and I forget what the other ones are right now. You pass in the path of the entity within the JSON that you're trying to update. And then you pass in the new value. Again, we return a 202 with a location header so you can poll for the status to see when it's completed. In this example, I'm just adding a domain, but anything in the service can be updated. So you can add caching rules. You can add restriction rules, add more origins, add more domains, whatever you need to change. So retrieving a service, get, and the location header. And so this is like a little snippet of what is returned. So you can see you've got the service ID, which is that good. You have the project ID. So that's the user that created this entry. And then you have the list of domains. So if you follow these examples, I created one with demo.me. Then in my patch call, I added the cdn.demo.me. So when I do a get, you see both of those domains. I've skipped over the origin stuff in here because of space constraints. But you see you've got the status. So when you first create something, it'll say create in progress. If you try to update something, it'll say update in progress. Once it's deployed, it'll say deployed. If there was an error for whatever reason, then the status will be error. And in the errors section, you'll have a list of messages on why it would have failed. You also have a links section. So what the links are telling you is, so in this case, you see two links, demo.me.cdn315.poppcdn.com. So because we're using the operator URL in the CNAME chain, so this is the URL that the customer will need to change their DNS entry to 0.2. This entry here itself will then go to that cdn provider. So if you've worked with Akamai in the past, you will have seen they'll have something like www.demo.me.edgesweep.net. If you've worked with Fastly, they'll be www.demo.me.global.prod.fastly.net and so on. So all the cdn providers have a similar way of doing this. And this just tells the user what to CNAME to, which brings us to the next step. Go to your DNS provider and update your DNS to CNAME to that URL. So what's next for Poppy? So these are the features that we hope to get to in this next cycle. So I've split this up into three categories for display purposes. So the first one is on the OpenStack side. So we want to integrate, have capabilities to cdn enable private switch containers. So we set them up as origins. We already support public Swift containers. We also support public S3 buckets. To do a private Swift container, it involves passing in extra headers, which specify authorization type information. We're working with the Swift team to figure this out, specifically David gets. We want to build out the designate driver. We also want to integrate into Horizon and get DevStack integration. And then finally, we want improved gate tests. So today we have a pretty comprehensive test suite in there. We have unit tests, functional tests, API tests, and end-to-end tests. At the gate, at the OpenStack gate, we're not running the API tests and the end-to-end tests. And we just need to work out how to do that. And we're thinking like to run the docket containers in the gate and stuff, but we're still figuring out that part of it. We do run the tests in our own internal infrastructure at Rackspace currently, though. And all of those tests are part of the Poppy repo. So for users, one of the biggest requests we're getting from our customers is they want to see analytics. So we already get the raw logs from the cdn providers. So some of the work that we've already started doing is taking those raw logs and pushing it into big data, so using Sahara and just getting the logs into there. Once it's in big data, we can do all the map-produced jobs, put it into an analytics database, and then that can be queried and you can show pretty graphs on Horizon and all that kind of stuff. And then the last one is for vendors. So talking directly with some of our cdn vendors, like I've had discussions with Akamai, I've had discussions with Hywens. One thing, a common question I get asked is how do we add the unique features that they have on their CDNs? This is also common across other OpenStack projects like Cinder and others. And so we're investigating how do we add those extensions? And then at some point, those extensions may graduate into the core Poppy base. Other times, it'll just stay as an extension and operators can decide if they want to expose them or not. So what's next? So we want your help. The team at the moment is very small. We want a bigger community. We want to increase diversity. So we are on IRC, come and talk to us. We're on GitHub, so the OpenStack Garrett system. So under the OpenStack slash Poppy section. And then also, you can read all about it on the wiki. Just go to the OpenStack wiki under Poppy. We do our weekly meetings every Thursday at 3 PM Eastern time. So if you go to the wiki, you'll see what the UTC time is to translate to your local time zone. But we want your help. We want your participation. We want this to be a big project, a diverse project. And we're going to build more CDN vendor drivers into this as time goes on. So thank you. Do you have any questions? And please use the mic if you have questions. Don't be shy. All right. If you have any other questions, come and see me afterwards. Oh, here we have one. Which is a CDN reference model implementation? Which CDN? Reference CDN. Other OpenStack products have reference model. I'm sorry. I didn't understand that. What is the CDN of the reference model implementation? What is the difference in the OpenStack implementation for CDN? So this project is not. Old balancer is edge-approachy. Yeah, so this project is not implementing CDN itself. We are using all the other vendors to do it. So this project is not going to build edge nodes. We're not going to deploy to our own OpenStack edge nodes. But what this project does is it talks to Akamai's API, or it talks to Fastly API, or to CloudFront's API. And we provision all the services through them. And so then the CDN is done through their network using their existing toolkit. Because we don't want to rebuild CDN. There's already hundreds of companies out there that do CDN very good, very well. And so we just want to expose their APIs and break that vendor lock-in, and make it easier to use those APIs. For example, testing any CDN, very, very large scale. So to do the testing, so we have Intuit Test, which Malini right here, she's our QE. You can stand up if you want. So she implements a lot of the QE tests for this product. So what we do is we have sample websites that the test will automatically spin up and deploy onto a nova instance. And it will automatically configure using Poppy to CDN and enable that website. The test will then make requests against that website and ensure like, is it getting cash correctly? If I expire an object, is it getting expired correctly? If I set up geo restrictions or IP restrictions, are those restrictions taking effect correctly? So that's how the Intuit tests are working. And that will work with any provider because the Intuit test is from the perspective of a end user. Do we have any other questions? All right, so if you're interested in contributing or learning more, come and see me. I'll be hanging out in the front here for a little bit. But thank you very much.