 All right, looks like it's about that time to get started. All right, so be with me for the next 30 minutes here. I'm going to try to play with fire here and do some live demos, because nothing could possibly go wrong with that with conference Wi-Fi. So Caleb Washburn, this presentation's on Tired of the onboarding toil. This is a presentation around using CF management. It is a framework that I created based on my experiences in the field, helping customers deliver solutions based on Cloud Foundry. Just kind of for awareness here, who has actually heard of CF management besides being in the conference presentation title? Yeah, got a few people. How many people have actually used it with customers? About half, all right, so good. All right, so why CF management? What was the reason behind this? It was a desire to have a declarative way to define basically any of the entities in your Cloud Foundry system, whether that's orgs, spaces, users, and just any other entity quotas, marketplace availability, et cetera, wanting to really have the ability to do, essentially, infrastructure as code, source-controlled configuration versus when you're managing multiple foundations, having to compare and contrast what features are available, what entities are there with those orgs and spaces and users, and really just trying to get your hands around that. There's actually a gap in the product when I created this, and that gap still exists, where you're unable to map LDAP groups to organ space roles. That's really the major motivator when I created this, was working with a customer that really wanted the ability to be able to onboard users by having defined LDAP groups created and then synchronized those user mappings into organ space roles versus consistently hammering out CFCLI commands to add users with space role or space developer role, et cetera. Again, talked a little bit about this, having a versioned and auditable representation of your configuration, because things go wrong. You want to maybe be able to rebuild your foundation with the same configuration without maybe having a backup to do so. Want to be able to see why did certain things change in your ecosystem. Who created this org? Why did this org exist? Who added permissions for this user to this org? Having this source code managed configuration be the master of that versus having to filter through any number of logs in Cloud Foundry and try to figure out how did this thing exist and why is it there? And then ultimately at the end of the day, tired of the toil of whipping out the CFCLI and remembering all the commands with all the options and wanting to have really a declarative pragmatic way of doing this. So here's kind of the overview for those of you and I'll kind of brush through some of this pretty quickly. So essentially there's some manual things you have to do if you're going to onboard using various different tools, whether that's UAA, the CFCLI, going into Apps Manager itself if you're using Pivotal Cloud Foundry. That's installing lots of some of these CLIs or Ruby-based that presents some fun depending on your platform that you're working on. Being able to add users, add membership of users using the UAA command line. The CFCLI now has the ability to create certain users with origin definition, which was a newer feature that was not there when I first started creating some of this. Again, and then once you've got Oregon Space, you need to be able to see if create the user, set the org role, set the space role, create the org, create the space. Lots of different things that are, again, not hard but toil at the end of the day. As a team that is operating this platform, we wanna focus on the business value, not necessarily hammering out a bunch of command line arguments to create orgs and spaces all the time. I wanted to provide this in a way that could be a little bit self-service. So automated onboarding with CF management is really taking that manual process and really trying to codify it. So again, having a definitive source of truth for each of your foundations, and that's, we wanna use a tool that, again, is auditable, has history, and we like Git. So we decided that we can use a Git repository or any other source code management repository for that matter, to store your configuration. That configuration is, this is Cloud Foundry, so it's gotta be in YAML, so because everything's gotta be YAML in the Cloud Foundry ecosystem. So essentially being able to declare that, declare your orgs and your spaces, and check that in. There is, essentially there's a CLI for all this, but we ship with a concourse pipeline as well so that you can leverage concourse in a system to drive and declare this configuration and apply the configuration for you. So what is it? CF management is currently released on GitHub. There's a GitHub Releases section. It's actually broken into two distinctive pieces. The original piece of this is just the CF management CLI, which is the second piece. It is for applying that configuration to your Cloud Foundry and all it leverages at the end of the day is the Cloud Controller APIs as well as UAAPIs to orchestrate certain behaviors in an idempotent way. So that anything that you've got added to your Cloud Foundry can be managed with CF management. It came along that there was a desire that hand editing YAML files is tedious. It is something that is error-prone. So it provided a CF management config CLI to help you automate some of the authoring of this configuration. Got a few users that have stuck the CF management config API or CLI. Behind a web UI to make this a little bit simpler, wasn't planning to provide a GUI experience for this other than the CLI, but there's some desire to do that in CF management. The config of that allows you to do it. We also released this on Docker Hub so they can be consumed via concourse fairly easily as well. And that really just contains the Linux CLI that can be used in any of your concourse pipelines or if you're consuming it for Docker for other reasons. So how do you get started with CF management? There's really two different ways that you can do that. You can use the CF management CLI and you can just say initialize my configuration. It will bootstrap you and create a configuration folder that has a bunch of YAML files that are configured with folders and structures. And we'll dive into that here in a little bit when we do some live demoing. Or you can actually speed up the bootstrapping process. If you've already been running Cloud Foundry, which most users that come across this tool have been for a while, you don't wanna manually have to reverse engineer your configuration into this YAML format. Again, that's toil, that's tedious work. So we can look at the APIs and introspect those and reverse engineer your configuration to make the bootstrapping experience much easier. So without that, it's demo time because that's enough talking. All right, so got a Cloud Foundry foundation with basically nothing in it. And it helps if you type the command right. So see if orgs, this is running just PCF Dev locally here. Hopefully everybody can see the screen, played with the colors, but this is as good as it may get. All right, so what I wanna do is, I wanna be able to reverse engineer. That is perfect if we can do that. Thank you, sir. Wanna be able to populate some stuff into this so we can represent maybe what an initial foundation would look like. So instead of manually typing out a bunch of commands, I'm just gonna create a bunch of orgs, create a bunch of spaces, assign some space developers and show you how to get started with this tool. This is probably a pretty consistent experience if you were managing this with just the CFC Ally, you would maybe write some shell scripts, something to help automate things and allow people to create orgs and spaces and various different things. So this is almost done because we're not gonna create too many. All right, so what I wanna do now is, just to show you there's no magic here. So I've got a directory with just a bunch of shell scripts in it. I am going to end up just initializing my configuration. To initialize this, I'm introspecting that Cloud Foundry Foundation and generating bunch of YAML files. So at the end of the day, what you're gonna get is a config directory with a bunch of folders. There's a configuration folder for every org, a configuration folder for every space. As you can see, you can do security group mappings, you can do space configuration or configuration quotas, whatnot. There's lots of different things that you can do. And we can bring that up in a little bit of a better editor so we can kind of see the structure here. And that's super tiny. I don't know that I can make that any bigger. So you can see that, for instance, the built-in orgs that are the orgs that I created, there's a space here. This is really where the meat of the information is. So it's basically declaratively saying, I wanna have an org, that org is named org one because naming is the second hardest problem in computer science. So I just go with basic stuff. Space, space one, users, these are the users that I mapped into there under the notion of a space developer. You can see that you can use LDAP users, SAML users and even LDAP groups. And then every role that, if you're familiar with Cloud Foundry, there's the space developer, space manager, space auditor, enabling certain feature flags, whether that's allowing SSH, enabling CF management to manage space quotas for you or security groups. Everything that we built into CF management tried to build it in with feature flags. So you can opt into the experience at whatever level you want. At the end of the day, table stakes for this is using it to manage orgs. Some people decide to use it just to manage the orgs and let give org manager and space manager out to people so that it could create spaces. Some people decide to manage it completely holistically in this configuration. Really lets you opt into however you wanna do that. So if I'm gonna just come back here and now run CF management. So if I wanna just run CF management and run the apply command, the apply command is something that is essentially, the same thing is running the concourse pipeline. It runs the commands in the right order, which inherently that would be creating orgs before you can create spaces, creating associations in the right order for you. So if I run that because I just reverse engineer that it's not really gonna do too much because everything's idempotent and it's just going to go ahead and manage that configuration for you. So if I end up looking at this, I've got CF orgs. You can see I got a bunch of orgs. If I wanted to go ahead and do a cleanup and wipe this thing back down to its original state, while I'm talking through this, I can show you then how I can take CF management and replace that state back to the way that it was and go from there. So while that's running, we'll hop back over into the prezo here. So now, if we wanted to go ahead and use CF management to add orgs and spaces, this is where we can use the CF management config API to do that. We can either that or we can copy and paste in a text editor and edit the YAML files and deal with an indentation and troubleshooting all that experience or provide the nice wrapper here that allows you to manage the configuration in an appropriate way to add orgs and spaces to your environment. Adding roles. So that's the other thing. This again, this is the main driver around why CF management was created, around the ability to map users. Orgs and spaces are created on a fairly regular basis. It's the onboarding of users and the offboarding of users that is important. Without a mechanism to declare that, a lot of times you'll probably have users that are left in your foundation that maybe have the permissions in an org or a space that they shouldn't have anymore. And that's maybe not centrally managed. So moving that into an LDAP group where most enterprises have an offboarding process. They have someone that's managing those group membership, managing those entitlements. And there's already a built-in process. So let's not reinvent a new process. Let's just leverage the processes that are there. Again, using these commands, you can go ahead and do that and I'll demonstrate some of that as we go. So now that everything's cleaned up, so if I just do CF orgs, you'll see I'm back to my initial state. If I wanna go ahead and run the good old apply command, he's gonna go through and determine what orgs and spaces are there and go ahead and create a bunch of stuff. And of course, it is a live demo, so why wouldn't it fail? Yay, oh, that's why. Because I gotta create users. This is a feature of CF management that is a little bit of a pain point in the fact that we don't actually automate creating internal users because internal users have this thing that we wouldn't wanna put in the configuration, which is the password for the internal user. So right now, we just air out and say, you must create these internal users first and then we will assign them. So I can create user demo one with password of password and demo two, I think I had five users that I created and five. And now we can go back and the apply should work. It's going through assigning all these org space roles, users, everything to our Cloud Foundry in a way that allows us to quickly do this. Because all this configuration is in Git, the nice thing about this is I can check this configuration in. If I wanted to, I could have developers or consumers of my platform submit something that should be native to their experience, which is a pull request. Instead of maybe a ticket to the platform team to create these orgs and spaces, can we manage this in a collaborative way that lets me do that via pull request workflow? So I can apply again just to make sure that things are right and potent, things are gonna be there and as this works. So again, go CF orgs, you can see all the orgs are created, which is kind of the point of moving this configuration into a declarative format. So the other thing that you'll notice as we get into this, when we reverse engineered this configuration, we have the admin user, because I created everything with the admin account. So I'm gonna go ahead and use that CF management config API to remove remove all the admins from all the spaces. And then when I, so if I, before I do that, if I do CF space users. Yeah, helps if I spell users right too. That's what you meant. Yep, hey, that's how it works. Again, so space managers, admin, developer, if I go ahead and since I've removed those, if I go ahead and rerun the apply, you can see that I de-entitled even though the admin user is a cloud controller admin, I de-entitled him at the cloud controller level. So then if I do see space users again, magic, the demo of the admin user is gone. So this is kind of, again, still dealing with internal users and things of that nature. So if I actually wanted to convert one of these orgs, one of these spaces over into use LDAP, because I've got an LDAP server running here. I've got, oh, that is super tiny as well. So I've got a, you gotta trust me if you can't see the screen here. So there's a space developer group that has three users that are managed in LDAP. There's no users that are in LDAP. So if I look at, if I show what users are in UAA right now, I've got a bunch of users that have a local origin of UAA, demo one, demo two, demo three. Anybody that's added LDAP users and tried to bind this together, there used to be a sequence of UAA commands that you had to run a curl to create users with the right origin. That was super buggy and super hard to get the syntax right. So again, wanted to be able to create all that users for you. So if I just wanna do CF management config, and I wanna do update space, I can look at what the help is. Got context sensitive help that lets me see how things work for the org, org one, and space, space one. I wanna do developer, LDAP group, space developers, space developers. So go ahead and run that. So that space has been updated. If I go look at the space configuration, you see that he's added a space developer, LDAP group into that. And now we're gonna rerun the apply command. Ideally, in using this more of in a production context, this would be checked in. You could have something that's listening for that check-in. It would then go apply those changes and make that happen. Now if I run that same command, one to show the UAA users, you can see now that I got new users that are created with a LDAP origin. They have the DN as the external ID or the CN as the external ID. And they've got essentially their username that's associated with that. If I end up running in CF space users, you could see now those users have been added as space developers. If I wanna actually show the idea of deprovisioning those users, I could see come in here and simulate that as when a user gets removed from LDAP, from an LDAP group, and then I rerun CF management. Oh, not that. Oh, I hope something went the right way, no. So if I go ahead and run that apply, you'll see that one thing that's happening is I'm removing CF management demo three user from that organ space. But because also that that user no longer has any org roles, I'm also removing that user as an org provisioned, org associated user. So if I go back and look at CF space users, oh, you can see demo three's been removed. Okay. So at this point, there's other features that we can kinda show you walking through this. Got a few more minutes and then I'll open it up for question. The other thing that you can see that is a newer feature that we added is around service plans. This is something that came up that it was kind of a pain point from a platform teams perspective to manage. I wanna add new features in the form of services to the marketplace, but maybe I don't wanna expose those to every one of the users. So I wanna be a declarative way that I can have certain features opted in, which is essentially the service access command. So you can see right now that I've got P MySQL that's available for DB large, DB medium, DB small available in a limited fashion to just CF dev org and the system org. If I wanted to expose this out, if I go into the CF management config, oh, it's at the org level. So if I wanna look at org one, you can see that basically for service access here, for P MySQL, I've been granted no access. So I can update this with all and rerun apply. And then if I go back and look at service access, you should be able to see, this is why the live demo was fun. Oh, I think it might not be all. I think it might be star. Maybe not, we'll try. And this is why I should use the command. Yeah, so it was star. So if I go back and look at service access, you can see it's been granted to that org. Or if I wanted to deprovision it from a given org, if I go back up into the CF management dev, you can see in his, he has access to small medium and large. I don't wanna give large to anybody to this org. So I can reapply that config. And he runs through all this. If I go look at service access, you can see he's removed them from large. So a quick way that you can declare which ways you wanna manage service access and any other thing. Again quotas, security groups are part of this from a management perspective, basically anything that you can manage outside of an app. So service instances are not, but the services that are in the marketplace are. So with that, that was a quick half hour. So yeah, so what else can I manage? Again, going through that, one of the other features that we did add that I do wanna mention is got a feature request to be able to do time-based SSH access. So you can actually put in that you wanna enable SSH access for space for say 24 hours and it will update your configuration and then there's a concourse job that has a timer that continuously pulls to see if that 24 hours has expired. If so, it will remove it. Which is kind of a nice feature if you need to allow some temporary admin debugging or debugging level at an application level. Private domains are another thing that are, as well as isolation segments is something that's prevailing that we're seeing a lot of people trying to build out isolation segments and managing the configuration of that. So with that, I'll turn it over to questions and if there's no questions here, I'll be out in the hall afterwards. For managing UAA users, there is currently not a facility for creating UAA users because of the user ID password constraint. It's not that we couldn't automate that process, it's just there's not, hasn't been really a clear path on if I set the password, how do I get the password to the user and do we wanna set the default password to the same for every user? Looked at maybe using like the notifications API as a way to do it, but nothing at this point in time. Most companies that are using this are using an external configuration, not using the internal UAA, so it hasn't really been a huge challenge at this point. Are we talking about UAA clients? Okay. Nothing with UAA clients yet, that might be something, again, they would have a client secret so we'd have to figure out a strategy to manage that as well. But yeah. No, you can do them as shadow users as well. So you can define groups or you can define LDAP users. So within the configuration, you could actually have come in here instead of, where did I have that? So instead of space developers, I could have just came in here and added CF management demo five. And he would have then created that LDAP shadow user in UAA and then mapped the groups to that. No, so, okay, so wanting to be able to essentially mirror your whole LDAP infrastructure into UAA. No, not at this point. It's essentially all we're doing is only adding the users that have entitlements within the space or org from that perspective. A user could log into the foundation and they get created as a shadow user with no permissions and we're trying to avoid that kind of user experience where we've onboarded them first because the workflow used to be, it's just easier for you to log in and see a blank page and see no orgs and spaces and then I'll map you later because I don't wanna create you in UAA first. This has allowed us to avoid that workflow. We've made a lot of optimizations in this to ensure that we're not with larger organizations with a lot of LDAP users and a lot of orgs and spaces. It was pretty chatty, some of the earlier versions. We made a lot of optimizations to short circuit the lookups which helped it scale. For larger companies it was taken like an hour and a half to sync the LDAP structure and now it's like less than five minutes. We can sync very, very large organizations. Again, yeah. Yeah, early on we did. It was fairly non-performance in the way that it would look things up. It was looking everything up per query, per org, per space. Right now what it does is it essentially queries all the orgs and all the spaces up front and then just iterates through those in memory. So there's a one-time hit to query the Cloud Controller and page through things but instead of looking things up based on org and space individually, we were seeing a lot of IO problems with performance. So yeah, I tried to improve that but there may be scenarios where definitely the way to provide feedback on this is try to be very attentive to the issues in GitHub since this isn't really a supported product from like Pivotal's perspective as company. This is more my side project that seems to get used on many customers. So essentially try to be very aware of the issues and get things updated in a pretty expedient fashion. Yep. Yeah, so the nice thing about this is there's nothing in this configuration because the system domain, the user IDs and passwords are all externalized. I mean, you provide that to the apply process. Say you were gonna deploy two foundations, you wanted to have those in an active-active setup, you can use the same Git repository and then you can basically apply to both of them and synchronize the foundation so that if your expectation from a consumer perspective is if I have an org in the active, I have an org in the other active, we wanna keep that in sync. There's nothing that synchronizes that natively within Cloud Foundry today because those are two distinct foundations. But we do see customers using one config repo that then they can manage multiple foundations in a consistent way. There's typically a different repo or different folder within the repository for managing your dev foundation versus your prod foundation. But as much as you can do, it really just is pretty flexible around how do you want, what do you want the end state to be expected? So if you want those to be in sync, then use one repository, don't use separate repositories, excuse me. Okay, we're at time, but I don't think anybody's beating down the door. So if there's any questions, otherwise I'll be out in the hall afterwards. All right.