 Great. So welcome to Bosch Day. Sorry for the technical difficulties. My name is Matt Ryder. I am a product manager and a director at Pivotal, and I lead product strategy for Platform, which Bosch is a part of. So today I'm going to talk with you about Bosch Manifest Sanity. And so some of these things that involve making Manifest more sane are things that you can take advantage of today. They're part of what we call Bosch 2.0, which you can read about. Things sort of shift from Bosch Notes, which you can Google. Just type in Bosch Notes, and it'll lead you to notes that are on GitHub about the things that are changing in Bosch. And then slowly as those things are implemented, they move out of Bosch Notes, and they're actually documented on Bosch I.O. So some of these things have moved out of Bosch Notes, and they're up on Bosch I.O. because they're real features that you can take advantage of. And some other things that I'm going to talk about are in Bosch Notes as things that haven't been implemented, but are coming. And then the last thing that I will talk about that's a thing that will make Manifest more sane is something that's not on Bosch Notes, but it's more of an idea that I want to discuss with you and get input on because a lot of people in the community as well as people within the foundation and some of the folks that are at Pivotal and IBM and SAP really would like to see some sort of packaging solution for Bosch. More self-descriptive, self-contained. We're calling them bundles as you'll see in a moment. So this is more of an idea. It's very loose. Nothing, no code has been written yet, but it's something that I'll describe with you today. All right. So before we look at these ideas, let's back up a little. This will also help for those in the room who don't know what a manifest is. I'm sort of hoping that most of you do. But we need to back up because in order to see why a manifest should change, we need to know what it looks like today. So I'm going to talk very high level and very briefly about what Bosch is and how it works. This isn't going to be an elaborate description of Bosch at all. It's just at the fundamentals. So Bosch has a special vocation. It provisions complex distributed software like Cloud Foundry on to virtualized or bare metal infrastructure. There is a CPI for that. And that little fellow on the left there, that's the Bosch director. She's directing the installation, the orchestration of those virtual machines on the right. If you're wondering why she's represented as a clam, it's a great question. And I'm not sure actually where the clam came from. I think it may have been James Baer. I think he may have actually created it himself using some. We don't have a logo. I know. And the clam is like an unofficial logo. Dr. Nick was talking about he doesn't have a Bosch shirt. So he has to wear his concourse shirt. Wouldn't it be nice if we had Bosch shirts? I'm not sure if the clam is what we should stick with. But all of my slides have it because it's cute. The yellow circle does bring, yeah. Next slide, previous slide. All the slides have yellow circles with clams in them. Why was the clam chosen? Because Bosch was originally known as the Bosch outer shell, this recursive name, the B for Bosch and the OSH for outer shell. And those circles there represent the outer shell, which is this orchestration layer that Bosch lives within. And then the things that it provisions and orchestrates, which are the inner shells. So we talk all about outer shells. Why not make Bosch's logo a clam? So for Bosch to perform its vocation, it needs you, the user, to tell it about a number of things. You need to tell it about the software you want to run, which is packaged as a Bosch release, the operating system to run it on, the properties for configuring that software. You also need to tell it about the storage devices that are available to it, whether it's that storage is being used as persistent storage or ephemeral storage shared or local to each machine. You also need to tell it about the resources that it has available to it, the profile of the machines, virtual machines, virtual or not that contain it. You need to tell it about the network layout and which components should be located where. And you need to tell it about the relationships between things so that they can communicate with one another over the network. The numbers of each component and the order in which those components should start. And finally, which kind of cloud you're targeting in order to provision all of these different dimensions according to the physics, pros and cons and folklore of each infrastructure that it understands. And you tell it all these things in one file. And I think you know the name of that file. And we're going to talk a lot about that file today. And that file is called the manifest, Bosch manifest. So over the course of the last six weeks or so, we've been talking with a lot of Bosch users. And we've been talking with those Bosch users because we want to understand how we can make that manifest easier. And how we can make their lives easier. So we've been traveling around. We've also been having a lot of phone calls with customers that have been here at summit. And you heard many of them talk today. We've been spending time on the phone with them talking to them about Bosch. Listening to them tell us about the tools that they build around Bosch. How they construct and change their manifests. Also how they track the changes that you learn about in the release notes of CF release, how they track those, how they understand them, how they understand how their Bosch manifest should change based on those those properties that they're changing in CF release. And so we've learned a lot. And these are all things that people in the room know about. And what we've learned is that we hear some consistent things. And we hear consistent things about the manifest, specifically the cloud foundry manifest, which is obviously the most popular manifest that people are going to write. And it's become really what Bosch's domain is. We've heard that Bosch is used for a number of different, you know, installations of different software packages. But really, it's most likely associated with cloud foundry. So let's talk about the consistent things that they say. They're really on the negative side of the spectrum. We're going to fill in the blanks a little bit. So the CF manifest is long. There's no denying that it's long. It's very long. It's thousands of lines long. The CF manifest is also repetitive. It repeats a number of stanzas for a number of things. For instance, I looked at a cloud foundry manifest and I searched for the word cert. And I found one called agent agent cert. And is this the cert for the Bosch agent, the Metron agent? I don't know. I don't actually care. The point is that this cert appears 10 times within the manifest. The same one. And that's just one thing. It happens to be a really big thing, but it's just a thing. And yes, we'll talk a little bit about security and encrypting in a second. Quick, take a picture of it. This is not a long, this is not a long lived environment. The CF manifest is complicated. This one's relatively obvious. We're building on the last couple of things. It's long, it's repetitive. So obviously, it's going to be complicated because long, repetitive Bosch manifests have a number of properties, as I described, for dozens of different software components. And the purpose and structure of these things either requires tribal knowledge or a fair amount of discourse about what's in the CF runtime release notes. It's overwhelming. Long, repetitive and complicated, those describe the manifest and this word describes the feeling that I have. I know that that's how I feel when I'm looking at 5,000 lines of YAML. I look at the length of it and the number of things that I don't understand and I get this feeling that I'm not in control. And I miss being in control and I miss maybe using some other tools that I've learned that I feel in control of instead of this one. That's not a feeling we want to give Bosch users. So the title of my presentation is Ideas for Sanity. So it sort of goes that obviously I must think that the Bosch manifest is insane. It follows that the current state must be the opposite. It's a bit of a melodramatic thing to say, but for those in the room who know me, I can go that way sometimes. I think that it's justified. And so the rest of the presentation, I'm going to lead you through the ways that it is insane and the ways that we're making it more sane with some additional ideas for the future. So first of all, why is it insane? So you may think that insanity has something to do with mental health, but that's not actually where the term is used. So people, mental health professionals and physicians don't use the term insanity. They don't like that term. It's really a legal definition. And it's used in the courtroom. And there are rules about the insanity defense and mens re, which is basically a person's understanding of being aware of their own guilt. And there's the general definition from English law that insanity is a mental state in which a person cannot distinguish between good and evil. So why are Bosch manifests insane? Because to date, they've been doing a lot of evil things. And they've been claiming we've been claiming that it's good or at least good enough. So fortunately, we in this room are not insane. We can distinguish between good and evil, I think. But maybe maybe that's the topic for another conversation. All right, so the reason I'm bringing up good and evil is two fold. First, there are things we should do that will shift us away from design patterns that are evil towards design patterns that are good. But really, I just needed a framework to contrast these things and why not good and evil. All right, the first design pattern is something called the hammer principle. It's a trap that a lot of people fall into in software. They choose tools for the wrong reasons. And it usually results in choosing the wrong tool. So we fall into this trap because we're more familiar with a tool or we have some unjustified religious stance on a tool that isn't grounded in reality. And in this case, Bosch manifests went for the sane and good choice, not the evil choice. And that's YAML. The wrong tool probably would have been XML, which was an option once. And it's actually where YAML came from. And the reason why YAML is the same choice is because XML was never meant to be human readable. It was meant to be read by a machine. And YAML's author, Clark Evans, thinks that maybe unconsciously he chose this format because of the wiki format that was meant to be read by both machine and people. So that was a sane choice. YAML is not markup language. That's his name. It's actually better. But then we come to a different notion, which is, oops, did I skip a slide? I skipped a letter. In capsillation. So there's good and evil here, too. This is another design pattern where the Bosch manifest has suffered. So on the left, we have the idea of encapsulation. And it's represented by a number of countries, each with its own language, tradition and culture. And on the right, we have this indistinguishable blob of a thing called planet Earth, which is some sort of evil global monoculture. And this, unlike making the sane choice with choosing YAML, we've actually gone a little bit towards the evil choice of not encapsulating, not drawing boundaries around things that are in a manifest. So the manifest has a lot of sort of global state. It's a pattern that Bosch is used from the start. On the left, we have a cloud foundry manifest. And this is one that ops manager, Pivotal ops manager, has generated. It's almost 5,600 lines long. And sure, ops manager produced it. So it probably has a lot of superfluous items in it that a human wouldn't have introduced. But the idea is the same. It's one giant file, and it includes the release, the stem cell, networking, storage, resource pools, configuration properties for each job, or VM that Bosch deploys. And the boundaries between things, they're there, but they take a lot of scrolling and searching to piece apart. Okay, so now let's look at how we've started to convert this evil thing into a good one. With some of the changes that are taking place in this thing, we're calling Bosch 2.0. Bosch introduces two new boundaries to isolate different concerns. And the first one is called the cloud config, which gives us a new definition of something called an availability zone that Bosch never knew about before. So this is a concept that was formed to Bosch. So you define this availability zone in the cloud config. You also are going to define definitions of resources and networking, basically all of the IaaS settings for AWS and vSphere and OpenStack and Azure and Google Cloud coming soon. So this allows for a boundary between the things that are IaaS specific, and which is represented here with Germany, and the things that are about your deployment, which in this case is a little picture of Iceland. Two boundaries around things that are distinct. So we're starting to tease apart and encapsulate different concerns by separating things out of the manifest. Things you may not touch ever again, regardless of how many deployments you do on that IaaS. Maybe you just write that file once. And we also have a new boundary for a thing called a runtime config, which is Spain in this case. It's how a global machine, global machine settings can be placed on the VM. Pivotal released an IPsec add-on that uses the runtime config. And there are other add-ons for adding users to a VM, changing the login manner and other fun things like that. Okay, so this brings me to another design pattern, which is the dry versus wet pattern. Dry, do not repeat yourself. Wet, write everything twice. So the Bosch manifest is pretty wet. A good example of that is where jobs need to reference other jobs. So like the NATS job, for example. So this stanza, which shows the NATS user and password port and location, it's repeated for every job, it's repeated 25 times in the manifest. And this is not the only example of wetness within a Bosch manifest. A slight detour here. How many of you were at Derek Coulson's talk yesterday? He was talking about how RabbitMQ was, Derek wrote NATS. And he was talking about RabbitMQ was originally the message bus in Cloud Foundry. And so I've always been curious about why NATS would be better than Rabbit. So I went off and I googled why NATS replaced Rabbit and here's what I got, which really isn't very informative. So I guess the reason for that will go with Derek to his grave. All right, so we want to dry up our manifest and we want to do that. We are going to do that and we can do that using another Bosch 2.0 feature. So these are the things that are possible again today and I'll get into some things that are not yet possible that we're planning for tomorrow. So we're using a feature now that's introduced, it's called links. So in places where one VM or one release provides something that other VMs or releases will consume, the component author will use a link directly in their Bosch release. So that redundant stanza of NATS, where you find NATS and how you log into NATS, it'll no longer appear 25 times. In fact, it might not appear at all if the release can reference one another from inside of themselves. So this silly picture of a fan that's blowed during a basset hound can be personified, in our case, to a human being that might be among us in the audience. This person is Amit Gupta who is a pivotal employee and he along with his team are deconstructing all the things that comprise CF release, making it more composable and using Bosch links rather than relying on these repeated stanzas in the Bosch manifest. All right, so here we go into the future. We just talked about the present and Bosch 2.0 giving us some new ways of drawing up and encapsulating, drawing boundaries using the cloud config. Yeah, I don't know that Demetri's here to answer that question. I don't think Bosch 2.0 is a thing. Absolutely. Bosch 2.0 is just a name for features that are new. We are going to, I'm not going to go through the speaker's notes like I was before. I'm just going to quickly wrap up because I was right at the best part, which was stuff that Bosch, thanks, things that are forward-looking, one of which is documented at Bosch notes, like I said, and one of which is not. And so I just, I wonder it, you know, it's forward-looking so the authenticity is questionable. Maybe I have like a shut-off mechanism in myself, you know, like a moral code like, oh, this might not happen. Shut down. I don't know. Okay, so into the future. Let's hope this little thing works. It does. Okay, so the first thing is, and I, we looked at a manifest earlier in the presentation, my presentation that showed an SSL cert as well as the NATS credentials that were sitting in a manifest. There are lots of ways that I've seen people extract those secrets and then later on inject them back into the manifest, but the best thing to do would be to get them out of the manifest completely. So this is up on Bosch notes today and the way in which we're going to be extracting secrets, this is brand new, is using what our friend Dmitri is calling the config server. So again, you can read about this. It's basically, it's a clean API on getting and setting values and this could be done in one of two places. You could either do it in the manifest itself or you could do it in the releases in the same way that links have references to things. Those links could also have references to these key value pairs that are sitting in this key value store. And so when the director is doing a deploy, it'll basically dynamically evaluate these references, replace them with the real thing, which could be a secret, it also could be anything, which is why it's not called a secret server, it's just a config server because this could be any config setting. And so that's step one, not only in shrinking the manifest down, which was part of the theme of the rest of the presentation, but also in extracting secrets from our manifest. So that's really great. Yeah. Mr. Nick, last time you interrupted, things didn't go so well. But go ahead. What's your question? Version numbers. So stage one of this config server doesn't involve encryption yet, but it's a clean API and you could swap something else out underneath it, perhaps. Okay. So packaging bundling easier ring. So this is the part that's not documented anywhere in that it's just a conversation that we're having. And I invite all of you to be part of that conversation. We could do it on the mailing list. You could email me or Dimitri or Nick or Chip or, but basically we would like to come up with a way of making software more self-contained. The releases, you know, they have all sorts of, there's metadata inside the releases today, but it doesn't, it can shorten the manifest if you're going to move, and this is a good practice, is to move as many things out of your manifest into your spec file. That's something that we're trying to do and Amit is helping with that, across the list of releases. But this is something different. So if you think about the flow of deploying Cloud Foundry, there are these two steps in here, right, that involve grabbing a template from probably Bosch IO and then editing that template, the first template being the Bosch init manifest, and then the second template being this really verbose thing that deploys Cloud Foundry. And wouldn't it be nice if it was something more like that? I think this is a little far-fetched. I think we still have to pass values, obviously. Bosch needs to know about your infrastructure settings, and there are definitely things that, that Bosch needs to know about the environments that you're on, if you're going to deploy Cloud Foundry and static IPs and things like that. But this idea that my friend Demetri, who's leaning against the door in the back there, hi, he's in the, he's almost in the room, another seven or eight feet, and he'd be in the room. He has to come in the room because he's on the panel, so we know that it'll happen. So we've been calling this a bundle. We don't know a better name, maybe it'll be something else, but right now it's a bundle. And there's not a whole lot that distinguishes what you see in these packages from what's in a release, right? It's a bunch of software. But what if there was something like a base manifest inside of the bundle? We already talked about separating concerns. We've got the cloud config, which is your IaaS concerns. We've got this new thing called a runtime config. What if the manifest now could also separate concerns? You've got part of the manifest that is very specific to the releases that are in it. And then you've got another part of the manifest that's specific to your deployments of that release. So the thing that's release specific, maybe it could be a little closer to the releases. So you could have sort of two of these manifests, and one could be inside of the bundle, and one could be outside of the bundle, something that you're responsible for. And so this is what the UX for this maybe might look like someday, kind of. You've got a bundle, which is maybe our latest cloud foundry, and you could upload that to the director, and that would have all of your releases, and maybe it would also have a base template for that release, so that you wouldn't have to keep recreating it with every manifest. And then we have another set of overrides, that dash O. And again, this is just conceptual right now. We're not sure that this is where we'll go, but it seems like it makes sense. You've got overrides to override or just extend what's in that base template. And the overrides could have something that we've been describing as a topology. So the first override could be your instance groups that describe maybe it's small, so everything's co-located on just a few VMs, or maybe it's large and everything is on its own VM. That would be the first override. And the second override could be something that's very specific to this particular deployment, with the static IPs for that deployment. So it might look like that. You target your Bosch, you update your cloud config with all of the things about your IaaS. So now those are out of your manifest. Then you upload your stem cell, you upload your bundle, and then you do a deploy, and you have two very narrow YAML files, instead of this huge thing that we started with at the beginning of this conversation, which was 5,000 lines and unintelligible. Okay, so I made it through that time. That's good. So I'm going to wrap it up. How I'm going to wrap it up is with a tweet that I found that I had tweeted in 2013, which was Bosch as a set of concentric circles. I tweeted this right when I think it was, Pivotal was just getting involved with Bosch. I had come out of VMware. I had met Nick on that first day when we launched Bosch. And I had all this enthusiasm about Bosch. And one of the reasons I was enthusiastic about it is because I had heard from all of these people, very smart people who were doing things like managing big production systems and wanted to distribute complex software in the cloud, that this was a really kind of crude but very holistic way of managing software in the cloud. And when I looked at it and I compared it to other solutions, it looked like it was comprehensive in that the things that we described before, configuration management, like Chef, sort of in the middle, then there are all these other things around the edges that other solutions and packages offer. To be fair, though, configuration management in these companies like OpsCode now Chef and Puppet, they do have a similar suite of technologies. So the question is, if we were there so early, why didn't this catch on? And I think the reason why has to do with what I've been talking about. Manifests are really hard. And it's a huge roadblock that gets in your way when you're trying to learn Bosch. It's very hard to learn Bosch. Nick has done a great job of giving us some tools that make that easier. Bosch, Gen, and bootstrapping tools early on to get on AWS. But it continues to be hard. And now that we're making it easier, maybe it'll be easier to onboard new participants, whereas some of these other tools like Chef are actually really easy to get started with, because they're domain specific languages and they have really easy examples. And you can do something like, oh, it's not going to trigger any problems. Don't worry. You can do something like, oh, I'm just going to set up an engine X and open port 5000. And it takes you five minutes to do something like that. The problem is that they've come at it from a different direction in that you kind of end up taking that DSL and constructing something sort of like where you would end up with Bosch, very complicated monolithic type of system that takes care of your snowflake environment in organization XYZ. We started out with the complex complexity right in your face. And I think it was necessary to do that, do it that way. Two years later, I think we're at the point where we can make it much more of a smooth transition to learning Bosch. Anyhow, I would show you the next slide, but it appears to be misbehaving. But thanks for letting me finish and have a good day.