 So, my name is Peter Gutts. I'm here with Stefan Wollick today. We're both from IBM. Hello? Thank you. Sorry. And we'd like to talk a little bit about the bit service today. So what exactly is the bit service? To answer that question, it probably makes sense to actually first answer the question, what are bits? So bits in the Cloud Foundry context are your application artifacts, that is, your package or source code simply. It's your compiled app, which is the droplet. It's build packs. Now, where are those bits stored? They're actually different backends. The simplest one could be local disk. That's probably more for your test environments. Could be WebDef. Could be Amazon S3, OpenStack Swift. It really doesn't matter as long as it is supported by the Ruby Fock library. Then usually it can be used as a blob store. Now, let's have a closer look at how the bits are actually used. And for this, let's see what happens if you actually push an app. And in this case, let's just push the app without starting it, actually. So in this case, the CFCLI makes a post request to the Cloud Controller against V2 apps. And that actually creates the app in the Cloud Controller. Then it returns a success code, hopefully. The next step is that it actually wants to upload all the files that make up the package, which, as we learned earlier, we call bits. And so for this, it does something that we call resource matching. And it's basically an algorithm to reduce the amount of files that we actually have to upload. So in this case, we make a put request against V2 resource match. And we provided a list of fingerprints that is currently SHA-1 checksums of the files that we want to upload. The Cloud Controller, in turn, talks to the blob store and asks which of these files do actually exist. Now, after it got the answer from the blob store, it goes back. We're here at this point. Cloud Controller returns the missing fingerprints to the CFCLI. Now the CFCLI can start and zip all the files that are still missing in the blob store and then upload the zip file to the blob store. Again, Cloud Controller will fetch all the existing files from the blob store, store the new files, and now that it has all the files that make up a package, zip them, and store them in the blob store, and of course, also tell the CFCLI, OK, everything was successful. We're done here. This is all just vanilla with Cloud Foundry, no-bit service involved. Let's have a quick look at how the bits are actually stored in your blob store. So in that case, let's do a quick demo here. We're logging into the blob store, Bosch SSH, and CDing into VARVCAP store shared where we store all the bits in a blob store. So in this case, let's have a look at the packages, which, if we do an LS or a tree in this case, is empty. And if we go now back to our local machine and do our CF push, OK? So let's just push very simply Dora with a no-start as we just did. Then it will do exactly the steps that we went through a minute ago. And now let's check on the blob store again. What has happened? OK, no rocket science, no magic. We can see that now we have the package here as a GUID in the blob store. We can check what kind of file this is, actually. We'll see it's a zip file. When we check the contents of this, so simple unzip list of that file gives us all the contents of Dora. Cool. OK, now if we can remove this, actually. Yeah, sorry about that. OK, yeah. So let's quickly check the droplets directory. And as we would expect, there is nothing in it yet. It's completely empty because we haven't staged the app yet. So now if we go back to our CF client and actually start the app, it will do its usual thing. We'll take the build pack, build our app, build our Dora app, and create the droplet. And we'll see in a second that it's going to upload the droplet. Yes, so now it's uploading the droplet to the blob store. And this takes a little while because it's a bit more. And then let's check back with the blob store and see what we have in here. And again, it's exactly what we would expect. We get the actual droplet and our Linux FS. And with that, I would like to hand over to Stefan. We'll talk more about the bit surface itself. Yeah, thanks, Peter. So the question is with all the stuff that you've seen so far, that all works, right? This is the reality today. So why do we need something in between? Why do we need a bit service? And the basic idea is that we want to free the Cloud Controller from any of the bits related work in order to be able to scale both the bits handling and the Cloud Controller independently. So depending on the characteristics of your deployment, you may have a lot of bits of stuff going on because your developers are really engaged and pushing all the time. There's lots of things going on. But you could very well imagine the opposite situation that you have something that's pretty stable and doesn't change very often, but has a lot of other Cloud Controller activities. So why would you need to have a big Cloud Controller deployment with many instances in the first case when all of the stuff is just going around bits? And in the other case, you want to have your Cloud Controller really beefed up in many instances, but there's not a lot going on with bits. So we want to have this ability to scale both Cloud Controller and the bits handling independently. The rest of the story is actually a classic microservice extraction story. So all of the benefits we get from encapsulating the bits handling into its own thing are stuff like better maintainability. So if the bit service is a thing in its own, we can develop it further. We can change things behind it. It actually also leads to simple operations. Like if you have this thing centered around bits, if there's something wrong with the bits handling, there's a certain place you can go to. It's no longer this big, massive Cloud Controller that does a lot of things, and today also does the bits handling. Now, if there's something wrong with bits handling, you just go to the bit service VM and see what's happening there. Also, following the extraction story, we get a way cleaner API, hopefully. So there is one thing, one set of API statements that we have that's only related to bits operations. And it's easier to find, easier to understand, less service to cover. Also, if we want to iterate over new features, for instance, the resource matching that we just talked about, if we want to make changes on that today, it will actually mean code changes in the Cloud Controller in the future. If there's any way to improve that, it only is changes in the bit service in this single microservice, which would hopefully make it way simpler to actually iterate over these things. So when we talk about the bit service, what does it mean from an organizational point of view? The bit service today is a CF incubation project. So there's a team, the Flintstone team, that right now consists of Peter and me as a pair. We had great help from another pair from Pivotal in London, Steve and Tiago, that helped actually starting off 46 months in this project. So we were two pairs then. But the whole focus of this project is really around managing bits. That's the sole purpose of this library. It's a Cloud Foundry-specific abstraction of object stores with a very clearly defined HTTP API. And as we said before, as a goal, it is really the encapsulation of all of the bits related functionalities and nothing else. In terms of deployment, it is completely Bosch managed. So if you enable the bit service, you'll find another job in your deployment manifest if it's generated. And you'll find, of course, this VM in your deployment that is centered around bits. When we talk about what the bit service is, of course, there's also a question of what it's not. And obviously, we're not trying to create yet another universal storage. We have that today with the blob stores that we're using. And there is no reason for us to do the same thing again, unless there is benefits. And we didn't see any. The benefit that we actually have is our clearly defined very domain-specific API for Cloud Foundry-related bits operations. That's what the bit service is. And it's not universal. Also, again, in the same vein, it's not a generic storage API. For instance, as we had seen in the previous session, like volume drivers and stuff, it's a completely different topic that is not what the bit service is. It's HTTP-based for a domain-specific bit service operation. A very quick overview of the API, I won't go into too many details to bore you with HTTP resources and verbs, just to see what we have. And as it is an extraction right now from the Cloud Controller, it is actually looking very similar to what you can see in the Cloud Controller today. So the first thing is centered around the resource matching. So there's actually just post verbs for anything that is related to finding out what's in the client, when the client is actually sending the charts, finding out what the client has to send to the server, and actually getting the whole thing out of the bit servers again with both the stuff that was sent by the client and what may have been in the backing blob store already. Then there is the build pack cache, which is something specific to applying build packs. For instance, if you have a Ruby application, there's additional stuff that the application has dependencies on, Ruby gems and stuff like that. This stuff is all stored in the build pack cache. And there is an important thing here to note. The build pack cache is the first thing as compared to the app site that is actually depending on the app quit. So whenever you destroy the application, the build pack cache will also be subsequently be removed from the bit service through API calls. And the remaining resources are actually pretty similar, but we decided to keep them separate from an implementation perspective. Again, it's all centered around quits for the individual resources. So there's build packs, there's droplets, and there's also packages from a V3 perspective, which all have similar APIs. Just put the stuff there with a certain quit. You can get it back and you can delete it when you no longer need it. And as you can see, this is not a complete list of all of the rest primitives you could imagine from a generic rest service. But it's really scoped around what is actually required for handling bits when you talk to a bit service. So it's really minimal in that sense. So what's the consequences? What's the benefit that you get today? And we will soon go into what the benefit could be in the future when we further develop it. First of all, like I said before, it's about that independent scalability. So with the bit service enabled, the Cloud Controller is now free from handling bits. And we can scale both Cloud Controller and bit service independently. As a drawback, of course, it's one more thing. Like it's a little more complexity in your deployment. It's one more VM, or potentially many, if you want to scale independently, that you need to take care of. As an application developer, extracting the bit service into its own thing, the benefit is, of course, that the CC has the potential to become more responsive because we can offload bit service work. As we will see soon, we're not quite there yet. We're in the process of doing that. But obviously, with all of the extraction and microservices stories, you first extract something in order to be able to make a change later on. It's like make change easy and then make the easy change. So we're in that first stage still. And of course, with more efficient bits handling, as an application developer, you would hopefully get also faster push times because you're pushing now something to something that's probably not as loaded and as a heavy load as a Cloud Controller. File uploads, downloads of droplets and such will hopefully become faster. If you're a Cloud Foundry developer, hopefully the API will be appealing to you because it's, like I said before, it's domain specific. So it's just centered around those certain resources and don't like the generic thing you need to or you could do anything with it, which brings us to where we are today, where we are in terms of status. So the bit service is part of Cloud Foundry since 241, which is back in August. It is enabled by a feature flag. Right now, by default, the feature flag is off. So if you install 241 or anything later on, you will have the bit service. But it's not enabled and used and also not deployed. But you can just turning it on. There is no migration needed. So if you push your application to a Cloud Foundry deployment with the bit service disabled, and if you enable it later on, there will be no migration needed. We take great care in maintaining the same layout in the backing blob store. So there's nothing that actually should go wrong there. And we, of course, do have tests for that, making sure that's the way as we promise. The API that we support is both the version 2 and version 3 API. So there is no difference if you enable the bit service. By the way, speaking of blob stores, if there's anyone in the room or if you know of anyone who is using anything else than the S3 or Swift backends with the Cloud Foundry deployment, please talk to us later on. I would be interested to hear, first of course, what you're using and also why you're using it. Because with many sessions, we would like to get some feedback on what people are using so we can focus our resources on building and supporting the right backends and also potentially understand where I assume for improvement. What's the next steps we would like to work on? Obviously, like I said before, we are in that sort of transition period. We have completed the extraction of the bit service from the Cloud Controller. But in order to realize the full benefits, there is some workflow changes required to actually read the benefits. So we are working with the CLI team. We're looking into what changes are required in the stature in order to be able to read these benefits. And as some examples on where we are and where we would like to go, we have some flows here. So first of all, there's the legacy flow on the very top. This is what you have today without the bit service. You just have the CLI talking to the Cloud Controller and the Cloud Controller stores some stuff in the Blob Store. And the middle is where we are today with the bit service if you choose to enable it. As you can see, it's just one more component when you can ask yourself, where is the benefit? As I said before, there is no direct benefit yet. But as you can see in the bottom column here, once we are able to actually make the changes to the clients and, for instance, have the CLI client actually talking to the Cloud Controller and saying, there is something to be uploaded. And we can have the Cloud Controller return back to the client saying, here's a URL. Here's a signed URL. We can safely put your bits on to it. Then it's only communication between the client and the bit service. And the Cloud Controller doesn't have to do anything while all this bit handling is being worked on. And we just have to get back to the Cloud Controller and let it know, for instance, like this, where you see and on the very bottom, once the bit service handling is done and the bit service actually responds with the package hash, for instance, this is the only second step where we need to let the Cloud Controller know that the bits upload was successful. So this is future. This is not reality today. There's a proposal out there, not very formal, but just to get some feedback. If you're interested, just have a look at this proposal and give us feedback or comments. Speaking of the proposal, there are some references here. You can find the code for the bit service, obviously, in the release. And you go from the release to source and CI and everything that we have. There's the public pipeline you can have a look at. But when I talked about the migration stuff, all of the tests we're doing to make sure we're not breaking anything, there's the tracker where I can see our backlog and potentially provide some comments or feedback to us. And finally, there's the presentation if you want to get into any of these references here. These are all online, and you can have a look at what we wrote there if you have any additional questions. With that, we're done. Any questions? Yes, please? Yeah, so the question was centered around Fog and all of the potential Blobstalk backends. And right now, there is no limit beyond the limits that are there today. Because we're still using Fog, the bit services is right now implemented as a Ruby app because of that status as an extraction from the Cloud Controller. So whatever you can use today, you can still use with the bit service. But obviously, this is very useful feedback for us that they're using NFS. Because in the end, it's about all of the stuff that we're testing. And if we're looking into maybe re-implementing stuff or optimizing stuff, it's important to know what people are actually using in terms of backends. So there's no plans right now, as far as I'm aware, to actually remove Fog. So you could still use that. But this is important as feedback for the future. Thanks for that. And Simon's sitting right behind you. So we can probably continue that. Simon's our PM, right? He covers the different zone team. And we can have a discussion around the details and later on. Thank you. Any other questions? OK, well, thanks then. We're done. Thank you.