 Hello everyone and welcome to our talk trust the hamster That is such a weird title. Isn't it who came up with that title? No, do you know? Yes, I know but this guy has left the team for girl called Irene left the team I see for a girl. I Well, anyways, we'll be talking about the bid service today My name is Peter. This is Norman and Let's get the most important things out first What is the most important thing? We have a new logo and it's a hamster and the hamster says hello It's a hamster because the hamster eats your bits and nuts Eats them all and rum rum rum and it keeps them for you until you actually need them again We also have stickers. So if you want to come come to us after the talk We have lots of stickers also at the IBM booth. There is more Yeah, so whenever you see the hamster, you know, this is about the bit service and With that, let's talk about what the bit service actually is and get some recap Yeah, what is the bit service? So the bit service is a official cloud foundry Incubation project the functionality is an extraction from the cloud controller We built an cloud foundry come an API for object stores and what does it it does handle bits So what does it means it handles all the packages the build packs and the droplets? So build packs or the runtime packages. That is your code and the droplet is the Container representation for Diego So why do why do we do it? right What we want to achieve is separation of concern in two levels one one level is the organizational part We would ship this the responsibility for this bits handling domain to a separate team And on the other hand, we want to have this micro service approach that we can For example get load out of the load away from the from the cloud controller have independent scalability Get better maintenance With that we get a lot of flexibility hopefully and yeah, and one one thing one thing more is that we want to also have Independent deployments without rolling a cloud controller. That's one thing and performance improvements. So performance improvements Was already something There was actually something yes, so we this year we did a re-implementation of the bit service in Golang and Since that it has become lightning fast But that wasn't actually the original reason to rewrite the bit service in go So let me back off a little bit Originally the bit service was written in Ruby and it was simply because the bit service was an extraction from the cloud controller So far a cloud controller handled all the bits now its bit service and for that reason it was written in in Ruby and There is nothing wrong with that actually It's just that our code base didn't age quite well. So we saw actually several problems One problem was that we had a lot of code duplication now our code Another problem is that we noticed that with the framework that we used in Ruby in this case the Sinatra framework It's actually quite difficult to get proper dependency injection. It was kind of annoying We noticed that our log files were a complete mess So whenever you would get a bad request or a 404 not found And you look at the log file to understand what's going on. Why is it a bad request? You couldn't figure it out. There's there was a lot of noise in those log files, but they were not very helpful We also spent a lot of time configuring engine X which we use as a proxy for the bit service or which we used as a proxy for the bit service and That was really a huge time sink and might just be that we're not smart enough to understand this conflict But it just didn't work very well for us Another problem that we noticed with engine X was with monitoring So we were using stats D like most of the projects, but it seemed impossible to get stats D metrics for let's say request response times request sizes Status codes and so on out of the box and so there wasn't a very easy solution to do that With all of that, of course, you could still fix it in the old code base But that at that point we felt like it's marginally more effort to do a rewrite in Golang And so that's what we did and it actually came with some really really nice benefits that we didn't even expect at that point And that's the performance improvement. So let's talk about those Yeah, it was really a surprise. So what did we we did a test where we fired 10 concurrent request against a bit service with a local Blobster backend and We did no special tweaks. We just used the Golang implementation with a max library and review implementation with a Sinatra framework and Did this this simple test there when we look at the counts Forget requests. We are 200 times faster than the Ruby implementation sorry for that and for for put requests, it's it's around 50 50 times faster and would We just want to emphasize. I mean, this is not a specific Ruby problem very likely it's probably the combination of Ruby and Sinatra and potentially other things how we do it in the bit service, but yeah, this is simply what we saw Yeah, it's it's as you see as you thought this is only what we what we observed on the on the test So with all the with all this great news, I mean, when is when is this available, Peter? When is this available in fact the bit service when GA last week, so now it's available in CF deployment as an ops file so it's very easy easy actually to use the bit service in In Cloud Foundry that didn't render that well, sorry for that But so what you really do is just you do your usual Bosch deploy specify CF deployment YAML and then you add two additional ops files One is the one use bit service and the second one is config a configure a bit service with Your desired back-end we support support all the back ends that the cloud controller currently supports that is s3 GCP web dev local open stack and so on So it's really that simple Those two line add those two lines to your Bosch deploy and you'll have bit service in your deployment But I think there was another a major mile milestone that we hit this year Norm maybe you can tell us more about those Yes, I can for sure big milestone is we have running the bit service also in the IBM cloud public So and with this we get this cool thing Matrix so let me phrase you the comparison table a bit We know that cloud controller is not a not comparable with bit service But what we here compare is the bit service the bits handling the main, right? So only this is the comparison not anything else. So let me let me let me explain it. So What is if you have the cloud controller? He handles the bits for you, right? So if you enable the bit service nothing change, it still handles bits but what you get for free is this kind of matrix which is Ah, good good to know So you get response times you get response size and my personal favorite response status codes Okay, no Sorry, so now sorry that was a joke and that is how it looks like in our production system And so just to add to this I mean this has real Value in production systems because what we saw at least in bluemix is when it's unclear If there are issues like your your system degrades in performance For example a cloud controller returns slow responses, and you have no idea where it comes from Just knowing it's just from that separation Knowing that it's actually a blob star handling which is slow and which is not which you can now see from the bit service Actually makes it pretty clear where it comes from whereas if those metrics are actually fine Then you know it's something specific to cloud controller So we get a lot more transparency here when it comes to bits handling versus other API calls against the cloud controller Yes, indeed. All right. So with that I have another Another secret to share actually Norman. Oh secrets secrets are good We are getting married get what getting married. What yeah, we're getting married we the bit service. Yes We're At least we got engaged. Let's let's say we got engaged We got engaged with Irini or I should I should say Irini like we learned Yesterday, so who knows what Irini is who saw the talk yesterday Me too. Yeah, who saw that who saw the talk yesterday. Oh Okay, she watched them definitely on YouTube. So I'll I'll give you Irini in a nutshell it's cloud foundry coming together with Kubernetes and The demonstration that Jules gave yesterday was pretty impressive. So you could see how you can do your CF push and Your app actually gets run in Kubernetes there is one small thing missing actually in that whole chain and it's an OCI image registry and The bit service can actually fill that gap because the bit service is already the master of droplets and An image for Kubernetes is basically just your root of s plus a droplet on top of that That's more or less your image. And so the bit service is actually the perfect spot to provide such an image and with that The way it looks like when when you actually Push an app looks like this Cloud controller goes ahead and says hey stature. Can you stage this app for me? And then the stature says, okay? Hey bit service. I've just built this droplet from a package. Can you store it for me? I'm a bit server says sure done and then the cloud controller talks to Irini and says hey Irini Staging is done. Can you schedule a container for me? And then I Irini says sure I'll ask my friend Kubernetes to do it And then it says hey Kubernetes can you schedule a container for me? You can get the image from bit service Then bit Kubernetes just goes ahead and said sure thing says sure thing. Hey bit service. I need an OCI image From the droplet you start earlier and then the bit service just says here you go So that's the basic flow how it works between cloud controllers stature bit service Irini and Kubernetes and With that we'll show a quick demo of what this looks like. We actually wrote a little Proof of concept so it's not done yet, but the proof of concept already works so this Okay, and Really sorry for the for the bad rendering here couldn't figure out what the problem was here All right, so what I have here is a couple of windows on the top right I have are you usual CF apps in a watch command? So we see all the apps that are currently running in our cloud foundry Right below that I have all the Kubernetes pots that are running Which which were started by Irini, so that's in the Irini namespace and then on The bottom right I'll show the events that Kubernetes will show us which happen in Irini So we'll just watch that actually let me Clear that and do it again Here The focus yeah, it all yeah Really sorry for that. I don't know. I'll okay. Let's do the following. We'll just create another small window So we'll make them all gray So now you can read them at least as long as they're not focused so we also open a log file for the bit service and With that We I'm here in Dora and I'll do your typical CF push And we'll run that So now what you see is just the normal staging that's happening So you can already see that it it created the app in the cloud controller on the top right It's Sorry, let me show you here a little more. So here you can see the whole build pack thing and then staging it. Yeah now it's installing all the Ruby gems and As soon as the staging is done, we should see some interaction on the top left. I'm sorry The bottom left and the bottom right So now you can see it Here Kubernetes just told us that it pulled the image right here From something that is called registry dot some IP dot nip IO and this is actually just the bit service running there and serving the OCI image to Kubernetes and It's done. So now we can Curl Dora We can see that Dora is responding and that's it that's Bit service working together with Irini and of course just like we did with Kubernetes any scheduler that supports OCI images like mesos Knative obviously Doc a swarm it will just work with bit service as an OCI image image registry So everything what works with docker pool should we work with us exactly? Right and with that We have office hour this office hours this afternoon at 250 till 320 so please come as I said, we'll have more stickers. We'll have time to answer questions Yeah, that's it that's it wrap up Yeah, can you start again? Oh, yeah, sorry, right the last slide Okay, yeah, awesome stuff to share really really really cool But let me say this this this stuff we did it not alone. There's a lot of people involved There are people which bootstrap this project. They are named here. There was Stefan Ulick Simon Moser Steve Witzel, Thiago Scolari and Mark Chung and the current team setup is Peter Goetz to be as Zipful and myself Norman and We are still not at the end. We are happy to receive contribution feedback from you Also, if you want to work with us, you can join the team Talk to us on Slack. That's the Slack channel bit service and And yeah, I would say that's it bit service is ready for you. So thank you. Thank you And yes, we have enough time for questions. That's right. Oh, that's one idea far away Thanks. Thanks a lot for for this presentation and a demo. I was wondering now that the droplet can be saved as OCI image in their way that the user could Then we found that in their docker support in Cloud Foundry as to get reproducible parity from non-prone to prod Currently when using these packs, they had to download the droplet and then use the binary build back to run that again there thoughts with the The Digo team and the CAPI team to make a consistent experience that the user can do a safe push get the droplet saved into bit service and then reference this image on the next CF push using Docker support What you asked or should I say the question was is this OCI image what you get from bit service Has this the same state like the current droplet in in what would Jago represent? Is that the question? Can I use this OCI image as a source from doing my safe push with the minus docker option? Yeah, okay Good question. Yeah Since so we're very early in this process. I don't see why it would not work On the other hand, I'm not sure yet if if we'll make if we'll expose this image to to clients But it's certainly possible. I wouldn't yeah, I think it's possible I think we'll need to have more conversations how we want to do this But might be possible Thank you any more questions All right People in the back prepare your next question Fun run was not enough for you, right? That's what I just thought one question Can you elaborate a little bit about the scaling? So should I have one bit service VM pair 10 API nodes or something like that? so We don't have good experience with that yet The way we run it currently in IBM is that it's actually co-located on the API VMs So it's co-located with cloud controller and that was That's actually also for now our long-term plan because the feedback that we got from the rallying team is They want to reduce Vm footprint as much as possible right now when you use the F deployment it'll create a separate bits VM But the plan is to merge that back into the API VM and this is also our setup in in IBM currently That being said my hunch is that we can do with much less bit service VMs than cloud controllers, so For now where it's still a separate VM. Yeah, I think like you can probably go with Maybe just three depending on how big your environment is obviously But I think you can go with much less than cloud controller because it's there is as we showed the performance is extremely good And also for if it's a non local blob store We just provide a redirect on all get requests and so it's served from S3 for example anyways Yeah, but long-term it'll will merge it back into the API VM that at least is the current plan It's again, huh? Yeah Yeah, should I move back first? I Like the metric stuff. So I think it definitely helps for troubleshooting and we had our experience also there Question in the same direction. Do we also have tracing of the request that go to the back end like Be it Swift S3, whatever So it At least at the moment. It's not really transparent if something goes wrong in this place. Yeah We don't so for As three for example, we have kind of a low-level logging Something that is actually provided by the AWS Golang SDK and when you enable that you get like the specific request that was made against the back end for open Stack we don't have something like this at this point because if I remember correctly There there wasn't like something out of the box to use if there is actually That we'd be happy to integrate that because it's super helpful But yeah, so right now It's we have this kind of low-level tracing only in S3, but we'd be happy to add it. Yeah Any more questions? If not, I would have one but like you showed how to enable the bit service in via Bosch essentially and the ops file What if I already have my blob store set up cloud controller set up Etc. Etc. Can I just add that line and assume that like all the blobs Continue to exist. You can just add it. Yes, exactly. It's all compatible and the nice thing is even if something did go wrong You you can just remove those ops files and deploy again. In fact There is a little flag in the cloud controller a bit service enabled true or false You can just set this to false that will mean the bit service is still there running But it's not being used anymore, but we have hundred percent Compatibility between cloud control and bit service Awesome. Thanks Any last questions? If not, thank you very much. No questions. Thank you