 Good morning. Good morning My name is Daniel I'm from the Netherlands. I'm the head of the platform engineering team It's spring in nature. I'm here with my colleague. I'm Simon. I'm a platform engineer in Daniel's team So, yeah It's not working. Ah now it is So as said we work for spring in nature We're a large research publisher One of the largest in the world We're a global company with offices everywhere. We have about 13,000 employees hundreds of developers We have some well-known online platforms such as spring a link buy in the central nature Scientific America just to name a few The story. However, it's not about publishing. Don't worry. It's quite generic and I think it will apply to many of people here in the audience, so Let's get started Yeah, so it's basically a story about how we took deployments From weeks to minutes really and how we prevented a lot of snowflakes So to set the scene a little bit, I think it's good to go back in time The story begins when the company decides to bring in one of the biggest online platforms we had spring a link at that time it was hosted By a third party, but also the other development was done by the third party support And by bringing it in we've got quite a big challenge Because not only had we to do all the development, but of course also the operations and actually hosting the physical stuff So yeah quite a challenge Yeah, so we decided to do this project with agile methodologies We have the ThoughtWorks and we use Scala everything was super cool and awesome Yeah, so the company realized we can do this in-house So more projects started to pop up all over the globe and this is great Unfortunately though the ops side of the business IE us had not at this point gone through the agile transformation So we were kind of drowning Yeah, so To paint the picture a little bit better of what it was like back then to go from an idea to production I want to run a little scenario by you so you get a bit of an idea So with any good project, where would you start Simon write some code write some code cool What do you need? Need to request a VM to put that code on yeah need some compute resources make sense and anything else DNS yes, the VM sure. I'm with you almost done NFS to store random stuff on will be nice and FS and FS Okay ready to go live or Yeah, except for the firewall in between NFS and the VM. So I need to request a change though Okay, you need a firewall change. So you can just walk over to the guys right and Request a change and then we can get going. They're not different countries are just taking a bit long. Yeah All right But now we're good, right? So it's working. You got the connection to the NFS app is live Yes, except for the public IP. Ah, yeah, people can't reach it. So I'm guessing you have to contact the guys in The Netherlands again. Yes. Unfortunately. Yeah. Nice. It's gonna take quite long now. So I really want to push this thing live Okay. Yeah, so we've done all this stuff manually up until now. So we need to do some configuration management Configuration management. Yeah, that makes sense. It's quite an important product. We expect a lot from it So but now we're good to go right now. We're good to go with nice VM one VM So we should probably think about nice another VM Okay, I'm guessing you have to go through the whole shebang again So now that we're finally live I got some complaints it's freaking slow. So Maybe you can scale it up a bit. Yeah, if you give me some more hard word, I'd be cool Okay, so we need to order more hardware. That's Probably gonna take weeks. Yeah No, this is not working right Think if we continue like this we go mad So, yeah, this clearly had some consequences we saw a lot of frustration in the company and People started to work around these issues and everywhere Silo started to pop up people created their own solutions lots of local optimizations which result in duplicate systems We saw graphites everywhere. I sing as people monitoring the monitoring systems One team was using chef and the other team was using puppet then Ansible was the new kid on the block So and suppose overriding puppet and all kinds of fun in the end unreliable services and What do you do? So I traveled quite a bit. I went to the various locations when we had our development teams and There were the ops guys embedded in the development teams. I met Simon So we had many conversations and we thought we can do this differently maybe it's time to bring all these ops people together and Think a little bit more globally. Maybe we can provide them with one solution and deduplicate all of this mess and That's when we created a new team platform engineering We we had a mission to make sure that people could release more often We're gonna fix all the mess and we would be the new heroes in town Nice and we would make everything go faster, right? Ah, yeah Yeah, so I mean we wanted to build something really reliable scalable and repeatable kind of like the six billion dollar infrastructure So yeah short recap on that I think Brought in-house development from outsourced Super nice, but it resulted in local optimization and silos, but the platform and it here and the nearing team is formed. So that's nice sweet But now what? Well, don't worry because we have the platform engineering team and we knew exactly what to do We knew of course what the people wanted So we went into the cave Had a meeting in Berlin spent three days with each other We listed out all the issues that we knew that were there and came up with lots of solutions I think that was pretty awesome, right? Yeah, it was super cool as Daniel mentioned earlier We came from different locations from different development streams So we knew where all the pain point worse for both us ups people and for developers So we knew where to focus and clearly what we needed was Faster horses faster horses obviously it's stupid to wait You know days for we BM and it's stupid to wait DNS changes for days We had crappy I made Configuration management, so we wanted to fix that so it's gonna be super flexible and awesome So we went into the cave again and started to walk And it was going pretty well. Yeah, we came up with some pretty good solutions until Services What? Yeah, yeah people started talking about microservices and And We knew we were in for some new trouble All this nice configuration management and all this nice automation Would have been nice with a couple of monoliths but when you start thinking about microservices and things gonna explode into 200 300 or even more little services That's obviously not gonna work. So we were on the wrong path and we needed to rethink our strategy So we went back got together and we decided to do things a little bit differently this time around Yeah, so instead of going into the cave this time We went out of it to talk to the people that we were supposed to help Turned out they don't really care about VMs and configuration management and other cool stuff They only really cared about They all code. Yeah, they just want to run the code, right? Yeah, wanted to take their application and put it in production as smooth as possible basically So we asked them is it a pass you want and they were like, yeah pass we want So this is where Cloud Foundry enters the picture night in shining armor night in the shining armor So we started to play around with the different pass systems that would out there Like Dyn and Flynn and days and all these cool docker based ones and open shift from Red Hat and obviously Cloud Foundry But unfortunately it was pretty clear from the beginning that the all more or less wanted cloud Yeah, something we didn't have our API's was emails and phone calls Yeah, so that was a bummer for all of them besides Cloud Foundry Cloud Foundry obviously has Bosch, which is super amazing and You can do all the multi-cloud Busworthy stuff with that Which was super nice because we could actually deploy Cloud Foundry in an actual data center where the actual applications would be running Which is important, right when you come from a kind of old-time infrastructure the application starts to take Weird shape right where they require millisecond latency to SAP and other back-office systems and databases So going public wasn't really enough option for us Yeah, so I think after all the spiking we were pretty confident that Cloud Foundry was the way to go for us we Created a series of internal presentations made a business case. We got all the people aligned everyone was excited So we got the green light. We were ordering the hardware with the upstreams wrecking it with deploying Cloud Foundry It was fantastic, right? No one was using it so We built this platform. We thought yes, we're The right track, but turns out no one was pushing any apps. It's just basically basically a big dust collector Well, I think there was a couple of things missing So the developers had a lack of trust in the platform. Is it production ready and also a lack of knowledge? How does it work? But more importantly, I think the problem was that there was business as usual Developers as you know are paid to deliver features and there was some work to be done at the code to refactor it and make it Cloud native so it runs in the platform So we know we had to do something so we started the selling Yeah, we of course use it internally for our cool stuff But what are one other people to use it as well? Of course, so we started recording videos and send them sending out through the company We started using subliminal messaging so every time we would get a request for help We would be like, okay, we will help you but Cloud Foundry in the corner is pretty done awesome You should try it out and at some point we just stopped helping people and we're like no Use Cloud Foundry for your dashboard Come on and it worked. We got some initial adoption people were happy to put the non-business critical stuff in there Yeah, we got some small traction, but then our little micro friends come in to save the day Where before we thought we were in trouble now we were ready No coincidence that one of our biggest open access platforms Was gonna be rebuilt from scratch they're gonna do it completely with microservices and Together we decided to deploy it on Cloud Foundry from day zero and it was really good. They got so much Improvement fast feedback. They were able to push to production without any dependencies to ops teams This inspired other teams to also look at their Monoliths and starting to peel off little things and bits and pieces So yeah, we got some initial traction So we started off with the faster horses, which was clearly wrong decided to listen Doing some research we came up with Cloud Foundry and we started selling it and we got some initial adoption It's good It's not yet great Yeah, so obviously with new tech new problems arise So yeah, Cloud Foundry super awesome. You can push really quickly which also means you can push snowflakes really quickly into production So we saw this quite quickly where new teams would pick up the product right Cloud Foundry and start building their own Automation around how to use the platform how to do their own zero downtime deployment rollbacks all this kind of jazz There were also problems for people because they didn't really know how to get logs out of this How do you get metrics out of this? How do I monitor my applications inside Cloud Foundry? So we solved that we got all the tech leads together from different departments and Discussed like how are you using the platform? What are some kind of generic trades you share and We created something we call the spring app anatomy. It's basically The 12-factor app with some extra constraints around it So today you can more or less clone a repo in any language of choice or Framework of choice and be really up to speed in minutes. Yeah, a new project, which is cool We also did some stuff ourselves, right? Yeah, so we we started generating pipelines for people We started giving monitoring for free So part of the app anatomy states that your application must expose an endpoint said endpoint must Have a JSON document describing how the application should be monitored and who should be contacted in case something goes wrong We started generating dashboards Logging logging big your case tech logging for free also some nice stuff. Yeah, that was so good So that's a little bit where we are today So we would just want to run some numbers by you So you get a bit of a feeling what we do? So currently in our sec we do about 700 deployments in 24 hours to development and production We run about 1400 apps Which to get a do three thousand requests per second and so global traffic It's all nice, but the real cool thing is that it has a very small footprint if you compare it to where we come from It's just a few physical boxes Spread across two data centers It's a very generic implementation Yeah, only three two to three people have to maintain this stuff That would have been impossible. I think if you look at this Comparison yeah, I won't rehash the numbers, but as you can see faster horses kind of sucks compared to Cloud Foundry Everything is really nice. We can support whatever we want in Cloud Foundry You know you can use any language you want how we try to do that with a faster horses approach We would have ended up with some pretty gnarly configuration management But the real benefit I guess I think is the fact that we can just destroy our Cloud Foundry environment and bring it up again really quickly and we could You know respond to security Frets really quickly. So we just patch everything without developers even knowing it Yeah, thanks to Bosch thanks Bosch everything is groovy in Cloud Foundry land. Yeah Something else I want to mention is that we we did see a from scratch So we basically use the community release directly upstream Which I think is quite cool for a large company a spring and nature is that we have all our mission critical business applications on top of community software Supported directly by our own people and relying on the support from the community I think it's a big change and I'm really proud of it. So that's quite cool and Open source has more benefits for us, right? Yeah for me It's really cool right to be able to look at the code and the issue trackers and be part of the community You know from mailing list or slack. It's like yeah, it's proper open source, which is super cool And of course being part of the community means that we can contribute back Which we've done, you know submit a pull request here on their open source projects They're all on GitHub if you want to check them out Yeah, that's good So the whole journey has learned us a lot made loads of mistakes. I mentioned a few but they're trust me there are more The biggest I think take away is that everything revolves around trust We couldn't have done this. I think if we hadn't put the trust and the power in the hands of the engineers So I think it's really important that you start to build trust first before you start building systems and just From a grasp of movement you can and you can basically Change the way the development and operations works together I think if you get all of these things right you Can really go from weeks to minutes Instead of having yeah, horrible big monoliths. That's deployed maybe once every twice once week You know with lots of local optimization and snowflakes all over the place You can have nice to something. That's more appropriate for 2016 Lots of small apps deployed all the time Global standards who doesn't matter if you in Pune or New York or London or Berlin you develop and deploy exactly the same and Yeah, you can do this all from a grassroots movement So at no point in time did any CEO or CTO come from us because they read something some cool pamphlet in CTO weekly or whatever We did this from the ground up truly, which is pretty nice. Yeah So to give a little bit of an impression what it's like today If you have an idea you want to go into into production Where would I start with my idea write some code write some code and I did some work. I think I need a VM, right? No, you just type see if push All right Maybe I need some DNS. You're right No, that's handled by the platform. All right. No request needle, but I definitely need some storage, right? Some NFS from my state sure But if you stop relying on NFS, you will have a much better time. Let's do some object storage or database or whatever Makes sense. Okay, that's cool But I want my app to be available. So that's probably some firewall involved No, you just bind your application to a specific domain and you're online on the Internet's school That's awesome. Did you use a lot of configuration management to get this done or not really none? It just works, you know, cool. Cool. So this is great, right? I have my app I didn't need anyone I was alive, but now I need to scale it. So now we're probably in trouble, right? Now you type see if scale so if you don't have RSI, it's fine. Okay That's awesome. And if we have more hardware, you just Yeah, we rock it. We add it and push the button and Impressive. Yeah, I would say that's pretty bad ass Yeah So yeah, that's Our story How we went from weeks to minutes. Thank you all for listening. I want to thank my wife for the slides She's probably watching at home. So thank you