 My name is Alex Williams. I'm founder of the new stack and you're here for our pancake breakfast And we take these pancake breakfasts and our pancake robot around the world really. This is our first time in Barcelona But it basically we what we like to do is just start the day with a discussion about pancakes pancake robots and the topic of the day What I first like to do is just you know, thank everyone for coming to the you know coming here this morning And also giving a big thanks to Intel who's been sponsoring these pancake breakfast So we really appreciate all their support. So thank you very much for that it's interesting to you know to be part of the open sack community and following it and what's interesting is Intel has actually been a big supporter of open sack for a long time and they're also starting to get involved in lots of other projects such as you know such as kubernetes and So the topic today really I want to talk about You know is the idea of open stack The idea of the programmable web and these new technologies like kubernetes and how our companies and how organizations Learning how to use these new technologies to really start to you know to to scale their their operations and their businesses The the challenge often with with this is just this tyranny of choice There's so much to choose from out there and and if you look at it, you know You just can see across the different projects here at open stack how that's represented But before I get started I wanted to thank my team here at the new stack for joining us and Joe Jackson Our managing editor is manning the pancake robot and we also then have our team here Judy Williams the owner of the business the better other half of the business Wong din and we have Norris D. John on our camera. So just come up to say hello any time where it will be We'll be all around about I'd like to introduce our panelists now and We'll start with Dan. Hey Dan Dan Bode of Intel Ricardo Ricardo Russia of Sarin Robin Bergeron of Red Hat and Tim Bell. Hey Tim So I was thinking actually about this topic of the this, you know the programmable infrastructure, you know And you know, how does it get done? You know What are the choices that People are making and I was just kind of looking at the you know the projects out there. We have Cola now Right, we have Magnum We have courier and I was looking at a new project called Carbore, which is a new open-sack project for migrating persistent container data across different clouds So we have all these different kinds of capabilities and you know and now there's questions about is it open stack? Is it Kubernetes is it Kubernetes on open-sack is it open-sack on Kubernetes? You have all these different other kinds of options out there. How is anyone supposed to make sense of all this? Robin do you have any thoughts on that? It's a big tent. It should be very easy, right? Yes, the big 10 idea, right? Yes So the funny thing about Cola is is that it actually started out as Just deploying with Ansible and not Kubernetes Which is if you've seen most of the demos and what people are talking about that's already working and making them happy That's all Ansible based. Can you tell us can you explain what Cola is for folks who are not maybe aware of it? Sure, so Cola is I may have picked the name just a yeah full disclosure It is a project to deploy Ansible or to deploy open-sack Services in containers docker containers using Ansible. It's basically a very large set of Playbooks and it's got a gazillion contributors and a lot of people who are quite happy with it And so and so it sits it sits underneath it is that it is I understand it You know it also is going to be integrated with other services, too Yes, so as I understand it now people are wanting to do a Cola and Kubernetes type of thing and I Think it's going to be fairly separate from Ansible like there's a set of containers You can orchestrate them using Ansible or you can now orchestrate them using Kubernetes Okay, once once that parts written so Dan so so your perspectives on this, you know So so you guys have been very involved in the Kubernetes ecosystem, but now we're starting to see this deeper Set of complexities arise and I think you guys have been you know looking at that through projects I just snap right where you're starting to think you need to think more about You know what's coming out of the data center and how do you process that data? How do you collect that data? How do you process it and how do you visualize it? Tell us about kind of the context that that Intel has about this idea programmable infrastructure and Kubernetes and and how projects like snap fit into it so For me when I think about programmable infrastructure I find that the types of systems that I always try to build are systems where you know you can write things as code or configuration where you can Describe your entire infrastructure in a revision control system so that you can do code review and really understand and have this log of Who changed what context of conversations around thing? You know, what are the things changing and then have a built-in review process that you can use to understand? you know what changes are being made who who's making the changes and why and This is this I would say is for laying down kind of the base platform For what you're actually going to build, you know start to look at tools like snap and one of its main purposes or functionalities is To embed a lot of the logic that used to be considered as configuration into the application itself so as opposed to having this whole out of band cycle of You know, oh, I've decided that my business requirements have changed therefore. I must you know change something for Configuration I must review that I must put it in and then eventually it'll roll out to a system snap was designed So that the actual configuration is something that's distributed automatically by the application so it's something much more dynamic and for us the way that that starts to tie together is that You're going to have a specification for how to build a platform but the configuration of the characteristics of that platform is more of a feature of the platform itself and to give Examples of the kind of things that I think this could enable in the future would be to Tie telemetry data into something like machine learning where it can become a feedback loop Where the data coming out of a system can be used and then interpreted by another system to determine how to make Modifications about what data to collect so so so this is essentially a telemetry environment, right? Yes, we're processing data and understanding what the data is coming off all those machines Yes, I'm able to then visualize it at CERN you guys have a Have an incredible complexity that you're dealing with now you you're dealing with the data of space essentially, right? And you know and a physics right like thinking about the data of atoms, right? This is a whole level different level of complexity that you guys are facing But you guys are you guys a distributed infrastructure? Would you consider to distributed infrastructure and what were you kind of your some of your concepts you were thinking about when you were When you were building out your platform because you have you know thousands of I don't know tens of thousands of Bare metal servers I expect right and then you and you don't use a virtualized I'm actually use a virtualized environment Maybe you saw us what it you know what it really is But what are some of those kind of the things that you were thinking about when you're building out this infrastructure? And what are some of the things that you're thinking about going forward and each of you guys can just chime in when you feel free? yeah, so I Think when we started looking at the challenges that come along with the Large Hadron Collider Data rates one of the focuses that we were doing as we knew we would have to scale out Was how you could go about scaling the number of machines without scaling the number of people that are running them? So basically setting off from the beginning to automate Knowing that we were needing to scale out and part of that was a question of then moving to automation software defined interfaces So basically making it so that where it is possible Automation would apply before any human gets involved and with that we were then basically able to scale a platform about five times Without increasing the number of staff. So how did you decide on open stack? And what did you guys get involved your brother earlier? so we start using open stack around 2012 right and In the end with all of these things if you have a toolchain that's reasonable then you are able to swap in and out different technologies as things evolve So open stack at the time was very young, but quite promising. So we tried it out and it worked So we carried on but the aim with all these things is that if necessary you can change as you go along Riccardo was so now looking forward with with with with sarin What are some of the things you're looking at to kind of you know to to extend the infrastructure to you know to deepen its capabilities to automate it further? so Now we have the basic infrastructure working already for a couple of years. So people have been doing a lot of their workloads on VMs Where I'm mostly focusing now and that certain there's a lot of people working on this is moving the typical workloads of physicists to Containerized environments The reason for this is that usually to kind of package and prepare the applications and the analysis of all this data We produce it's kind of complex and the experiments and the end users need a lot of help to do this So we have this opportunity now to have an easier way to have single units That we can give to people and they will very easily launch them in their environments and get immediately An environment where they can focus on their work instead of having to know the details of the infrastructure that that's a lot of the work we've been doing just making their life easy easier and While scaling out. So these are the two main things we get from containers like simplified Deployments, and then the ability to just scale out With those applications that now that you're just able to hand them this little bundle that's pre-configured that I don't know If you're doing scientific things it might seem to be wise to have the Environment in which the data is being processed So that people aren't questioning your data Yeah, a lot of people doing their work in physics area. They do their PhD. They end up being also computer science Scientists because they have to know a lot about details of the infrastructure We're trying to break this so that they can just focus on their and so so the topic of containers You know, what is the context Robin for you about containers? And you know and when you're listening to what CERN is doing what Intel's talking about what you what you're hearing from customers, you know What is the context here with OpenStack? How do you see that the themes evolving? What are what are the some things that you're listening to that you could share with this group that you think are like worthy of you know crystallizing so I think one of the things that I see Commonly and I think maybe some of this is you know Speculation press stuff, you know, there's this whole like there's gonna be serverless and you know, there's not gonna be servers anymore Occasionally reminding people that like yes those things will actually continue to exist to provision these things and you will still need to continue to manage the configurations of them and you know, just like you were saying like Yes, we can all willy-nilly go off and go on to Docker hub and download some things But you probably want them to be consistent and first, you know They those things can still be managed in some sort of configuration management type of of way and this is what we're seeing with You know in the land of Ansible like people were doing, you know basic, you know provisioning VMs and Talking to all their infrastructure and you know, it's a very nice abstraction layer now people are abstracting it You know abstracting all their containers through Ansible and configuring them that way But what about it's sarin it's sarin. Are you thinking about containers? You start experimenting with containers as they're an efficiency that you find you can see with containers that you know that You would relate to your scaled-out infrastructure So we are running the the magnum project On the CERN cloud and what we found attractive about magnum as a framework was that we'd already got in place the open-stack Infrastructure, so we had the project frameworks the workflows all that extra tooling that you have to invest in around a service and we could then plug magnum into that and Provide the users the flexibility of choosing their own orchestration engine CERN users Very technically advanced and to have a central team telling them you must use Kubernetes or you must use mesos was not going to be an easy Cell Whereas with magnum what we're able to do is to tell people make your own choice But we provide the underlying framework. I think it's more at the moment as a question of flexibility compared to efficiency Okay, flexibility is really is the key here and I would say you know, you know speaking specifically about containers In the context of Docker for me It's really two more features that definitely You know allow you a lot more efficiency. It's both an overlay file system, which for folks who aren't aware It's really this idea that you build layers and change of change on top of each other And the really amazing thing that I've seen I'm actually running on production applications in Kubernetes right now And like one of the things that I find to be the coolest is if you have that shared base image of that file system As soon as a workload hits a specific node, that's there So you have you only have to pay that download cost for all your bases once and then after that you just download the additional layers You know one other thing that I that I've really found to be great about Let's say packaging things in Docker containers because it's really just a packaging format is I've found it also reduces Entropy a lot just because it can easily make that guarantee of What instructions build what things then knowing that those things are on your system as an atomic unit? Which I've found definitely reduces the amount of entropy that I see That's a that's a cut. That's a concept. I haven't really heard people talk about is that entropy issue How does that relate to you guys? So what what you're thinking about and doing in your work? Entropy you were mentioning you're talking about entropy as an issue within as it's something that Kubernetes helps and containers helps resolve Definitely, I think definitely having the ability to package More things together simply and then have that be an atomic unit that gets placed. I have found reduces entropy Yeah, the same the same for some applications I was talking about and user applications more but also for the infrastructure the ability to package everything in tiny year Well-defined components allows people to scale out much more easily That's what we're seeing most of the interest we have is from services that need this scale out Levels and that will vary this this level of service with time So that's where the interest okay, we don't have a lot of time Does anyone have any particular questions about containers and and their uses in you know, you know in their own environments questions? They have Rob Herschfeld could you of course Rob does? Here you go So one of the things that people like to talk about with containers versus virtual machines is efficiency density You know better utilization of the underlying serverless infrastructure Of the of the actual infrastructure. Can you talk to what you see in that? I mean in regards to the snap project and then CERN and performance. I'd be really interested I Would say I don't have a whole lot of data on on you know We're pretty early and in our phase and not quite to a point of understanding what our density is going to be Especially relatively to VMs, but there's definitely you know should be less overhead And in terms of things being able to share more of the underlying operating system But I'm sure the guys at CERN probably have way more specific data So we certainly had a look at things like a virtual machine overheads versus containers and certainly it is interesting to trim off the Few percent that's currently there is overhead On the other hand when we look at that as a percentage compared to the overall people efficiency that comes from from the framework Then actually we find that that is a reasonable overhead to be paying for the extra Person automation that you get out of it and when you say framework you mean open stack Yeah So in those terms by having something which is allowing us to deploy More with less people then the actual virtual machine overhead for us is a small factor But something that's always good to have a look at it actually reminds me of a phrase that I came across when I was doing Research into you know things like alpine versus a bunch of some of the busy box Tiny containers and and the quote that I like is disk is cheap. My time is not. Yeah Yeah, this is cheap. My time is not Yeah, I tie time is never cheap, right? So so Robin as you're looking forward in the market, right? And you're seeing the rise of of These these different container type technologies, you know What what is the what is the route that you see with Ansible where you know? What is the direction for Ansible over over that? So I think part of it is you know similar to what Trent was saying just because there's all these fancy new things does not mean that You know unless you're a unicorn that just got a billion dollars of start-up money You probably already have things that were you know, they became legacy infrastructure five minutes after you you had them basically like they're they're there Ansible's a great abstraction language I'm here at open stack summit. I do my best to work You know we all do our best to work very well with other communities There's you know 400 plus Ansible modules that are all Related to every other open-source project on earth and being able to work with them So how does that relate to kubernetes then an open second the work that you're that you're that you're seeing inside red hat And what you're seeing with your customers We've got multiple folks who have been working on like kubernetes modules. We have a project called ansible container that where you can literally just type ship it and it will take a configuration of Docker containers and push them into kubernetes and be ready to talk to each other Which is you know fun and interesting although we do tell people like we're still very much in a feedback gathering phrase Please don't ship this into production. Please tell us if it sucks or not so that if it does we can do something different But a lot of it's just following you know looking at what our users are doing like I would love to say that We are all very brilliant and know everything but we're the developers and it's you know Yeah, fabulous actual users who are doing things that tend to come up with the creative solutions that they need to get stuff done Can I take a poll? How could you raise your hand if you're using containers? Oh? Excellent well and would someone be okay and telling you how you're using them We'll give you raise your hand just quickly and just you know No, I know Rob will is anyone else out there could talk talk a little bit about You see anyone out there. I know I saw several people raise their hands. This would really help everyone a lot Okay, that's okay One of the cases that we're looking at that's really interesting is using ipython jupiter notebooks Which are these technologies that allows you to combine both your programs your data and your research paper into a single unit and Then with this we currently are making available CERN data from 2012 Through our open data project and then using packages like this. You'll be able to at home do the same research Oh my gosh, so like so we could actually start like helping you like split out of you can be a physicist. Yes Hey, that's a project for us at the new start splitting atoms That's amazing. So that that's an example of a real that that's a distributed infrastructure right, huh? Any other use cases? Yeah, I can say my use case is I'm building a testing platform for development teams And when we think about our success criteria, it's all about Running tests in the shortest amount of time possible and we do find that we get into end-to-end time improvements by encouraging our developers to run their tests on containers when they can Okay So well, there's some work and not necessarily from the experiments at CERN, but also some some small groups We have these massive detectors and a lot of the data selection is done very close to the detector Typically using hardware and electronics and some farms very close to the detector and people have been looking at Training neural nets to replace all this hardware and all this complicated infrastructure with smaller units That could make this filter easy easier and they've been using this Distributed infrastructures and container clusters and GPUs to train the neural net and just use that model to To make it much more efficient. Let's conclude with you as one does No, I think it's you know From my perspective, you know, I think all of the new technologies and new exploration is great I think you know the the thing that I always worry about is like We're all committed to this thing and it's the thing that we're all going to use and then like in two weeks Like there's there's something else and we're all committed to that too And then there's another conference and then you know, we all have to go to that and and You know then my kids are flirty and wondering where their mother was but you know in all seriousness I'm glad to see the amount of you know Exploration that people are doing and I'm glad that the folks are actually getting better Doing open-source stuff and actually participating in all these communities to you know Give the feedback to the you know folks that are doing that all man They're participating in all that work because that that actually means it might all be viable That's a whole other discussion on the role of open source and all this and I think that's kind of the one of the themes We're really hearing a lot about what you're saying is this richness right and these new projects that are constantly coming out and That is such a complex, you know world to live in And the clarity here that you guys are helping provide is really valuable. So thank you very much for for coming and joining us today Dan Ricardo Robin Tim. Thank you very much And I really want to thank Intel for sponsoring the breakfast here and thank you all for coming out We have to run they need to flip this room pretty quickly So I hope you enjoyed the pancakes and we'll we'll see you again soon