 Very good. So welcome everyone today, we'd like to Welcome you to participate in a panel. It's not just us talking to you We'd like to hear from you. We'd like to have you participate in a panel which is talking about the experiences of carry service providers in Working with OpenStack first experiences second experiences lessons learned and then we'll head into What are the types of things we want to see from OpenStack and what are the types of things we want to do with OpenStack And sort of explore the futures as we move through. We have 40 minutes. So we'll keep it pretty brief Do feel free if you want to extend a question or if you want to challenge and answer To jump up and do so So my name is Christopher Price. I work at Ericsson. I'm a board member for OpenStack and OPNFV and I will be Trying to give these guys a hard time during the day I'll let them introduce themselves now starting with with Adam Hi, Adam Dunstan. I manage Estonian and NFV engineering. So Kind of rare. I'm a development manager, not a labs architecture. You kind of guy Norange and Avila have a fellow at Verizon network planning and I'm in the network team that designs the architecture for Our cloud platform and evolution planning for the Verizon wireless network My name is Greg Stigler. I'm from AT&T For the last three years. I've been charge of our cloud our OpenStack cloud AT&T integrated cloud In the last few months. I've picked up additional responsibility for our domain 2.0 virtual network function platform Where I have integrated testing and I also have integrated design for the entire platform Very good. Thanks gents So just to briefly introduce the topics we get forwards We talk about telcos as being special and then we tell them they're not special and and it's kind of confusing to understand What is the telecommunications requirement? Why is it a challenge and how does it differ from someone else trying to run a data center? I think If you take it from a telecommunications perspective, they have a lot of infrastructure They have a lot of new capabilities they need to bring in and old physical infrastructure and capabilities They need to replace They have supported by networking challenges Which then need to fit into an open stack based cloud architecture On top of that they have existing operation systems. They have new operation systems And all this has to fit together in some way to deploy the services that we as consumers of these Services rely on and enjoy using So from the perspective of trying to fit OpenStack into what is already a very complicated architecture and and well physical And services architecture I'd like to sort of get into The questions around OpenStack So the first question I guess to the panelists Is When did you start working with OpenStack? When did you take it into production? What were the challenges and the main hurdles that you had to overcome? As you took those first steps and first brave steps to bring things things live with OpenStack Taking volunteers Well, I can get started So we had a small group in Verizon We started looking at OpenStack back in 2013 time frame It was around the Havana release that was the first one of the first releases We started looking at and we had a very specific objective in mind We had a specific use case around packet code We were looking to virtualize And we're looking at what are the deficiencies In the current feature set and what do we need to do to get get it deployed and one of the first things we Stumbles we ran into was around IPv6 Because we wanted to build a cloud that not only supports Never Calamants that you know that run on IPv6 We wanted to build infrastructure itself on IPv6 That was the one of the first feature That we wanted to get implemented in OpenStack And we honestly were quite pleased, you know, it was one of the first Good things that came out of OpenStack We got IPv6 and we have our Verizon cloud platform deployed on IPv6 today Yeah, yeah, yeah Cool Many many moons ago We were a founding partner, I think on the foundation Well, so we got started back with Diablo And uh, we we talked the other day at uh at at our prep session about I didn't know what was before Diablo And I think we figured that out. What was the sea cactus cactus? Yes, we didn't I didn't know that so But but there were people before my time that were doing Diablo because that was greater than three years ago So they they learned a lot from those early implementations that we were able to take into our later implementations today Uh, we're running anywhere from kilo, uh to mataka Yes, so we we I've only been with the company nine months And in fact, I've only been at I only have nine months of experience as a carrier Up until nine months ago. I was a vendor so 20 years so I'm kind of a different guy here But our journey at century link to virtualization started quite a long time before me We actually have a big complex of vmware Running virtualization as well as a number of complexes running Quite a number of complexes running running open stack and We're on our third kind of major release with a lot of minor releases in between that that we're rolling out kind of right now So that complex that We that we the first kind of major open stack system we put into production was uh, you know was last year, but We started this journey in 2011 I can looking over at someone who's trying to nod 11. We started the journey in 2011. Yeah, my rena crowds here, so All right, so following on from that and then Nine months in have you had to upgrade any sites? Sure And tell us about it. Look, I mean the I mean the You know, we talk a lot about turning things upside down As you talked about carriers and complexity, you know, we kind of reveling complexity Everything is so complex. We start these programs and we try to carry all these features into these programs We ask all these different people to get all this consensus around building this thing If you actually get enough consensus to get a program started by the time you've got it started It is so big and complicated That you know, you're likely to getting this thing into production is really really hard what I've what I've learned is You know, there's a lot about there's a lot more challenge in process than there is in engineering inside inside large scale carriers So experience in upgrading little things are easy. You know, little things are easy Our tooling isn't where we want it to be. You know, we want it to be really automated Um As we go through our into our next release, we're really simplifying our open stack complex I mean, we are taking it is bare bones as to what's in this thing And the purpose behind that is really to make it easy for us To go and get this into production and it's not just the engineering and building design and automation and tools It's how we get it through the whole complex of people. How do we improve the processes so our people can You know, take this into production that scale rapidly How do we march it all the way through such that we can easily integrate it with all of our parts and at the core of it's You know kind of simplicity. So I think we're doing it now um Our next upgrade is not going to be that easy Because we're jumping a bunch of we're jumping a bunch of releases. We're going to go to newton And we're jumping a bunch of releases about a whole different set of packages inside the thing We're actually stripping a ton of stuff out. And so it's not going to be easy I'm looking at the guy in the room who does this and it's not going to be easy yet. But you know, I think Tooling's the answer to this very good Greg From Diablo onwards, I guess you've had similar challenges. Yes, then and I'll play off of deployment and production too because It it was absolutely one of the most challenging things we had so um, I know the slide before mentioned the mega data center or something close to that But we're deploying mini clouds and in central offices that are close to our customers for latency reasons I think we're all That's nothing unique So that to be able to deploy that many clouds that quick The mechanisms weren't there. So we had to work very hard to develop those and Had worked back with the community on those the interesting thing is containers is taking over at a time When the mechanization we did before is really going to become outdated and we're going to move to the container containerized control plane as well. So Pretty interesting very cool. Yeah, the models, you're not quite different, right? I mean if you're doing a compute kind of thingy You sort of a bunch of controllers and maybe have a handful of these things spread across the place But we do an nfv and we have a we have a different seems to different things. We're not doing a unified cloud We have a compute and service cloud for the kind of normal computer-oriented stuff And then we have our cloud that we work on our bit that we work on is an nfv system. It's designed to run vnfster forward packets and it's optimized around packet processing performance And so it's a different puzzle and in the same way there's a controller and every at every every rack of stuff or every location It's a controller and there's a controller and it's a single entity running. So The tooling that we have to put together means that as we go and roll out sites We're not just adding compute nodes everywhere we go. We're building systems every time we do it And so I'm sure you've had to do tooling and we have had to do it too because that's not the the deployment mode That's more common in a generalized compute storage and networking mode when we're very network heavy Yeah, absolutely. I think the only difference from what we're doing Is we want unified code for the cloud So we made deploy different amazon would call them instance types You know for for compute power where there's gpu or memory intensive or cpu intensive So we really see the advantage being to have single set of orchestration single set of apis That manage both of those platforms and the only difference is the flavor series which toby ford To know it was it was our open stack foundation member and he coined that name. I don't know if it's taken off or not I kind of like instance types myself, but Yeah, we were very similar in a sense that the difference is that We have a common hardware, right? So so we're all trying to get to the same place We're probably following a different go path than you because they're all go paths in this stuff, right? So so so we have common hardware and we have common we have common hardware control across the complexes And so we're building common hardware. So someone who wants to run something that you know is We traditionally run an appliance for we don't have to do all the extra work If we just want to control hardware, you know, there's a lot of aspects to doing this So let me step back a little bit So one of the aspects of this is just common procurement, right? I mean you got to go and you go and do this to save money and Common procurement of servers is really helpful. There's all these different ways that you build up a business plan to actually do this kind of stuff But if you go and say, hey, I want to run some Particular very fixed function thing that used to run in an appliance and you got to run all these steps through open stack And it's never we're never going to run too many of them up and down They're never going to go up and down. They're never really going to scale We bind them plug them in we go well, we can get to that later But let's build a hardware structure. So we have a common hardware structure. So our our Our kind of management sales the guys will talk about we have one big cloud and we kind of do it's all in the same hardware But it has instances inside and they're not necessarily as uniform in our effort to try and You know build the right thing for the right job And I think that a cloud a common cloud can be built And I think we could do it. I think it's just a whole lot more work So it down our path towards simplicity And getting velocity we've said Let's build an NFV one and head down that path from a velocity perspective just one short comment for sure This strategy has not been an easy one. Yeah, because culturally it's difficult for different departments to get That's true, too It might that would make it even harder merge of network and it so Culturally to get them to think along the same lines of the same set of code But different instance types has not been an easy thing I think we're we're reaching a level of understanding across the I'm glad I haven't had to try and fight that battle. Yeah, I've had skirmishers with those guys already But I haven't had to go that far. I haven't I haven't Yeah, they've a very different view of what we should do, right So what to get to point about upgrade? I think upgrade is largely an unsolved problem You know, we have like Greg said we got many clouds. We have kilos Instances we have newton instances and kind of figured how we get from kilo to newton It's complicated with the fact that the infrastructure itself needs to be taken offline to be upgraded And the applications to run on this infrastructure there for the most part not cloud native applications today These are network functions. I mean we were running up, you know, these network appliances effectively appliances purpose built hardware And largely what we have today is, you know, vendor supported the software to run within virtual form factors But still it's the same software it comes with this, you know, it doesn't support elasticity You can't take it offline state is tightly coupled So taking sites offline doing upgrade that's that's a challenge And I think there's a combination of factors that need to you know, that will help us One is to Greg's point open stack is becoming containerized so you can leverage these new technologies To help but on, you know, us rolling updates with open stack and applications started moved to become decomposed to the microservices Decoupled state so the applications themselves become more cloud native I think that's the evolution that needs to happen for this to be truly seamless So on that topic then looking looking forwards We'll have purchasing practices we we strive for common hardware. It's not always possible, especially as approach an edge We talk about the need to be able to manage applications as we go through upgrades on sites And even hitless upgrade is is one of those telco things that we have always kind of had There are different ways to skin the cat in a virtual world How do you see this as we move forwards? Are you able to take advantage of Mobility which which technologies you find for application mobility would be the most useful in the context of how you're building your Clouds and and how that runs out infrastructure as a service layer Well, live migration is not the right answer. Let me start off with that I think application mobility really means Applications need to be able to tolerate failures You must assume that things will fail because they are going to fail So you must be able to take, you know duplicate state Locally, you know, gerund and fashion and you know, when you look at state You have to kind of look at one size fits all you have to apply the cap here You know, what are we talking about? What application is this? What state is this and what kind of consistency and availability do we need? So decoupled application, you know to into call processing functions state management functions And a unified front end and be able to you know leverage that I think that's part of getting to the cloud native You know to the entire get of being cloud native This is a great topic to what we just talked about before the difference between the cloud guys and the network guys, right? So it's a it's a great topic and this comes up when we talk about this type of conversation So so one of the one of the things that we certainly have in our network and I assume it's in other places as well Is there's been a proliferation of bgp, right? So bgp is everywhere And so so we we're engaged in an upgrade right now and we've got the sites all over the place And so, you know, you can imagine that there's one in Arlington and another one in in balsamore, right? And and from an upgrade perspective if we need to upgrade that complex All we need to do is figure out how to move the VMs between those two places, right? Because in reality, they don't really have to be in the same place now They're different complexes and they're managed by different controls. They don't be in the same place But for us in most of the use cases because they're networking use cases hooked up to our bgp infrastructure You can seamlessly move this over by just figuring out where to put it next So the way we did this is of course the solvers incomplete in the, you know, open stacks Layout is not going to help us with this problem. So we write a solver to go and do this And if you add a basic layout system to versions plus latency You can go and say hey, can I pick up that firewall and move it up the street to Arlington Put into the bgp topology You know fail the other guy it'll reroute everything's hunky-dory and then go and reinstall my problem So I think we think about this problem of as networking guys a little bit more than perhaps Bit more than perhaps our it our it and and And cloud partners do Because I think we you know, we think maybe we do maybe we don't we have a better understanding of the network and our ability to use capabilities in the network So I don't think it's solely a how do I make open stack do this? I think there are other ways to get this done and we've got to be a bit scrappy to make it work Because at the end of the day You can't get used to work if you don't scrap you about it We need help from the vnf vendors first and foremost I mean most of the vnf's that are being delivered to our table today are not cloud native Yeah, that's got to change and and we've updated our vnf guidelines They're out on our website. So go grab them, but We we've got to have vnf's that are actually microservices that we can scale Um, that's where the industry is that's where the web scale industry went And the network service industry is behind that. So we need Help in writing these things the right way I would also say we could do a better job with open stack internally when it comes to microservices and containers as well Yeah, amen to that. I agree. I think Network functions are largely direct ported, you know applications today And we're going to need you know need to break it up into microservices decompose them separate state from transaction processing and that's key to Elasticity to getting rolling updates going and so you know feature velocity, etc Very good. So before before we sort of pivot. I wanted to move to futures Or even as we pivot to futures I did want to open for the audience to get involved here and ask these guys questions We've heard what I think is three very similar but slightly different approaches to how to address A number of the challenges we have If anyone has any questions the mics are here and and I'm more than happy to have you guys get involved So I have to give these guys a hard time. I think that would be ideal But if no one's standing up then I would like to ask There we go So thank you for the discussion and please introduce yourself. My name is Andre Seller. I'm director of service provider architecture for infobox We're a vendor DNS HCP IP address management. So From so question and comment. So what I'm hearing is I hear, you know, definitely different viewpoints And I hear AT&T, you know, obviously Doing a good job in leadership here in in OpenStack Contributions broadly across all the different working groups But you know built a custom proprietary front end which is needed to interface with their infrastructure So there's a this drive for standardization That isn't necessarily being pushed to the rest of the world. Then I hear century link talking about You know, they've been done a lot of acquisitions. So so I would think that standardization would have definitely benefited them from a business standpoint If if we had adopted this, you know broadly or more broadly and then I hear Verizon talking about how, you know, they're very focused on real operational problems and how to solve those operational problems So appreciate the the candor and the comments But I guess what I'm asking is that doesn't it make sense that we have a Really embracing the standards approach. Isn't that the way that we can help leverage this moving faster? And then and and when we look at microservices and containers as the next evolution How long will we continue to support the existing infrastructure? In our current, you know, nfv the nf base solutions because I see that as being, you know A four or five year effort at least as we get containers ready to go into a service provider environment Can I do one thing first? You'll like this. You'll love this. I need a button So I'm thrilled to be here, right? I'm thrilled to be here because Most of the people in this room write code. I hope But since we're the carrier audience that's probably not the case, right? But most of them a lot of these other rooms write code. I want to draw a parallel for a second here nfv And kuba and containerism containerism. Is that a word? So Started at about the same time How far has containerization got versus how far nfv has got? nfv's nowhere The whole container infrastructure has gone along at a rapid pace. Why is that? They didn't bother with standardization They didn't go to babble fests where they talk about writing documents They didn't do it, right? And so so you're right things have to work together But and and there is a this lack of comfort around You know not having standards inside the carrier world and there's a balance to getting it right But I think we're responsible for our own destiny here, right? We are responsible for our own destiny And so we need to take responsibility for that destiny if that results in things where we collaborate together That's great. If it doesn't it doesn't we get economies of scale and we all want economies of scale But i'm not a supporter of this. Let's go to standards Yeah, so what what AT&T? Front end were you referring to the econ? Ecomp has been outsourced as onath Fully outsourced now, so it's out there for we'd like you to join the community in fact And you know write a check and be part of it so We would love that but but we have fully open sourced that it's been combined with triple o And I think we have open open open. Oh shoot wrong one open. Oh, thank you Somehow I still I think 34 percent's the number i'm gonna get in trouble with the media guys But I think 34 percent of the major wireless carriers across the globe have already joined So it's it's got a lot of momentum behind it In fact, I thought you might be talking about orm which open stack resource manager which we presented back in austin And I think the only reason that one's not an open stack because open stack hasn't consumed yet But we would be happy to give that as well So we this this is open and I'll tell you this is kind of back to the future Telecom companies have always done business together. We have had to have interconnection agreements and ways to do business This is back to the future what we're doing here But I but I do agree with your point We have to move and we have to decide and we cannot linger and I think there's a little lingering going on That's why we started the elku, which is the large contributing open stack operators The large is important because we need people who can scale Okay, we need to figure out their scaling problems and open stack. We need to fix that The contributing part is we don't want people who are just gonna Take we want people who are going to give and be part of the process And of course the operators part is well the vendors have led this thing for a long time And we want to partner with you. We need you But we also have to get our perspective heard And of course, I mean we we contribute code back in they upstream our code to ipam plug-in framework But and I understand it. It's a your wing, you know business opportunity Especially like in the century link example, right versus, you know trying to move forward quickly So that was why the second part of my question on you know as we move forward How do you see the The Containerization Moving forward in terms of what we're supporting today in an NFV Framework Yeah, you have a guy I got an opinion, but I'm an opinion. He can be easy side of the question I think containers is largely going to be an opinion. I mean I think the industry is spoken I mean, there is there is the darker world the kubernetes to CNC of world I think we'll largely, you know pick around converge around one of stores proper public communities But you know to a point about standards I think standards have a role like 3g pp and ietf We talk about protocols and interoperability between network functions between operators But this is an implementation, you know, where do we how do we deploy this? And there, you know, whatever we pick now may not be the best solution for tomorrow So we're going to need to experiment around it. So It's largely becomes, you know, what's what's our platform explain that platform clearly Lay out a roadmap and and you know, give you the avenues to come and test and then I excel in the platform You can solve a lot of problems with a container with no js in it Yes Very good. Thanks guys. We have another question No answered Maybe again, does anyone else have any questions? I would like to I would like to explore Some of the themes that have come up this week The themes around containerization containerization of open stack running containers over open stack How are you looking As you look forwards, how are you looking to take advantage of that? I mean, as you said, there's a lot of opportunity a lot of optionality a lot of ways that this can work In your mind's eye if you look, you know, two years down the track How would you like to see your machinery humming with the with the composition of containers and open stack and infrastructure of services and application management Well, uh, let me give you my view there I mean, we have open stack which is largely built to support virtual machines and a virtualized network functions But uh, like I was saying earlier, you know our vision has largely been, you know We got bad metal purpose build appliances today If we want to get to cloud native, but we can't just take the leap overnight and we you know We need to stepping stone and virtualization of the stepping stone Which is what we're executing right now But our vision has always been to get to cloud native architectures So which also means building infrastructure that can support cloud native architectures I mean, what is a cloud native application? Effectively an application is built around on the cloud efficiently So we have to have the infrastructure that has to think, you know We got to have a container orchestration engine be able to specify What's my deployment unit, you know, what whatever language you pick in whatever declaration language you pick in having, you know A log analysis infrastructure so you could expose your logs to that, you know having You know key management functions as part of the infrastructure networking that is seamless So that's you know, that's the direction we expect to go in that regard So Containers are extremely important Open stack is extremely important. They both go together. There was a great keynote The other two days ago by the google fellow, I forget the name But it he showed the layers and he showed the application layers kubernetes doesn't handle so there's our v&f right Then there's kubernetes that handles the containers and then there's open stack that handles the infrastructure as well So It's it's a marriage. It's a perfect marriage in fact. It's one that goes together very well There's always going to have to be something that pixie boots hardware that manages hardware that That does the things that you need to do in the data center and You know, what a great thing to have ironic and some of the options You can do to tag everything in your data center and get an automated inventory right away So so I think they coexist together for a long time, but they have to continue to come together I don't want that to be a long standards process But what I don't want I might not see it what I don't want I might not be I know But what I don't want are five different companies developing products and selling them in that space When we can come up with one way to do it. That's already pretty much there Open stack helm if you're interested So You want to know I go ahead So I was kind of one thing which is I see a little bit of divergence between the open stack world with vm So the container world when it comes to networking I mean, we need high performance networking. I mean, we're all never companies We do run high performance Elements that have high throughput high packets per second requirements So what we've done in the open stack world was to do things like sri We which give us high performance networking, but it comes with the set of operational challenges But now we're moving into user space networking, you know with dpdk ovs tpdk vpp and all the advances So networking has gone into user space Within open stack, but if you look in the container worlds, what are containers fundamentally? They're kernel objects, you know kernel c groups and kernel namespaces So everything around containers must be managed in the kernel when you move networking out of the kernel Well, it doesn't really fit natively within the container architecture So what I also see going on in the Linux world is kernel networking is making a comeback There's a lot of new promising technologies bpf xdp and so on where they're looking at high performance kernel networking So I see right now a little bit of divergence We're on the evolution path and we really need to figure out what is the you know What we need is a common networking infrastructure virtual networking infrastructure That is that ties in these two domains And I think that's a little bit of a struggle we have today. Where is where is this going? You know we've got to think two different paths. Yeah, I'm gonna I'm gonna go again Okay, I'm sorry. I got something to talk about but that was so good because The networking pieces is the weakest peaks in open stack and I'm sorry to neutron leaders and everybody But it just we didn't ask them to build it this way either right now We're doing it ourselves at different ways So we all are now because the layer three parts not included And that's where we have a bunch of five different vendor solutions. It's more like 50 but That's something we've got to solve for now gluons one way that's that's a proposed project to solve it I don't know if that's the best way, but this is something that's got to get solved for so we can module That's tough word. So we can plug in modules Okay, whenever we want to we could run multiple sdn components when we want to we have sdn dash I think through the entire alphabet Uh of different small microservices for sdn components. So You were right on that's all I wanted to do is jump on the bandwagon. I mean, you know We'll take on doing some of this stuff ourselves. We have a whole pile of dpdk stuff that we've written and Because that's where we came from right as I said, we were in the land of vendors So we kind of built a bunch of dpdk stuff. So we have a bunch of those but But I but I think you know across the state here are america's three largest carriers Right, I mean just to give you a context as what's going on here and and we're doing similar things But I think if we were to if we're allowed to them, which were obviously not to drill into the details Once we drilled into the details of what we do. They're all different enough to be different, right? I mean so so as we drill down in this path and so there's a couple of things we're thinking about the first thing is You know How different are we really right and so when we interact and it's a new experience for me being on this side When we interact with vendors here we go and the vendor a bit don't cry He's gonna cry right to about vendors right so when we get into this vendor thing You got to realize what you're doing when you pitch to us is you're talking about all the differences Between your product and another one and so I was recently the thing we were talking about big data systems You know whose databases we're gonna buy and they said adam you must have an opinion. I said I don't care They said you must have an opinion adam you got an opinion on everything. I said I don't care They said why don't you care? I said look we're gonna buy a big data system here We're gonna make some decisions. We're trying to rationalize another databases that we've got and you know in the big scheme of things The vendor solutions They're talking about the 10 percent margin that makes them different But we probably only use 40 of all the features anyway And so it's very likely that they're all exactly the same So we keep arguing because our vendor partners and we love our vendors too But they steer us down this path to argue about the 10 percent When it doesn't really matter that much. So I said guys, whatever you want, we'll fix it later on Buy whatever you like and you know ultimately We kind of will fix it later on as well So, you know, I think there's a couple of interesting aspects in as you think about how different we are and how the same we are How we work our way through that and then what we expect people to solve for us because our systems are all different And we all have slightly different packaging and so now you get to my other favorite topic Which is everybody wants to package open stack every call I get is I want you to I Here's my My packaging for open stack and it doesn't matter. It's an open source organization on commercial one too And I go yeah, we went down that path. You know, it's a whole lot of hard work. We didn't get what we want It's easier to spin it ourselves, right? And so I think as you enter this world with us and as we want to make this world better You got to think differently about hey, I sell this stuff because you know, we we don't want to keep paying for packaging I think the keys the maybe I went off into a vendor thing here in the future Procurement mess the future keys are customer service. I mean, it's about support and it's about execution So that that what makes the biggest difference to me in the first place anyway I want to pick up on your thing about packaging. So yeah, funny story Fender sellers. Oh, we tested this in kilo, but you need to re-certify it in newton and metaka. Well, what's the difference? It's all the same right the difference problem. Is that really right? I think that's a bouncer. We need to be able to figure out how to run vnfs on the cloud I mean the versions change, but that's just a management plane. It's not the to play domain where the applications run Yeah, and coming back to containers, I mean, I'm not really sure right I mean you quite right about networking assassins networking is hard to do So, you know, we you know one of the requirements is we run, you know, very high speed vnfs And so in a standard build off the shelf it's hard to get high speed vnfs to run And so we've done a lot of work to make it run that way And and so as we as we march towards containers and we talk about all the benefits of making these things smaller You know, we have to go back and go. Okay. How do we make this run fast? So so being that I'm an engineering manager I don't have a long a big opinion of architecture because I'm not an architect Right. I'm a guy who's trying to get us to build stuff But we do have some interesting stuff going on We have some work going on with the university who's trying to figure out how to get put control plane In containers and and I think some of the honest guys trying to do this stuff too, right Which is get control plane for routing in a container but get the forwarding down under open flow control And it's immensely difficult. I mean it's easy to get the one to work where you use AVS It's really hard to get the one to work where you use Broadcom chips. So, you know, it's some interesting stuff going on here Very good. All right guys, we have three minutes left. Um, yeah You didn't cry. No, I'll save it for later. Okay. Um, you know, I will Yeah, absolutely so Take a few seconds Final message 30 seconds a minute just Where do we need to go? What do you see this week highlight from the week message to take forwards Start with you, Greg. Okay First and foremost, we're grateful for this community We got to start with a lot of lines of code. We didn't start from scratch Not only that we get advice from each other. So it's that's why this thing is moving as fast as it is Is because we're able to work together Uh, and we're committed to being open and doing that again I think it's in the execution of the customer service. So when we come back to open stack um There are things to improve for large operators and and in You hit on one of them with the networking piece Uh, there was another one with oh, you just talked about container containerizing the control plane And that's relative to the open stack helm project and I want to get you to look at that too But uh, I think those are a couple of the major areas that open stack really needs to focus on and really needs to harden For for it to be viable long term and keep the threats away Thank you So I would say, um You know the operators at least those we bet made the bet to go to the cloud You know open stack and whatever comes next, but it is a cloud See my call to actually be you need to build your network functions to run natively on the cloud Embrace services based architecture. Do we still need diameter and I'm not picking on that But do we need our telcos silos? What do we embrace what the web skill world has you know pick up on embrace services based architectures web services based interfaces And you know build the applications around natively on the cloud. I think that will make everybody happy So where we spend a lot of our time thinking right is Simplicity of constraints in the design and the ability to do placement Right and so and so if you think about open stacks placement mechanism It's pretty rudimentary right and the complexity that we're getting lower down in the stack here because remember We're not just placing beams. We've got to connect them to networks the right way We get increasing amounts of complexity in the networking component above So as you think about I'm asking the people who do Building stuff here as you think about what you go from a networking perspective Think about think about the simplicity the problem in networking and the level of complexity it creates in placement And and that's an important thing to do Also, we're you know thrilled to be here. Thank you very much for having us And I also want to let you know that we do most of our open stack development here in new england And I have 30 rex open and I can put someone over here And take your resume as you walk through we're great guys. You'd like all right. Thanks gentlemen for joining us. Thanks