 Okay All right. Thanks for coming everyone We're gonna start try to stay on time We are the big finale but actually I'm John Zanos I'm with canonical and You've been to yeah, and I'm Toby Ford from AT&T We're also both board members on the open stack board and We decided to attack the subject of NFV and open stack and open source. So We're going out big with the big subject at the end of the day What we hope to do is give you a sense of where we see challenges how we think things are evolving and also Look for your help and your support and your input Through the course of what we're trying to do with open stack NFV and what we're trying to do with neutron sure so One of the big questions that we have with regard to supporting NFV in the telco context is how how much can open stack do and I think in the open stack community itself This is a challenge is what what is exactly the scope? What is it going to look like and how how big is it going to be? You know as we show here is is it in the end just going to be a set of API's and a whole lot of vendor plugins to make it Work or is it going to actually have implementations that are meaningful and that can scale to the telco use cases? Is it going to devolve into some type of standards body then because if it's just API's then It's just a long discussion about how how the API's evolve Hopefully not. I think open stack has been a very good example of how you can evolve a system with practical Deployments with practical code to actually make an API that's built from real evolution. Yeah, and we think You know, we have this question of is it going to try to be the ex service of everything Is it going to try to solve all network management problems? You know at the end of the day, is it going to be the mother of all open source projects? We all know it's grown rapidly We're all contributing it's moved faster than any other open source project ever And I think lastly we're trying to stop some of the insanity. You know, this is a project that's moving so fast We're wondering if we're going to drive our all all of ourselves insane given that when you overlay NFV SDN Open stack, how do we do this? And what we are proposing is some concepts that really are designed to kind of put things into a Much more manageable box to some degree make things a little bit more sane This was my okay So when you look at the telecom telecom industry, it's going through rapid rapid change New challenges new opportunities new evolution new competitors And it's forcing this industry faster than ever before to rethink its fundamental premises How does it operate? How does it take cost out of the equation and lastly? How does it live in a new world where there's different competitors different opportunities and more importantly different models? Yeah, exactly. And when when we look at the competitors that we're at we think we're gonna have to be working against in the future if you look at Google Fiber or Google Loon or Some of the things that Amazon Netflix do in the TV space. These are clearly coming for what we're doing So and when we look at them the track record they have in terms of Efficiency just taking like this metric of how many devices Can be managed by one person We are at a significant disadvantage. We have hundreds of thousands of people doing what Really could should be a much larger scope and this I think this is a broad Problem across of the telco industry. How can we get away from some of the old concepts? of operations and move on to the some of the newer concepts without Obviously risking the stability of the environments that everybody uses today So we recognize I want to step on this side the way the stage is set up. It's a little awkward being so close to Toby Even though I like Toby The reality is you're seeing a very different model, right a model where Scale-up appliances were structured for stability familiarity And rock solidness innovation Happened happened in five-year cycles This new world is a very different world looks like this innovation happens And the world we live it happens every six months probably happens every week Just takes us six months to kind of let a new release come out Incredibly fast incredibly different than the multi-year five-year six-year seven-year planning cycle of a traditional telecom Now in a wire line world that was an absolutely rock solid decision the fact that we all can turn on our cell phone in China, New York, London, Paris, and it works seamlessly is in upon itself an accomplishment that we shouldn't minimize and When we go to bed pray to all the telecoms that it works again tomorrow, but it's a very different model than this model Yeah, so today in the typical telco space We're using Essentially what what I think of is an equivalent to using mainframe computers very vertically integrated boxes that come from one or two vendors or three vendors that do a whole lot of Scaled function, but it's typically one function and typically things that also when you look at it from an IT lens is Duplicative of what we've already solved for so like take one of these examples a session border controller It is essentially a firewall and a load balancer for SIP traffic This I think could easily be replaced not just by Taking the session border controller and virtualizing it and getting all the benefits of consolidation and stuff that we've gotten out of Virtualization over the last ten years, but also possibly even being part of a SDN service chain so that we're not having to Have something unique and special to manage, but it's actually part of the fabric itself So the shortcomings of the existing setups We can't underestimate the the challenge that we have of moving on to the Moving them on to a new platform while keeping them running So we have in our world the challenge we think about all the time is do we add Augment these with virtualization To the side or do we do a completely net new thing and start over again? Do we wait for 5g to happen or do we wait for? Maybe some acquisitions to be finished before we start this process Or do we dive in now and try to get the benefits that we see have happened? Not just because of virtualization And like I was saying consolidation high asset utilization and moving faster But also the benefits that we see in terms of the ecosystem Can we get the benefits of the PC revolution? The Linux revolution and the open-source revolution that has happened over the last 30 years And I think of I think open-stack as kind of the pinnacle of that work is how do we Actually maintain a project that has many federated Aspects that work together, but very independently and move fast and move fast very quickly that evolutionary model that we've created though Scares the telcos it scares us how how can we? Rely on something that's moving quickly And it's continuously deployed and is very agile at the same time we maintain high levels of nines For your phone service So it's interesting right we're talking about speed and stability innovation and you know Planning and operational efficiency, but at the end of the day the benefits are very big very big for us as end users of this technology Because we see the innovation. We're all using the innovation It's very big for the telcos because you get a benefit just from the commoditization of the hardware We can debate the percentages, but we think the hardware is a minimal impact That's compared to a benefit from virtualization, but if you just hit the button Toby the big play out is here, right? We're changing the operating model So think of it what we are talking about is a metric of how many machines managed by a person if you look at the Google the major cloud providers. We're talking thousands hundreds by an individual the ratios are much much lower in the average telcom so It's not only about innovation and speed But it's about fundamental P&L impacts and as you all know if you read anything in the Wall Street Journal the financial times This is an industry that is changing competing in a very competitive market and also trying to figure out how to make investments at the same time Bringing forward innovation and at the same time Competing against companies that didn't have the legacy burden of running a wireless and wireless network The responsibility to make sure your cell phone works in New York as well as in Shanghai as well as Anyplace else in Africa, right? This is a model that was built on stability and in a framework of Solid planned innovation now they're in a world that is all about rapid rapid rapid Yeah, no to add to that I think if you gave me a hundred really good people and Enough capital. I think you could reproduce what a telco does Fairly readily, but that's not really the challenge that I have the challenge is all about taking Thousand person ten thousand person ops teams and getting them to Think differently to to realign their paradigms This is a much bigger challenge than that that I believe than the actual technology But I believe strongly that the open-source model and the community around let's say open stack and other open-source projects like Linux and like other Projects that are just now coming to bear Can help make this work better work faster and provide examples for how we can make this change So this is a picture brought to us by our friends at Etsy one of the standards bodies That we work with very closely to help with getting all the telcos to think in a similar way around how things work We helped to drive this Concept within Etsy to try to at least at the beginning Standardized nomenclature for what pieces are now there's still a lot of argument that maybe this isn't exactly right But I think it does provide at least Some very common terminology that we can use to talk about what needs to happen And especially for me. I focused primarily on the underlying NFVI or the infrastructure of NFV and how to solve for this piece And then in this area open stack plays a very significant role as well as do SDN providers and as well as a number of other entities that are do workflow and operational management tools Maybe for you we get off this so, you know, if you look at this you recognize where open stack fits in You know where there's elements of Open-source projects and there's also a very important opportunity for Application automation and management and there's a number of projects right and a number of communities as well as individual companies attacking this where We see a lot of benefit is The open-source community trying to attack these problems But we also see some risk and some of the risk is this is the whole watermelon of the problem, right? So it's a complex problem to take a telecom from where it is today like AT&T to this end state is In one swoop is a very daunting problem We think the way you need to approach it is look at it as iterative steps and a little bit of what we're going to be Talking through the remainder is trying to coach all of us as a community to think about this in digestible blocks And that's the important point right because right now if you look at this project This project for any telecom is the holy grail of end state But actually implementing it in one big swoop is just not even on the table Yeah, and another point to add here too is Clearly in the past when we ran the physical versions of these network functions They each came with their own orchestration and management tooling. I Think there's a great opportunity right now not just to make that more cohesive or contiguous Want more homogeneous making this as much simpler model, but also to kind of bring in innovation Like say instead of traditional BPEL or BPM tools Instead maybe think more broadly and using paths tools and be more part of the developer Lifecycle and how they work and how they bring code into the picture So making this this is very helpful for an operational person in a telco to understand how things change But I think how we bring the developer into this picture is going to be equally important So, you know Toby said an important thing and it's a word I think historically hasn't been used in the telcom industry as much as it is now and that's developer Right a lot of the innovation we're talking about is now encapsulated in software as compared to the appliance and a lot of the innovation is helping the telcos make this Transition to you know virtualization of next network functions Now as we've talked about not all these are easy not all these lend themselves immediately to scale out There's some steps some evaluation But at the end of the day to bring that innovation on top of the network to bring those new features to the end users What we are talking about is a development exercise and that's being deployed through the exercise of some communities Some of it's proprietary, but all of it is software oriented. You want to hit the second point Yeah, and then it what we've seen within open stack is an example I went through yesterday was about the VLAN trunking issue is I think and it provides a good Good sort of use case to look at and has how open stack Needs to evolve The people within the community need to be more aware of the NFV use cases and what what kind of challenges we have This particular one represents Unfortunately sort of a legacy concept that has to be brought forward it for an interim period of time Until we get to a maybe a longer-term vision where the vnfs are truly Cattle-like or scale out. This is one of those things which people have not universally agreed with But is essential for us to make progress and to be able to deploy today So this is an example where both we need to kind of think differently think more broadly about the use cases for what open stack can do but also Working with a new set of people new set of specialties around telco and and and a new set of vendors like Erickson's ALU's and then also Helping those companies that provide vnfs to evolve their platform to be more scale out So so at the end of the day one of the things we're advocating and we're Encouraging to the extent we both can and our respective companies can is we recognize this NFV network Virtualization is a very daunting problem as we said and not easily digested by any telcom of any size But we do think there's a path forward one. We believe you can get to a Purpose-built NFV so NFV in its broadest sense is about all the functions of the network or the vast majority But you can deploy NFVs in very discreet areas where maybe you're focusing on one or two or three functions Why is that important? That's important that it allows you at least to get Into it in a digestible way in a p-sets manageable. The second part is we do think this is possible via Manageable open-source collaboration though We wrestled with these four or five words because they always don't line up together But the idea that if we can get ourselves as a community looking at the problem in definable chunks and then attacking them as definable chunks Not just attacking the total problem even though that's probably the more interesting problem We can make it iterative steps forward and that's probably going to be more important in the foreseeable future both in terms of actually having impact within the telecoms and also in terms of doing something that's actually achievable and that we can see the results and Proof out the benefits that we all believe exist with virtualizing the network Yeah, and as as John is saying for us. We're not going to try to do a whole mobility system Right away. We're going to try to take pieces and parts and focus on let's say a small enterprise set of customers To begin with and then try to make progress there with a subset and then move on and gradually iteratively get to the broader set of function that we're getting to or needing to address so in general the open-stack collaborations that we've had to start certainly to me start with code and working together as a teams but Within a telco. It's it's been a bit of a challenge and I believe it's true for more than just AT&T is actually Convincing people that you need to apply resources and developers that actually can can make a difference and then participate in it without all the normal encumbrances of old companies like IP and and and certain sort of Challenges with competitors and such this this idea of working outside of AT&T is really unique it's very much a type of Environment where everything is about what you're doing today within your organization Opening up and being more aware of what's happening outside is is a very new phenomena for the telco So it starts with that and it always starts with that and then obviously there's legal challenges with licenses and such that we're hopeful that You know we can pivot the lawyers to the right in the right direction Obviously beyond that is about partnership with an ecosystem You know and I think John and I represent some small subset of that Yeah, I think you know the other thing coming from the side that a company that's DNA is based in open source You know we recognize that this is a pretty big pivot for most telecoms You know that pesky IP thing gets in the way of a lot of discussions, but we recognize that what we've done We've opened up Our technology our software make the code available all the stuff we do and open source projects And we team with the telcos to help them navigate that understanding and build a degree of confidence and comfort in doing that very thing so Manageable collaboration is easily said challenging to do and the only way we think we move forward successfully is You know to use open stack as an illustration We continue to build up the number of telcom developers that feel like they're unable to be very active active Toby's done a great job of representing that helping them understand that you know getting together on IRC and talking through The problems they're having with code isn't giving away the family secrets to the other guy who happens to have a V in front of his name as compared to an a you know that evolution is Going through in a very thoughtful way within the telecoms because there is as we pointed out in the beginning a very competitive Landscape, but I think the benefit and again the reason we think about this as kind of purpose-built NFV Is that we want to create little proof points that demonstrate the model works? And then the other thing is this comment Not that I think the telecoms are imperfect but like all of us they have a couple challenges of evolving in this model and This is an area where we as people that live in an open source Community or the business model that is open source need to recognize that you may be dealing with somebody that's going through a bit of internal Evolution and while their intent is to be very active. They may have constraints internally that are pulling them one way or the other Exactly, you know when I began at AT&T Open source was outlawed and then you couldn't get the IRC from inside the network and hopefully We've made progress in that regard over time and it's also Contributing code using github or or something of the like that was completely unheard of In fact, we didn't even use get or anything like it before that. So this is Something that we've we've moved on from I think Now one of the things that we're struggling with and this is partly my ask to this group today is you know Where does it go from here? How do we what are the next steps or what do we participate in and to make this work to actually? Build on what OpenStack is done. Did that mean? Bringing in all sorts of other open source projects Working with them and then trying to integrate things You know, I think I'm right now as as I begin the steps into OP NFV I kind of have existential question is How much can we get these different groups and also the vendors that go along with them? to actually work together and collaborate obviously there's a lot of of Motivation given the trillions of dollars that the telco space represent but for me This is a very complex story when we try to put it all together Yeah, well we when we compare notes. It's an interesting problem for all of us, right because It's like puppies. They're just keep propagating and propagating and we all get the calls and Toby and I talk about it Can you participate in this? Can you assign some engineers in that? Where the future where the future will collaborate with everybody and it's it's a challenging thing when you think of the talent pool Is it? Limitless right there's talent that we want Dedicated and focused on complex problems and as a community where we Propagate so many efforts that are slightly different but near same we create a problem of distributing our focus now This is also where innovation occurs, right? So this is a very careful type rope to walk because The separation and multiple efforts have created a lot of innovation that we've all been part of but what we're recognizing in NFV is Given the size of the opportunity and the magnitude of the space under the telecoms of the world there's a Rapid propagation that we need to be careful about because what happens is we distribute ourselves into many many projects We distribute our efforts our resources our dollars and we're wondering collectively But I'll say just Toby and I are we helping or hurting the problem? Exactly now. I feel like in some cases obviously I want to see innovation continue And I like the idea of a person starting something up and making progress But at the same time if you look at the problem globally and across the tocos There's actually in many cases a lot of redundant work if you look at just take SDN as an example at some point I find that we're actually only perpetuating maybe our ego or You know, there's I made this and so I want to see this baby grow that at some point Defies logic and makes everything very dilutive and so I think in many areas We have to really challenge yourself and justify is what I'm doing really differentiated or not And then help because as much as you know an agile when we iteratively build everything At some point we have to refactor and in in this space across all of what we're doing I think that's probably a good idea for us to look at and can we refactor some of these projects Yeah, and I think that the thing that we also want to support is where there is differentiation being created and real innovation We absolutely want to put support and focus around those efforts now if we all had the perfect magic ball We would know that at the beginning of the process, which is hard, but we certainly should watch it very carefully But if you do have a proprietary thing For us that has to be wrapped with interfaces and in a way that allows us to to work within open source projects It cannot be an independent thing The days when we bought a an entire solution to this problem from one vendor is gone It's going to be a very You know many vendors and many different solutions that actually solve this problem, right? so we wanted to use an Example and this is an example because it's gotten a lot of recent focus, but we look at this and we say You know the many more and many efforts and doing many things and the question that we've been wrestling with is Is it the answer? Is it making things better? Is it not now? Everybody can have their own perspective But we do think you know we should be asking these very questions and most importantly when we allocate resources How are things progressing and how are they not? Exactly, so this is one of the existential problems that I'm wrestling with right now is Can I even explain how I can make? Or why I would make a cloud stack work with a day open daylight with a zen You know and what are all the different permutations that we really meaningfully want to test? I Like this idea I think it's going to be helpful But the question is is it going to be helpful enough quickly enough and is it going to be too complex of a problem to solve for? Where where do I even begin and so that's What we're wrestling with this doesn't make sense in the end right and again this gets to this concept of purpose-built NFV or more importantly Manageable definable projects that actually have a chance of reaching some sort of coalesce conclusion in a foreseeable future And actually resulting in something deployed that can be used exactly so in the end, you know, I think We we absolutely need the help and then I think in a number of ways, you know We we I I want to accentuate the part of about open stack that is a standard API I really like the idea of a multitude of solutions in the open stack community Some of which are pure native implementations, but then others where creativity is involved But as long as the APIs are standard and consistent then that what happens behind the scenes can be continually Innovated so, you know, we've made good progress in the foundation on def core and ref stack and the like But that I find to be typically It is good progress, but it is somewhat biased toward the implementation side And then whereas I'd like to see and we're making progress here in the next period of time Taking that and then making it more about a trademark for API compatibility, which I think will have much broader Applicability to the problem So this is the part where Toby and I decided to step a little close to the edge of the ledge and at least try to give What we think as two individuals some guidance, you know, certainly We are all aware that neutron has a number of shortcomings We think the right way to attack it is to focus on the API side of it and let other people plug in it and expose more Network functionality we we believe NFV is a big proposition that we should all be focused on and spend time thinking about But at the same time we think we should break it into Chunks that are manageable digestible and actually implementable and demonstrate and proof out the value that we all believe is in there And not get so caught up in solving the mega problem that in two years from now. We haven't moved forward at all And I want to leave us with one one possibility that that I've been considering is Given the track record of OpenStack, can it actually be the center of gravity and then can we maybe Flatten some of these outside efforts into OpenStack so that it can maybe more broadly solve for Network issues as of one example. Yeah, and I think at the end of the day You know we view this as a big we problem and big and big challenge not everybody can have the same opinions And not everybody needs to have them. We wanted to do a couple things in this session One at least share some perspective of what's going out there Give some ideas at least how we wrestle with it not that that's the perfect solution or the only solution and try to put Some guardrails on the conversation at least from our perspective We hope you found it helpful We're happy to answer any questions we're certainly here today and tomorrow and Please feel free to Step up to any of us and give you to your opinion good or bad and thanks for your time All right, any any questions if you have a question. There's two microphones here in the front in the back It will be this is Shrikant from HP question on the NFE value proposition how much of it is hinged upon the VNF's being really written for an NFP word and I don't mean just replicating VMs for a scale of To create a cattle scenario, but essentially being taught off in the new NFP paradigm that's Question number one and the question to a week Yeah, so the first thing I think there's gonna be an interim state and we can get value out of NFV Without going all the way But I'm hopeful before we implement something Significant like a 5g or the next generation TV platform that those platforms are end up being designed in a more More integrated and more scale out model. So Okay, the second question. Okay. Yeah We've heard yesterday. I think it was multiple flavors of Android and there was an analogy drawn for Multiple flavors of open stack. Do you see an NFV flavor for open stack? kind of Becoming more and more popular or do you want to just keep everything? I Think it's inevitable that there's gonna be an NFV flavored open stack distribution of some kind whether or not we make it or Another telco makes it or that's the actual output of opnf V I think that part's inevitable, but I'm hopeful that We can actually work together so that there's not a lot of those I Would just add one point. I think we've talked about this We want to keep a very strong core of open stack the core and other things so you don't have you have the Improvement around the edges, and you don't have so many propagations of different models that portability and interoperability is lost in the process right now Rod Randall from Cirrus Capital one of the things that has come out in a number of conversations is That there are a large number of virtual network functions that would be implemented in a substantial Implementation of NFV many of which have their own Management and orchestration in the presentation just before you guys came on the presenter from Stefan from from Deutsche telecom Mentioned that they decided that orchestration should have only one owner in terms of actually deciding resources and That all of the individual functions needs to sort of request from the broad view of Orchestration who then will resolve all the issues and allocate all of the components As I understand it open NFV hasn't chosen to take that on should that be something that should be standardized Or is that an area for innovation and if innovation comes with a difference from how it's being Being viewed. How do you see integrating that into the into the real world of making these things work? Sure So my view is a little bit different. I think that's a that is maybe a goal But the interim between now and then is going to be very much of a patchwork of different BNF managers and right now open FV is not dealing with the the BNF manager problem at all I think in a way that provides an opening for innovation to happen And I'm very hopeful that as I said in the presentation that we start to take on More paths like and developer friendly concepts because in the end the goal is is more of a DevOps model or infrastructure As a code model rather than just perpetuating the same Same wacky belief that a business analyst is going to be able to to configure a whole system in a in a BPEL tool And you know the one thing I would add is in my conversations with the telecoms Toby's point about being more past centric or developer centric or dev ops centric is a important evolution that's going on So when you think of the holy grail of solving that problem and innovation It changes very differently if you think of it in a dev ops model as compared to a traditional telecom model from an operations perspective And I think the the real opportunity for innovation is if you look a couple steps out And we think the first couple steps that we said are some smaller Finable projects, but ultimately if you want to solve that problem Start thinking about it from the perspective of a developer who's creating new Code to create new value to the end user as compared to running a network Right. I got you. Okay. Thank you. Yes. So best call and I'm from the V Please come up And and I'm a I'm a big proponent that we do need to get more involved in in this But but I'm struggling with you know, just the not invented here kind of attitudes And and I think also and I'm sure you run into this as well that The way that it services are developed and delivered or is it different? You know, it's a different balance things are in different locations in the organization and I struggle with well Why did you do that that was a stupid thing to do you think think it through, you know, sure No, I mean in with an AT&T we have the same not invented here issue now in some cases I think there's it's justified if we have real unique something unique and that's fine, but Understanding and this might where my external content Concept comes out is a willingness to look outside to say wait a minute What are other people doing and then really understanding what is differentiated or not? That's that's one part of it for us And I was just saying this John earlier today is about the I've been at AT&T for eight years and I've been on a kind of a constant Journey to try to converge efforts And I've gotten through like maybe five or six clouds to be all part of one thing because I honestly believe without being all Part of one common infrastructure will never have be cost effective But still there's four or five more to go. So are you gonna share your? Challenging stupid not stupid, but challenging stupid thing. I'll say it's stupid. What's in AT&T. I have a challenging stupid thing It's so classic So we we added a new product and we we changed the billing to was a new way of billing things and Business analysts put together the requirement and the IT person, you know Wrote up the code and they forgot that if you're going to do usage-based billing 1,000 is not the same as 1,024 Yeah You know I would add because we deal with Telecoms across the world and and I think there's a couple elements to this as well one is a lot of difference per Geography per culture right and this pace of evolution in terms of the internal thinking It's very different. So as a community you have to be sensitive to Somebody in Verizon may be different than somebody in Deutsche telecom that may be somebody different in sing-tell compared to NTT and I should point out we all do talk to each other Well, you know the interesting thing I think ultimately there's a model to be replicated right the reason the wire line world works isn't because everybody ran a Wire everywhere right you have your arrangements with other telecoms And I think it's interesting from somebody that's in the open-source world that you were able to navigate as an industry Making those relationships and then in this cloud word world. You're a little bit more sensitive of your IP I think that's actually driven by a little bit of fear and newness But you have a great model that you've already executed completely sex successfully in the wire line and wireless business to allow calls to happen around the world and one of the things we've talked a little bit about is how do you Allow that model to propagate itself amongst the telecoms Again in this new community base open-source world. Yeah, I'll relate this story John's referring to is and it's actually a beautiful surprise not not necessarily stupid, but I Didn't mean any offense. We recently reworked to be a much simpler organization and then And nowadays I own some part of cloud architecture and someone called me up and was in a confessional mood and said You know Toby we forked off of your open-stack setup two years ago And we built out this environment. It's now deployed in 12 data centers, and then I Know you don't know about this, but that that was by intent So certainly not stupid, but a good illustration of the different pockets and the size of telco Question is on devops you mentioned devops few times yes during the time so the question is it is One thing to have continuous integration continuous development DevOps type of flow for the services that AT&T is developing or the any operator is developing But do you see the same thing applied to network functions? virtualized network functions, and if so, how do you? Yeah, see it working with multiple vendors who are supporting this network functions or supplying the network functions to you Well, one thing as a base to answer your question about applying devops NFV I mean, I'm a strong proponent that you cannot achieve a resilient system without having Some aspects that you need to make continuous integration deployment work right you need complete test Coverage you need complete automation of testing. This is actually for us helps tremendously Beyond what we were doing before we were very manually targeting testing and that that takes years sometimes to You know one of our systems we just went live with it took us five years to implement because of the testing involved And that's that that old way goes away when you have continuous integration now The idea of deploying all the time and such that's going to take work And so that's one of my challenges of vendor community around us is how do we how do we work together? to to bring things into a sort of a common Repository and then build and deploy off of that in a reliable consistent way. I mean, that's that's a big challenge So and and I think ultimately you got to recognize that the telecoms are competing against a much bigger world Right and the benefits of devops They see him every day in their competition. They see him in the innovation that's happening in other places So that balance has to be struck, but but they're in a car, right? That's running very fast against other cars So, you know our advice always to the telecoms is don't become so inwardly focused that you worry about it as compared to how you Did it yesterday the really question the real question is how are you doing it compared to everybody else that you're competing with and If you're not able to run fast enough and keep up Even though you may have a bias or preference or you're worried about stability You may not have the luxury to hang on to that model I think we're probably at up. Yeah, I apologize. We're going over next thing has to happen Feel free to come up and ask questions. Thank you very much. Thank you