 All right. Good morning, everybody. Good morning. Morning. I'm just trying to get them to shut the door. Wow, it's going to be a long day. Yeah, it always feels, already feels long. Okay, I think we're good to start because we want to finish on time. What I would like to do is just introduce myself and my co-panelists, and then we're going to have a bit of a discussion around a number of key topics today. Predominantly around, you know, NFV and how that's getting deployed and different perspectives from operators who are actually working very hard to drive that forward. So without any ado, my name is Daryl Jordan Smith. I work for Red Hat. I lead the ITC Vertical at Red Hat. I'm actually based in California, as you can probably tell from my accent if you're from the U.S. and if not, I'm originally from the U.K. But I want to take the time to hand over to my panelists to maybe introduce themselves a little bit. But while I'll do this, I'll just run left to right and let you know who's up here with me today. So to my left, we have Ray from China Mobile Research Institute. Next to Ray, we have Michael, who is from Telus. Next to Michael, we have Fred. We're going first names today because I'm not very good at second names, from Verizon. Three major operators in this field. We're all very excited to have them here. Let me hand over to Ray just to maybe introduce himself a little bit more further and his responsibilities at China Mobile. And then we'll go with Michael and Fred. Hello, everyone. I'm Ray from China Mobile Research Institute. I'm the project manager and senior researcher of big data and IT department. Currently, I'm responsible for the IT infrastructure. Thank you. Thank you. I'm Michael Bag and I am at Telus Communications, which is a Canadian telecommunications company. We have a nationwide presence in Canada as well as some locations globally. My role is a cloud architect in the cloud architecture group at Telus. And that's it. I'm Fred Oliver. I'm from Verizon, which is both wireless and wireline environments. And I've been also working on cloud architect at Verizon and we're working on architecting the OBISAC platform for Verizon for NFB. Great. Thank you very much. So without further ado, I want to try and do is start off the panel by my first question. And we were doing a bit of a review this morning over breakfast and we decided the first question would probably take the entire session. So hopefully it won't be as bad as that, but we're going to go into hopefully some level of detail. I really wanted to ask the panelists one by one to just talk about why they're here today and where they are with their OpenStack implementations and NFE in general. And then specifically where they see some of the immediate issues in terms of some of their deployments. There's various questions within that. There's a lot to talk about and cover there, but it would be a good baseline for the conversation here at the panel today. So Ray, I'll hand over to you. Okay. Well, China Mobile has always been a supporter and promoter of new technologies. We are continuously protecting new technologies to promote our network and business even ocean. And for the now CT and IT are deeply converged in which NFE and OpenStack are deeply involved. For us, China Mobile, we have two driving forces. The first one is we want to shorten the time-to-market of telecom services. And the second one is we want to reduce the company TCO, especially the operating cost, the CAPEX. And we hope NFE and OpenStack, these new technologies can help us to push and to make it happen. And for challenges, we have many challenges when introducing the new technology. First, by introducing 4G, China Mobile got the accelerated girls of our 4G traffic. The traditional network architecture and operating system can hardly adapt to these rapid traffic girls. So we need to embrace new technologies to help us to solve these problems. And second challenge is the software and hardware or traditional telecom equipment are binding. When we upgrade the network, it's making it inefficiently. And the operation and maintenance make it very difficult. So we have to reform the network architecture and the whole network management operation mechanism. And for management, it's more complicated because we have legacy OSS and BSS system. We also have the management platform for different cloud platforms. For example, we have private cloud and public cloud. All the management systems are different. By introducing NFE manual, it is becoming more and more complicated. So we have to make the specification standard of the manual and to integrate all these management platforms and to make it with new technologies with an open stack. And with development or IT and internet technology, it brings huge challenges to operators. On the other hand, it brings us also development opportunities. We hope to grab this opportunity and use this open stack with new technologies to face these challenges and to solve the problems. Great. Thank you, Ray. I'll come back to some of the things that you were talking about. Yes. Michael. My answer is very similar to what a colleague here has to say. There's not much more that I can add. Except that for TELUS, a number of the drivers that we have for what our introduction to NFE and SDN are all requiring of the cloud delivery mechanism. So when we talk about business drivers and our initiatives around providing health connectivity and services to the healthcare industry, the connectivity and service that we need to offer to internet of things, particularly in the automotive and retail sectors are very much going to require the kinds of latency that can only be delivered by the NFE and SDN micro cloud deliveries that we are going to have in our POPs and our COs and our MSOs. And so as we begin to augment our legacy networks and our distributed services to achieve that unified control plane, we will begin to have those footprints of cloud to deliver those sort of business services that are growing. So our play is somewhat coupled to the private cloud evolution. So as we deploy and build our initial NFE pods that we put into these sort of locations, we will likely be binding them to a private cloud delivery. Yeah, so again, I agree with pretty much everything else. You guys have said from Verizon's perspective, our biggest problem is that we are in advantage that our data growth is growing at very high rate. And unfortunately, our costs to run the environment are growing at a higher rate than the revenue we're getting from that data. And we need a way to somehow break that curve and modify the curve. And we see OpenStack and NFE as a potential solution to modify that curve and bring us back more in line with where the cost structure is more in line with the revenue we're getting for that delivery. Some of the challenges we see, I think I agree with very similar things, we have a very lots of legacy systems, lots of integration with different management systems, OSS environments, BSS environments, how do we connect the existing environments that we have into new NFE enabled OpenStack clouds that we can integrate all those services together. Great, so I'll start with you Fred because I don't want to get you to be the last person down the road every single time. But you just touched on some of the challenges that you were seeing in terms of integrating some of those legacy systems. How are you uncovering those challenges internally in Verizon from an operational standpoint? Yeah, so I think we certainly have a dedicated team that's very familiar with our internal services that's slowly working through all the challenges they see developing custom build adapters to the existing environments, services and infrastructure and feedback that OpenStack can provide and adapting those to the services we have. It's a slow transition. I think we're in the midst of getting ready to deploy our first service and it'll be an interesting challenge. Great, Michael? So I think that we have a couple different approaches happening in Telstra right now. There is in a group that I'm in, we are for the large part approaching a green field approach where we're trying to land a componentized service that is completely green to the legacy environments that we are in. That being said, there will have to be some eventual integration as we move forward but if we wait until we have that integration well working, we'll very likely never be able to move forward with some of the opportunities that exist. So as we begin to deploy our initial NFE pods we're going to be very careful and deliberate with what kinds of services we begin to integrate into that environment. And I think that some of the, we're trying to aim for some of the low hanging fruit that exists there. As we mentioned earlier, the idea of what it's going to take to sort of upgrade and reform the legacy environment. A lot of it comes from the mere fact that some of our 15 and 20 year old pieces of hardware are no longer delivered as a hardware platform. They're delivered as a virtual machine. So when you receive this virtual machine, how are they going to live in an environment where there does not exist the kind of infrastructure that's able to provide a virtualized stack? I mean one idea would just be to merely land some machines tied together a v-center or some sort of ESXi cluster and load that VM on it but that itself does not lend itself to any kind of centralized management or all of the features of NFE and SDN that we want to explore and exploit. So, Cloud. So Ray, from your perspective you were talking a little bit about the challenges around management of this very complex environment. Can you expand a little bit more about what you're seeing in that area and what some of the challenges are that you're seeing? From a China Mobile point of view because we have a larger management system, we have different management platform, legacy platforms. We are from the research department. We are pushing all these new technologies. We are carefully defining the specifications and the standards and also we have many POC and test activities and field trials to verify the technical solutions are ready for us to move forward. But as a new technology, we face pushback from traditional departments, especially from the operation and maintenance department. It is normal because new technology introduce many difficulties and challenges. The NFE have to reform the whole network of management systems and so we are carefully defining the specifications and by testing to verify the technical solutions. So, Michael, what would your perspective be on that? One thing that we're kind of missing a little bit here is the risk. I think a lot of the pushback that we're going to encounter comes from a very risk adverse set of operational groups. But the point is, as we mentioned at the curve, can no longer the demand... The cost curve is too hard. The curve is too hard to manage. Once upon a time, you'd be able to buy a piece of gear that you can expect to last for 10 years and hire a group of people that can also last for 10 years. But we're now at the end of the curve where that's no longer possible. As you begin to add devices, as you begin to add people, it's an exponential problem. No, I think there's... Certainly, I think both from a... somehow with the change, our legacy management style as well as that automation into this environment as well. Those are kind of the two things we see in this way to break this cost curve is to automate a lot of the current processes that are done manually today, enable some of the automatic functionality of scaling products. And that's, again, kind of the NFE enables some of that scaling functionality and performance as you need it. So do you see much pushback from the risk-averse part of network operations in your businesses? How are you working through some of those challenges? Fred? Yes, we are. As everybody is, I think, obviously going to face, we're... Again, as Michael was saying, we're not going to kind of take on the major load of the environment right away. We're going to have a slow ramp of bringing in and proving that the services can perform at a reliability and performance as kind of our current environment and that our operations team can actually run that environment as well, if not better than they can their current environment. It's kind of interesting that our operations team actually is excited about moving this direction. They see that they have to do something that can't keep building kind of custom solutions for all this and are looking forward to enabling agility, enabling different kind of ways to provide services. Great. Michael? No. It's a new environment, a new world for a lot of the network engineering teams we have. Mixing in this idea of doing automation and building scripts and templating and providing central control to many of the environments is just not how things have been done. So this is a new education process for many of those environments. Great. Now, Ray, just like building on some of the things that the colleagues were saying here, we're barking on this journey using open source. And there are probably some interesting nuances with regards to network operations, people using open source technology, but from a China mobile perspective, how do you see open source? What role does it play in your thinking in terms of driving that cost curve and the agility that you're looking forward to and the innovation that it potentially brings to? Well, open source is a trend of the IT industry, and China mobile will follow this trend, but we also saw open source now has a gap with operators' requirements. So previously, we also tried to deploy the open source software in our current systems. For example, Red Hat Enterprise Linux in our private cloud, and also we deployed the REL OSP as a management platform for our hybrid cloud. And now for NIV, we still have to be very carefully because, for example, we need the carrier grid, so we will do the test verification very carefully and we raise our requirement specifications for this, and we will develop the requirements what is currently not supported by OpenStack. We develop it by our server partners and to support our carrier grid requirements. Great. Michael, from your perspective? You know, I've had a number of discussions around what exactly is carrier grid, but let's not belabor the point of whether a carrier grid is any great deal different from what a well-run network requires. The gaps that we talk about are often around the dependencies of all of the multiple kinds of protocols and vendor pieces of the pie that live within that environment and how much lag can we live through with each iteration or growth of an OpenStack release. So, for instance, there may be some network or switching tools that have a four-month lag from when Liberty is released to when we can actually deploy and use Liberty. And that's just one vendor, a switching vendor. Now, what happens when you start building into that the other kinds of releases of different kinds of protocols as you begin to mix in IPv6 and where that kind of lives in the network. I see that the gap from OpenSource is pretty broad there. From your perspective? No, I think the OpenSource brings one of the big advantages is the ability to innovate rapidly, bring new technologies, new techniques into the environment rapidly. But along with that, again, I think you get the... It's not completely fought through in certain circumstances and also hasn't been completely tested and validated in all the environments we want. Again, gaps from the existing environment and this is kind of where the kind of carrier-grade model comes in is being able to qualify, validate that all the services are capable of tolerating the rapid growth, the event issues that we get, the failure modes that we see that the environment can actually handle all that rate we need to handle it at the kind of low overall failure rate that we require in the network. So do you, Ray, coming back to... You made a very good point about the carrier-grade-ness of, say, OpenStack and the additional features that need to be built in that. Do you think that's an inhibitor to the adoption of NFV or do you think that because of OpenSource we can reach and meet those gaps collaboratively together? Yes, as I already mentioned, OpenSource is a trend. OpenSource has many advantages because it has an open interface, the open mechanism of security and the mechanism. So we will go along this road and we will define the requirements and we deploy the OpenStack, OpenSource technique step-by-step, from easy to hard. Ray, my microphone for you. Precisely. As I mentioned before, we're approaching it from a green-field approach, landing green-field environments that we can begin to integrate as much as we can as we can. There will be some other efforts by other groups in our company to better integrate into our legacy environments, but I think in those cases it may be using OpenStack as the mechanism to help orchestrate some legacy environments. I'm hesitant to think that that's going to be too very successful, but we will try. Great. So Fred, from your perspective, are you seeing in the deployment any low-hanging fruits in Verizon in terms of use-case examples that you can share? I think there are lots of use-cases that our potential to share were a lot of the issues that we're facing are basically short-term expansion of the environment. So we're looking to, for certain circumstances, things like some video service as a video becomes hot in the environment that we don't have all the videos distributed across the environment that we can scale up an environment to share those videos at real-time as go. Other areas that we're looking at are specifically for being more nimble in terms of deploying services, new services or existing services more rapidly, and there we're looking at enterprise mobile services. There's a lot of capability in extending services for enterprises in an open-stack world. Alright, Michael, tell us, what are some of the use-cases you're looking to share? We talk about there being low-hanging fruits, but in my experience so far is that a lot of it is having to just eat the elephant. We try to differentiate what are going to be some of the easier approaches that we can do and we find that it's just as we may as well go after some of the harder ones, but the cases where we are beginning to begin to play is just looking at defining our control plane, the kind of control plane activities we can to begin to automate some of the services. That's our focus right now and as we begin to mature we'll begin to look at some of the use-cases that we want, but until we get our controlling, our ability to do the management and configuration of some of these services, we're not moving too much forward. Ray, from your perspective? Actually, China Mobile has already deployed AV in many areas. First, we already deployed our RCS system. RCS is a rich communication which is more like a RCS platform, but with the carrier-grade requirements and then we did the lab test for control plane, we just mentioned control plane, virtual IMS in our lab and also approached the field trial in some programs of China Mobile and now we are doing the lab test for virtual EPC in the lab and we also have a project called CIRA which is focused on the wireless which is a migrate traditional base station to the general IT platform. It is also in the research and the lab test period. So again, we are doing this step-by-step from easy to hard, from control plane and then data plane and finally the wireless radio access network and finally we achieve the goal of the whole system based on the IT platform. Great. So, I mean from my perspective, just listening into what you were saying and summarizing a little bit, there seems to be some specific progress. There seems to be some gaps in and around OpenStack that you were addressing from an R&D perspective and we are trying to address those in the upstream community collectively but those gaps do exist and we are having to pick the right use cases. But from your perspective Ray, do you think that some of the technical issues you mentioned cloud ran and jitter and latency and all these big issues from a carrier perspective are pretty major? How are you seeing that in the lab? Are you seeing that that is something that has been realistically deployed in say the next 12 to 18 months? For wireless maybe it is difficult because we previously I was in charge of the CRM project and we did a lot of tests in our lab for wireless we have to meet the 3GPP specification which has extreme real-time performance requirements. So we have to do many improvement from the open source technique. For example, we have to use real-time virtualization but to leave the wireless we just talked about the core network it is easier to realize. We just did a lab test for the decoupling test of NFEI and VNF previously we have some tests but with coupled system which means traditional telecom equipment provider they provide both VNF software and also the IT platform but this time we did the test with different code from different companies and we did some gaps for example the open stack now we need the resource management for both physical resources and also virtualize the resources but the open stack now only support virtualize the resource management or physical resources and another point is we found the open stack this stage is still very new the easy to use and stability still need to be improved this is what we found throughout the test. Michael do you want to add anything from your perspective on this? Only that we have a number of vendor supported proof of concept environments and it's very difficult for us to find some common thread of stability across all of those environments. We may find some pros and cons of working with some Cisco ACI for instance or going with another vendor or some other vendors that we're looking at and of course the monitoring utilities that we have available to us in the testing utilities there's a mixed bag of what exists there that still needs to be fleshed out. So I'm going to change the question up just a little bit for you Fred if you were to look at the vendors because Michael made a very good point and there's a number of probably vendors in the room I'm sure what are the key messages that Verizon because I think you're actually hosting this week NFE 12 what are the key messages you're giving the vendors to come to you with things that can be deployed or implemented can you give us a hint on what you see there? Again we're encouraging all of our vendors to provide NFE solutions for your virtualized network functions and to be independent and I think this is some of your topics independent of the between the virtualization environment and the hardware they're running on they really need to be in a mode that we can have a different hardware vendor a different open stack vendor and a different V and F vendor and I think there's today not enough standardization in the environment to make that work cleanly and so I think we're trying to identify what the gaps are today goes with various standard organizations to make all that work again we're asking all of our vendors to support us and provide understanding about what the gaps are how they can modify either their applications or their infrastructure or the hardware to make some of that environment work better what's the joke if one standard is good 20 of them Ray from your perspective what are you asking the key vendors in China for your business actually the same we are incurring all the industry partners to cooperate with us including the traditional telco providers also the IT partners because now we as a new concept it includes the IT platform and also the traditional telecom software to combine all the resources to make the reality because it's a huge challenge now one of the things that we see from a red hat perspective and I'll convert to a question is the traditional vendors the network equipment providers like the Cisco's, the Alcatel Lucens, the Nokia's and so on and so forth they're having to adapt their business model away from providing traditional appliances and becoming more like systems integrators which begs the question do you see incumbent integrators coming into the space providing you the best of breed solution and actually working with you to do that or do you think that's a bit too much of a risk in terms of their understanding of the network so I'll ask Ray first it is a very difficult question currently we can't say which is better the traditional telco providers or the IT platform providers so we are doing the test by different method we are encouraging both the traditional providers and also IT providers they can both be the integrator and we will see the result which way is better is more suitable for us so it's a learning exercise as this business is transitioning Michael from your perspective what do you see it tell us I'm expecting that our vendors have better well provided open standard compliant hardware and devices I want to see switches that support open flow for instance I want the option to be able to use the vendor hardware however I see fit rather than being locked down to a proprietary use case I certainly agree with all those things and I think one of this challenges that this whole environment brings from Verizon's perspective I think we need to take on some more than integration role ourselves because we don't necessarily have any single integrator that has all the knowledge about our network as well as what the application that run on it we don't have the right solution yet I think we're still exploring what the right combination of vendors is and who the integrator is I think internally we're currently being the system integrator ourselves this is all part of the upgrade and reform that we're talking about yeah right you want to say something yeah we are also trying to to develop the specifications the requirements also by ourself based on OpenStack to see the possibility of China Mobile as an integrator ourselves yeah so one of the things we talked about big vendors and one of the things that we see in the marketplace is a lot of small ones that are bringing very innovative solutions so with the theory that innovation sits elsewhere and some of these guys are positioning themselves to work with China Mobile or Telus or Verizon and it's very difficult to engage because you're very large, complex businesses in your own right any advice for those vendors who may be in the room it's kind of a difficult question as well a smaller company, how do they engage should they work through a bigger vendor what would your advice be at the top of your head I'm going to ask Freddy I know he's racing to think of a good answer but I won't I'm not sure I have a great advice certainly I think you're right Verizon is a huge task and if you're kind of a small vendor and not prepared for the kind of the challenges of dealing with a large vendor like Verizon you're probably not going to be providing us with the services we need but I also think from Verizon's perspective that a lot of small vendors provide innovation and as well as motivation for some of our existing vendors to modify their behavior and innovate along with them and so I think there is both a direct usage perspective from innovation of small vendors with them Verizon as well as hope that some of that innovation feeds back into the larger vendors Michael what do you see it to we speak to a number of gaps that exist I think that the small vendors may be able to provide some solution that may evolve from being a short-term solution that covers some gap that exists and to being part of a larger solution if they're able to close that gap address some of the challenges we were talking about in terms of carrier grade from your perspective how does a small vendor engage with China Mobile actually by introducing the VNF software and IT platform at Carpool which lowers the gap of the small companies and we already have some very small companies which just develop VNF software and already contest for the future test and the IT platform is still making it easier because there are lots of internet companies they can develop the OpenState platform just based on our specification meet our requirements then they can join so we are encouraging all the companies not only the big ones but also the small ones, new ones that's great well I think but what I would like to do is to thank my panelists for their time, we covered a lot of ground with three very distinguished individuals from very major carriers around the world and I hope that we gave you some good insights in terms of some of the things that are top of mind and top of the agenda I'd like to thank Ray, Michael and Fred for spending time and sharing some of their experiences good, bad and ugly as it may well seem but I'd like to thank them and I think we can close the session because I think it's how many minutes we got but feel free to reach out to me with any questions I know that I think you can reach me through the names that we have posted and I'll be glad to answer as much as I can, any questions you have Great, thank you Michael Thank you