 Good morning. Welcome to this very first session this morning. We have such a huge space and few audience so you can feel free to ask any questions after the session. I'm Fuqiao from China Mobile. I'm responsible for the China Mobile OpenMV Test Lab and also I'm engaged in the OpenMV community. Wanli, would you like to introduce yourself? Hi, I'm Leo. I'm from Huawei and I'm now working on the OpenMV testing projects. Okay, so this session we would like to share the ongoing work and experience we have in China Mobile when we're doing large-scale integration and testing. We would like to show the difficulties and problems we have and how we solve this problem using the open source tools that OpenMV provides us. So this is the logistic of today's talk. I will first introduce the NFV integration and testing work here in China Mobile, our requirements, our scenarios, and the automatic integration testing framework. And then Leo will give more details of the open source tools we're using and finally show the demo. So the key words in this session is network function virtualization. The concept of NFV is to reconstruct the current network made of lots of bulky physical hardware into virtual surfaces running on the cloud. So through these years, NFV has actually developed from single concept on the white paper to frameworks that proposed by SNFV and now to lots of experiment field trials and even some commercial deployments. Here I also listed some of the VNFs that has been widely considered as the first group of VNFs that could be virtualized in the cloud, like virtual IMS, virtual EPC, which has been talked about and discussed by lots of people in an NFV area. Most of the tier one service providers actually have began their experiment on NFV for several years. In China Mobile, we actually began our NFV related testing a few years before and we established our open NFV test lab back into 2015. And late last year, we have began this work on what we call the NoverNet experiment network. It is composed of multiple data centers from four provinces in China. We will do multiple virtual service testing. The virtual services that I mentioned in the previous slides will all be considered in this network. And then we are going to, we are doing the integration and testing for multiple infrastructures, hardware and the virtual services. By doing all this work, we hope that we could figure out the problems and difficulties when we are really doing the field deployment and try to smooth the whole procedure and to reconstruct our future network. So now we have at least six infrastructure vendors that we are considering this network and we have like five services each with two to three vendors. So you can see there could be lots of combinations of the infrastructure vendors and service vendors and the integration and testing work is tremendous. It is almost impossible for us to work all this work manually. So that's why we propose this automatic system. And these slides show how our future network actually structured. We say that it was structured with multiple data center, which we called telecom integrated cloud TIC. TIC is actually just standard unit for our future network. We say that it should be structured in a layered manner, taking consideration of the telco nature services that it actually supports. So you see that it is, at least point, it is different from the private and public network. So we say that it was in two layers, the upper layer is what we call the core TIC, which basically supports the control plain services. And the lower layer is called the regional TIC, support the data plain services. We actually did a calculation of how many TICs we should have when we finished the reconstruction of the whole network. We do that calculation based on the throughput of our current network. And the result is like 50 to 100 core TIC and maybe stayed in most of the big cities in China and 1,000 to 3,000 regional TICs stay quite close to the end users in counties and towns. So when you zoom into each TIC, you can see that it's quite a standard unit for NFA deployment. We will have limited number of design template. So what we mean template here is nothing of the hate template or things like that. It is actually means a set of choices of the infrastructure, the VM and the illustrator. So we say that we could have, we think of four templates to support multiple VNS of different kinds. One, the first template could be a template for the control play, for example, virtual IMS, which will stress around the compute capability. And we will have template for data plain services like virtual EPC, the stressing on the data forwarding capability. And we will have template for edge services like virtual CPE that will stress on low cost and light weight. And finally a template for storage services like virtual CDN, which stress on the storage capability. And also TIC will have unified hardware models and network design. Taking consideration that this TIC things is quite standardized, you can have a quite standard integration and testing procedure. So here I just to give a quite generic view of how this procedure should be. First we have all the hardware prepared and configured, of course. And then we will have this what we call the platform to be ready and tested. This will include the hypervisor, the VM and the illustrator. And finally we will have the VNF deployed and tested and finally the service launched. So here is how this automatic integration and testing system will look like. We basically use Jenkins to help us manage the different workload and automatically execute different integration and testing tasks. And we will have a Jenkins slaves responsible for each TIC to do all the integration testing. And on the slaves we will have like deployment tools, testing tools, and we will consider of certification tools as well so that we can do the compliance and the certification using the same framework. We also have Git and the system to help us manage the test cases. So here we have, we now have the open source tools that OpenNF will provide us using on the slaves. So you see we use compass as the installer to do the deployment work and we use fun test and yardstick to do the functional and performance testing. And we use dovetail to help us do the compliance and certification testing. But you see that this framework is rather open so you can easily add on more test cases like proprietary testing tools in this framework. For example, we will consider adding proprietary testing tools from XEIS, Barron, and etc. So here comes the flow chart for all this procedure. You can see that the blue box actually means the deployment. And the purple box is the testing. And they all share a root cause analysis system in the red box. Anything that goes wrong will go to this system to help the maintainers check out what's the problem it really is and how to solve the problem. So the basic procedure is like we first have the NFVI plus VM to be tested, to be deployed and tested. And then we will have the sample VNF deployed and do the onboarding test to make sure that we have all the network compute stars that is okay. And finally is the VNFs. We will have the actual service testing running on the system and final services successfully launched. I will give the rest of time to Leo to give more details of the open source tools we're using and show the demo. Hello. Since the system using a lot of tools from the open V, so I will introduce these tools one by one. Okay, the first one is the COMBUS, a powerful installer of the open V. It can install the open stack and also a lot of other interesting features to the environment. So it has a lot of features, but I guess there are some more. There are four key features that you want to know. The first one is that it can of course deploy to the bare metal. And it has a mechanism of outer discovery of the hardware and it can support the rise of the kind of servers and the switches and the storages. And also it's a package installer. It has actually its release is just a ISO of all kinds of packages for an off-line support that for now it's supported for the XENOS and the Ubuntu and also it supports a lot of open stack versions like from Juno to the latest Luden. And it includes the VNF, something like VMS, VEPC and etc. And also it's very extendable. It can be easy to add new applications to the installer. Actually, we now support the this COMBUS can install something like the ODE-R or something like a SAF. Also, we can add a lot of other features to the installer. And also it's the OS installer. It will use the cobbler for the PXE and then we use the open stack for bare metal and also it has an application management using the most famous compilation, ELK, to monitor and lock the application's status. And also it has a web UI. If you don't want to use its command like interface, then you can use this web UI to see every step of this installation and you can change the parameters and do some manually modification and then you can deploy this environment by yourself and with any features you want. And the next is the OpenMV testing projects. The OpenMV has been devoted to NFV testing for a long time. It currently has a lot of testing tools, something like the functional testing, which we call the fun test and the performance testing we call YAR stick. And also there's REST performance testing, something like the way as per focused on the OpenMV switch and the store per focused on the storage and also there are Qtip, bottlenecks. There are a lot of testing projects which have a different field to test. And here we can see to the left the first one is the dovetail. That is for OpenMV compliance verification. And I will introduce all these tools. The next page to the right there is the test tiers and we have a from the health check, smoke check and also something like WinF including a lot of test cases. So we can run tests from tier to tier. And here we can see this fun test cases including four fields. The first one is the OpenStack. Here are we PIM test cases. This test case adjusts to create VMs and verified connectivity. And also there is a rally bench test to benchmark the OpenStack deployment. And also we have a Tempest test to run OpenStack native tests. And the second field is about OpenMV feature projects. There's not a lot of feature projects. Like I said, the copper test cases focus on the policy management. And doctor test cases focus on the thought management. And the domino test cases focus on the templates releasing. And the multi-side focus on the multiple OpenStack cluster cascading. And also there is a promise test case focus on the resource, resolution and schedule. The SDN VPN test cases focus on the VPN. So you can see there's a lot of test cases. So we can use them to test your NFY platform to say if it matched your requirement. Also we have the third field is about the SDN controller. There are test cases for ODL and test cases for ONUS. And the fourth field is the weighing app. Here we take the clear water with IMS as a sample to test the weighing app. The next one is the yardstick. The yardstick, in the last picture we can see the yardstick is connected to the other to arrest the performance testing because it supports a plug-in mechanism that it can easily add the rest performance testing projects. And so all these performance tests can be in one project that we can run all these performance tests in one touch. The yardstick also supports the restful API that can be easily integrated with the other projects. And as we see, the dovetail can easily use this yardstick for its certification. And the yardstick also supports some system network configuration that it also can support test case control that means you can specify a single one test case to run or you can select a test tier that includes a lot of test cases. And there's a lot of ways to... to dispatch the test results. So you can save all these tests result into local fire. So you can review this data later or you can upload this test case to a remote database then you can... for persist storage. Or you can upload this test result to a remote flux DB and you can use the Grafana to view the data with all kind of charts and tables. And all these test cases are decomposed with the this testing framework that you can freely add or remove the test case without a change of code. And finally it's the dovetail. Dovetail is a project that delivers actually the test cases that from the upstream testing project used to say if the target system can meet OPMV requirements. That's just for the compliance verification. So you can see in this picture that after you run the test you can put the results to a remote DB and add the reviewer to check your data and give a report about your system that if your system is matched with the OPMV requirement and then they will give a certification. But for our system we just use it for testing so we don't have to upload the results now. Here we just use dovetail to test the MFI platform. Okay, so enough talking. We can say a demo. Since it takes a lot of time to deploy so we provide a video. Okay, full screen with the moment. Okay, here we first we will log into the Jenkins so we can see a lot of jobs there. Of course we have already configured these jobs so here we just point and click so everything is good. And everything you need to do is just to click the start button and then you can go out if everything is okay. And here we can see, let's go, this is the deploy job. Here we use this deploy job to install the open stack, the open O and the VMS or in one. So let's check this job. And sorry about this Chinese text. But I guess the most important part is in English. Here we can check all these parameters and if they are right, correct. And we can just start building. Okay, start building and we can see what's the output. So there's not a lot to trace the status of this installation. It takes a lot of time so here we fast forward. So it seems much faster than in real world. Okay, it may take several minutes so just take some patience. Okay, here is the package installation. And so here is the open O, the open O installation. So this is from sparring communications and while this is going, I thought maybe a good time for a question because we're going to this installation process here. What kind of test tools or traffic generators are you are using in your platform right now? Basically we are using actually for deploy, we're using Compass and actually there are four installers in the open way. So we choose the Compass because it has a lot of years experience about how to use that. And also for testing, we are actually using the most of the testing tools. Actually maybe all of them. So we don't have to worry that we are losing something. Okay. Okay, then we start to run the fun test to see if it's functional right. The same thing. So here we check the parameters, we start building and then we just wait for the, here I see. Just check the connection to the open stack. So we can see some open R state fires. And also we will check some resources and creating some resources for the test. And here is the test case. Just okay. Follow every test case. So everything is good. Right. Because we select a simple test case, so it takes not that long time to see the results to save our time, I think. Okay, after the fun test, we will say the yard test. And here we manually click all these to start all these testing projects. Actually we can put all these jobs into a pipeline that they can run automatically. So it's just for demo that we manually start all these tests. So there are all the same, just a lot of logs. It's done. Next one is the dovetail. Start building and we check the logs. They'll pull a lot of Docker image and start containers to run the test case from upstream testing projects like fun test, like yard stick. Here we just run the test case from yard stick. It's the HA test case. So it's very easy to add or remove this test case that you can decide what to test. Right. Here we can see a lot of testing result here, the past, all the test cases passed. So we get the success. Right. It must success. And then finally we will use OSD if the wing app is working. So we will make a phone call with this VMS. So we use a simulator to call the moment. Actually I can, so there are two people that using the VMS. So you can hear the voice, but we can see the warning is changing. So there are people talking in English. You can see there's an indication that they are still talking. Maybe they have a long chat. So we can wait. They really like chatting, I think. So that's it. So we can see that people are using the VMS to make a phone call. That means the VMS is actually running and working well. So any questions? I guess we have reached the top of the session. If you have any questions, you can come to us after the session and we will give the space to the next session. Thank you very much to everyone. Thank you.