 test Okay, welcome to Intel session Today Our topic is let's accelerate the design process of hardware device Okay, let's begin This is today's agenda. We will cover the following topic introduced the user scenario and firmware validation Compatibility validation and show the benefit here is a real user scenario from Shanghai hardware Validation team this overview is automated development Validation cycle let's quickly go through the workflow at first after developed modified firmware component and Check code into the source code repository this action will trigger the CI system to pull the code from the rebel and Asigned a compiling job to the build agent VM each component will be merged into one firmware image And then the CI system will flash the image to the specified SUT after that the system will run the test case on yet and Collect the result at last the subscriber could receive the report mail And our mission else We focus on the Red area and aim to help hardware team to improve the process What's the problem statement in this case and the hardware validation team needed to test the mainboard Call it as system under test Acronomy as SUT you can see the left instrument Left illustration. This is the original logical topology of one set of the SUT test platform there were two instrument between the automation test host and SUT And The left one is the control box the test host will command the control box to Through the USB cables and then the control box will operate the SUT to flash the firmware and around the test case The other one is the IP based KVM The test host used yet to record the mainboard video output for image comparison and ensured all the test procedure could be traced back And The middle picture is a real photo from hardware teams lab You can see it is messy on the bench and take a lot of space and Hardware validation team have to purchase the Desktops to act to the role of the test host because they'll test framework decide one test host Correspond to the one SUT but not only that when bench could only place two sets of SUT test platform so lab space cannot keep the pace with the SUT number groups. So engineer have to share the limited lab space as for different SUT as and it's It's difficult for them to set up SUT as many as possible in the same time Here's our improved solution Ideally without change the CI framework. We use the VM to take place with the physical automation test host and use the USB over Network solution to connect all the physical devices to the VM Set up NUC as USB agent server to Converge the USB uplink and then redirect the USB connection to the specified VM and meanwhile we use the Multi-port KVM to replace the single port model. The result is there's no physical test host in lab All these devices in the topology could be mounted in the standard server rack So make them make their life easier Okay Sounds good first problem is how to redirect a USB connection. Now, let's put a question to Zhenzhen Thank you Mo. So We have moved all the test host into the our open stack cloud And then we also Consolidated all the USB cables into an USB cable appliance So how could the test cases in the cloud to still be able to use those USB devices? It's obvious that the USB appliances should redirect those USB devices Over the network. So we have two options here This one is the first option is to illuminate the USB host of controller interface that is our HCI in virtualizer test assistance. I mean the guest OS Actually, this is the approach of for most almost all the commercial Solutions and we also found an open source project called the USB IP Of course these solutions can work but What's the question? What's the problem? So this first is a traditional virtualization way It's difficult to manage in our open-style cloud environment and we Of course, we also want open solutions Just like most of the people and we don't want when they're locking So unfortunately The open source project at USB IP seems to not actually maintained And it's not good enough to be used in the production environment so another option is to Enumerate the USB HCI in cumul either hyper risers Fortunately, we already have support from hyper risers either hyper way over the IDP or spice for cumul As we are using the KVM. So we did some investigation for cumul We have two options for the cumul solution One is the TCP redirection and the other is a spice USB Redirection channel both of them depends on the same open source library called the USB redirection So to emulate the USB HCI in cumul we need to thanks Red Hat They provide the USB spice protocol with USB redirection channel in the spice which is a stable and easy to use but You have to use it in the spice client because it's a one channel of the USB protocol So we are bonded to the spice protocol So this is not good is not feasible to use in our environment Because we concentrated all the USB devices into the USB appliance so we cannot to run Tens of us spicy clients in just one USB cable appliance and The person had to sit down Before the screen of this USB appliance clients appliance so and The USB spice candle also does not share the USB channel at the moment. So We cannot use this in our environment So in the other world in the other side USB TCP redirection, it's not a bond with any special display protocols So it's just implementing the cumul hypervisor and it's simple and the pure So we can be it can be used with all kinds of access methods. For example You'll redirect this USB device over the TCP redirection Socket and you can access the virtual machine with either SSH or VNC or spice You are you can always get the USB devices redirected to their guest So but it also has some downside to Currently the USB direct server is too simple. We do not find any Good solution for the USB redirecting server In the open source. So we just got a one test us code from the library USB redirect So another another problem is that the code in the queue side You can only try to connect to the USB redirecting server at the boot time So if when you are boot the instance or the the guest if the USB direct server is not ready Cumul will die. So this is it's not good So with with this limitation, we won't have Reconnection and hotplug support, of course We can improve this For example, we can have an agent running in the same machine as the cumul runs and the agent We are fixed. We will use the agent to fix the recollection problem and the hotplug problem so With this agent we can also because we don't put code the end of code in the cumul process So for any kind any version of the cumul Which supports the TCP reduction we can use it so we We don't need to depends on Ops in or put merge our code into the cumul process But it's still not good enough why because We are not using the traditional virtualization way. We are using open-style cloud So we still need a way to work with the cloud cloud environment This is our answer so we are Planning a new project called a potter It's going to be a service to manage All network redirectable hardware resources So the user can use those Resources in an on-demand and self-service way just like a single volume or For example, ironic notes, right? We can add the add you this kind of redirectable resources and Then attach this resource to a specified Instance so here's a high-level design Well, how to say is is a very early stage. We're just a planet so At the left we have hardware direct services To export the hardware resources, of course, they have to have to be able to be rejected over the network in our case it's a this is the USB resources and They are running in the Consent data USB cable appliances The We have a part agent in the right side, which is running together inside of the node computer the computer node so with with this agent the Instances running by a cumul they can connect to the agent at first So because the agent can always run it, right? So we don't need to well the chemo wouldn't be there and then we always have the connection and the agent will also help to fix the reconnection and Hot plug kind of stuff So we at the top at the top layer. We also have a part of CI which will Find this hardware resources and register this hardware resources into the part of service So the part of service is a key to manage all the resources it will Manage the resources and also when you try to Have a boot instance and then you want a one USB USB device to connect it is USB this instance you can use this CI to say but attach this device to this instance so With the help of the part agent The part agent will then see establish the connection to the hardware with their services In some remark, we also have to use them a part of proxies just like as a no VNC proxy because of the network may be not can we they cannot be connected directly with the Computer node and the outside network With regular service So as just I said, it's still in the very early stage and we will appreciate all kinds of any kinds of help from the community That's all any questions Okay, so I know I will Give control back to moon Thanks, Jen Jen's answer tell now. That's all for the firmware Validation how about the compatibility validation and telling it's your time for always provisioning? Okay, thank you. I will talk about the OS validation like the competitive edition But before that I just want to make a short survey I Just want to how many people in this room know about ironic before so please hands up Wow, that's more than I expected Okay, then let's start So recently we have our internal internal customer to build a deploy system based on ironic our customer is mainly working on hardware design and provisioning and They will want to validation their device for the functionality So they have a very strong and high frequency request to do the OS provisioning So then we to we introduce ironic to them so here is our Abstract infrastructure of our solutions So we use ironic as a standard project without any other Projects in open stack and this it is located in our at data center and using internet work ironic is able to access the And the hardware device like seo t1 seo t2 and seo t3 which are located in hardware validation lab and ironic is able to do the Control this device and do the provisioning so Even some of you already know ironic, but I would like to introduce ironic again So what is ironic ironic is an integrate project of open stack It provides a Biomental as a service So you can either use ironic with other projects That's neutral or Nova, but you can also use it for standalone like what we did So ironic is mainly make use of pxc technologies But it also has some many advanced features like the First is the BIOS configurations and the BIOS upgrade and also like disk arrays So here here ironic is able to fast deploy the probe OS and So here is a Simple use case for how to use ironic So we have one central node which is running ironic service like the ironic API and ironic conductor and the database and We have a bunch of bare metal nodes which without OS installed so The admin it's only he he's only need to send a simple request to the ironic service and says hey, I want to deploy on Fedora 20 on the node one and then ironic will help will choose a note one and and Set the node to pxc boot and using ascasti and the DD to deploy the Fedora on the node And then then in in two or three minutes you get You get a system install on the node. So it also works for Windows That's pretty simple and fast so first At the first stage when we talk about our customer we want to use ironic directly, but unfortunately It's not gonna enough Of course, it's not because of ironic is not good enough. That's because our customer Have some specials request But they are our customer, you know that we have to satisfy them So They have some request like they want to deploy the system with your efi and the legs they sit and They also want to make use of when pe when pe agent When pe the small windows image is it small wins small windows systems It is running on the memory. You can it's a very useful tool you can use that to manage your bare metal machines and Deploy windows images Our customer are very familiar with this workload. So they want to ask to help them to keep this one So so then we decide to use ironic to put when pe agent directly to the dedicated nodes and The search requires a little special because Our customer is working for the hardware device Validations they want to validate the compatibility of their device with the drivers So they don't want us to install the Image Install the window complete the windows image. So, you'll know that during the windows image first install process their step to restart the system and Load the driver they want to keep this step to validate the functionality of their device so we will provide a semi-finished product of the windows image to the past this image to the node instead of Provide a complete image. So with that we can mimic the driver instance driver install process so after Many rounds of discussion and redefine redefine Our solutions finally we get a final usage scenario So the deploy system will still has a regular PX manual and The device I mean the SUT will put into PXC put into when pe agent as default But the admin is still able to select Dedicated OS and the select the dedicated node to install to install using ironic So some future work. So our citizens work well, but Currently we are only able to use command tool to send request. It's not good enough. I mean it's not vivid So we want to create a dashboard So in the future our customer should able to select get the status of their nodes from the web and They can select the dedicated OS and dedicated nodes from web as well and Another work we want to do is for enable ironic advanced futures Like hardware management and something else. They are very helpful and useful for our customers but this next step and Personally, I think ironically they are very useful and well developed software So I would encourage everyone who is interested in this to play around by yourself So that's all from my side. Thank you Thank you. Telling introduction all being said it is important to show the benefit About benefit the first one. Yes Previously a user could leverage the IT OS provisioning tool to rebuild the desktop OS via the network and then to set up the test environment by themselves After they migrate the test to platform On the cloud they just need to renew the template and quickly replace the bench of the test environment on demand And also it is easy to extend or shrink the scale The second one yes after the couple the cable connection between the SUT test platform and Automatic test host we can easily to switch the USB connection from the SUT to various A test or debug VM environment user don't need to move the SUT to the other place in order to connect different Work environment in different workflow stage. So it is flexible for development and validation transition The third one yes after the whole workflow running on the cloud It means the test the test case the report a benchmark and others value the data will be saved on the cloud no data right to Desktop in the custom lab and rely on the access control on the cloud It is good way to prevent the information disclosure and hardware disaster You can see that we could only put two sets of the SUT Test platform on one bench before now we can land eight sets of the SUT Test platform on the rack actually we can make it higher density than eight So one rack could help to replace two bench lab space at least After convert to the square feet the saving proportion is one to ten with increasing number of the SUT test the requirement. This is good proposal to prevent the lab lab space sprawl Also, this rack infrastructure support a various SUT model such as the server board the mobile device board and etc and Also the rack tray could be customized for different SUT or peripheral instrument Okay That's all any question Okay Excuse me You will bring the ironic and The part I know you have the part of what the next statement you said. Ah, okay So for for the product project, right? Yeah, it's still in the first stage and you know for We we have a several components to be implemented Yeah, this one is that we need to enhance the USB redirection server so For the proxy at the first stage I think is it's okay without this But the port agent we already did a very initial implementation This one we are trying to Leverage other open stack project as code. So we plan to be Get the first Initial I mean the initial risk in the M risk, maybe Yeah So does it looks like interesting for you? Thank you. Ah, not yet. So Yeah, the plan is to we are get the first initial release in the M risk M cycle and we are definitely we are put into the GitHub We we want to you know as a big patent model Obstacle foundation is encouraging everybody to put your Projects under the cover of all the stack, right? So we are trying to do that and we also of course we also leverage a bunch of offer all the source code is so Stay tuned In our case, this is a USB redirection service USB redirection server. So it's a if you have if I were other devices for example, maybe RDMA or other slow slow speed PCI devices which if they if they can be over redirected over the network So we can use use this use the portal to manage them Yeah, we use this to redirect the USB devices into the instances running in the cloud How? In our case that we you know, we have a USB redirection protocol, which is implemented by I think it's a very high guy Yeah So we also talk about as I like we can use the spice right spice also have a USB reaction channel But it's a bundled into the spicy protocol and you can only use it with a spice kind so What we both probably both it's a little bit of Cp redirection. That's that's a library. Yeah You mean the library or our code? Hmm. Yeah, that's I think it's I don't remember it But it's on the get up. It's on the free desktop GitHub git repository. I think it's Apache or MIT. I cannot remember it any other questions Okay, so stay tuned for the portal project and we we are appreciative any kind of help from the committee. Thank you