 רבדי, אני שרון גראצ, אני סינר סופטור אינגיניורט אובירטים, רדאט ועכשיו אני אעשה על הפרופורמוס וירטואל-משיאנס שזה פיצה רשמכי עדד עוד אובירט 4.2 והיה עמסת אובירט 4.3, שזה הרבה אובירט פרופורמוס בואו נתחיל, מה זה פרופורמוס-VM פרופורמוס-VM עוד פרופורמוס-VM בגלל הכ הרבה אובירט, ולבי פרופורמוס-מציגים כדי לעזור קמות מהרדאט יש כוסף שפędשים מוסוatched בה עמסת אובירט ועד הירט פרופורמוס, ורדאט אז עבור אובירטי פרופורמוס ועמוד אופרופורש, שאתה יכולה לעשות בבקשה בבקשה בבקשה, ולכן אנחנו רוצה עשית בבקשה בבקשה בבקשה, כמו שאנחנו יכולים. כשאנחנו עושים על הלב, אנחנו עושים more about a predictive issue. We're talking about predicting that a set of operations will be able to be completed successfully with a result in a given amount of time. So, real time doesn't necessarily deal with high performance. It's two different things and we are not supporting real time for now anyway, only high performance. So, that's a high performance VM. Let's another thing that I want to emphasize is that in today's presentation I want to show no numbers and no graphs because showing the performance improvements is a bit tricky because to show that the VM performance was improved it depends on a lot of things, either on the VM itself and on the host that the VM is running on. It depends on the settings, it depends on the load, on the memory usage, etc. So, instead of showing you graphs of how performance improved, I will just stay on describing the new feature, what we did and how we simplified things in this feature. So, why we added this feature? Why support our performance VM? Why we needed that? So, first of all, application requirements. There are a few applications that require better performance than others, for example, and this example is taken from OVRT, actually, SAP HANA. SAP HANA is an in-memory data-based management system that is processing a huge amount of data in a short amount of time. And SAP HANA could not run on our VMs, only on bare metal machines. And they asked us to implement some kind of easy way to create our performance VM so that they can run on OVRT machines, on OVRT VMs. So, it's just an example but there are other applications like that. Another reason for this feature is that actually users could take the old VMs even before this feature and tune the VMs so that the performance will be better. They just needed to go over the VM settings one by one, understand what they are doing, search for the relevant settings and improve the performance, but it was not a straightforward mission to do. And now it's much more easier. Another thing is live migration. Live migration is a very important ability for VMs. Live migration means that a running VM can switch to run on another host if required in a minimal downtime. And live migration was not supported for such high-performance VMs before and now it is supported. And last reason for implementing this feature in OVRT was that there was a lot of missing functionality and we implemented a lot of additional functionality now for improving the VM for performance. For example, we added support for Atlas VMs, we added support for huge pages, for pinning IO and emulator threads, and for multi-cues per virtual interface for networking, etc. Now, when you create a VM in OVRT, there is, you need to give the name of the VM and other settings and one of the settings that you also need to give is the VM type. It's called Optimize 4. And the reason for this field is to know how this VM should be used. Should it be used as a server machine or as a desktop machine? And upon this feature, we will know how to optimize the VM, which devices should be removed, how we should set that. So we used that field, the Optimize 4 field, and added a third value called high-performance. And now when you have an application like SAP HANA that requires high-performance, like the yellow here, yellow application here, then we ask the user to set the VM type as high-performance, and then automatically the VM will be high-performance VM. This is how it is shown in OVRT UI. This is the UI for editing or creating a virtual machine. So you can see there is the Optimize 4 field and there are three values, server, desktop, and now the new value, high-performance. And once you set this value, you will get a high-performance VM. So it looks very easy. It is easy, much more easy than before, but not that easy because by choosing this new high-performance VM type, actually, behind the scenes, two things happen. One, automatic settings. The VM will be, as I said, preconfigured automatically with a set of configuration settings, recommended settings so that the performance will be higher, but in addition to that, there are a set of few settings that cannot be done automatically because it's very hard for us to calculate in advance what will be the host that the VM is running on and exactly what should be tuned on that VM. So there will be still a list of short manual settings that the user will be recommended and asked to set, but we will help him to do that and check if it is already set or not. It will be much easier for him to understand what he should do and I will show everything later on. Two things to emphasize about that is one, nothing is mandatory. All the automatic and manual settings, a non-mandatory user can choose if you want to set all of them, none of them, part of them depends on what you want to do. And another thing is that you can do that for a new VM or existing VM. You can take a server VM and switch it to be IPerformer just by changing the VM type and setting the manual settings. Now, you can ask me the following question. If it's so easy now to create an IPerformer's VM, just choosing the IPerformer's VM type and then setting a few manual settings and by creating IPerformer's VM type, I get a VM with the highest possible performance as close to bare metal, actually. So why not creating all VMs as IPerformer's VMs? Sounds reasonable, right? So the answer for that is that as everything in life, everything good in life, there is a price for that. And the price for the IPerformer's VMs is flexibility. So there is a balance between performance and flexibility. The higher the performance will be for the VM, usually the less virtualization flexibility you will have, for example, when you create IPerformer's VM, automatically you will not get a graphic console. You will not get a USB device. The list of hosts that the VM will be able to handle is limited, more limited than a server VM, etc. So there is a price to pay and you need to understand that and balance between those two according to what you need. Now let's drill down to talk about the list of automatic settings for the IPerformer's VMs. So everything I will say now is automatically set for you and will increase the VM performance, should increase the VM performance. So in the console area we are talking about enable-adness mode. And in addition to that enable serial console. Adness mode means that there is no graphic console at all but you still have the serial console to manage the VM from remote, if you need to. Regarding the VM devices, this is the list of all the VM devices that will be disabled. I'm not going over them, you can read them now, but this will be automatically disabled for you by default. No USB, etc. Regarding networking, we will enable the multicuse per virtual interface. That means that if you need to serve multiple networking requests, you will have multiple queues for that. Everything will be handled in parallel, which is great. But you need to keep in mind that for managing each one of those multiple queues there is a thread, a virtual thread. And this virtual thread is served as for managing this queue instead of serving for processing, meaning instead of serving as a virtual CPU. So if you know you have a VM that running with a load, a networking load, then please stay with the recommended settings. But if you know that you won't have a lot of networking load, but on the other end, you need a lot of processing load, so you will be able to disable that. Now regarding entropy, we will enable the random number generator to prevent entropy starvation. The same entropy as the same random number generator as done in the host itself. And regarding CPU, we will set the CPU cache layer 3 in addition to layer 1 and layer 2. I'm continuing with the automatic settings. So regarding storage and IO, the disk interface by default is set to SCSI. The storage allocation for disks is cloned or preallocated so that read write operations will be much quicker. And we enable the IO threads while the default we choose is 1, meaning that there will be one thread dedicated to serve for IO requests. Now regarding the host pinning, we will do two things automatically. First enable the pass-through or CPU, meaning that the CPU model and the CPU flags used by the host will be identical the same as the ones used by the VM itself. No layers in between, that's why everything will be much higher, performance higher. And another thing is we'll set the IO and the mulatto threads pinning topology. Those two things, everything regarding pinning will increase performance much higher than other things. So those two things have a big influence on performance. Once you let them, you set them. And the last bullet in automatic setting is live migration. That means that we will enable live migration. And this doesn't have anything to do with performance, but we still need that because we want live migration as we talked about it before. Now I want to talk a bit about migration. So when a high-performance VM is migrated from one host to another, the performance may be decreased because you move from one host to another and you don't know if the destination host has the same settings as the source host. Therefore, we chosen so that the default mode for migration will be manual and not automatic. Meaning that we want the user to control the migration. So for high-performance VM, the manual, the migration will be manual. The user needs to trigger that. Although automatic migration is still supported. Another thing is choosing the destination host. Source and destination host should not be identical. We don't require that. If they are identical, that's great. That means that no performance decrease will be done. But they don't have to be identical. What they do have to be is they should be compatible. Because otherwise the VM won't be able to run on the destination once the migration is done. And they should be compatible in everything, in the CPU, in the memory, in the huge pages, on all the host settings. Now, because they should be compatible, the list of host that can be supported as a destination host for the VM is limited. There is a subset of cluster host in comparing to a regular server VM, for example. And another thing to consider is that not all those hosts that are compatible and the VM can run on will supply the same level of performance results. Because it depends on the load that the host have on that moment. Free memory that is left on those hosts. So what we recommend you to do is let all viroch choose the destination host for you. Not choose it manually. You can choose it manually, but the difficulty is to let you will do the manual and migration and let over choose the destination host because you can calculate all the details and all the parameters together and choose the best host for running the VM. Now let's talk about manual settings. So there are actually four manual settings that we asked you to do. The first one is CPU pinning. Set the CPU pinning. CPU pinning means that each virtual CPU is pinned to a physical CPU in the host. The next one is declare virtual nominode and set the noma pinning topology. Nominode mean that the processing unit, the CPUs are closer to the chunk of memories that they are used. That is a noma. And of course that if you do that the processing is quicker. Now we don't just say please declare virtual nominode on your VM. We also say pin the virtual nominode to the physical nominodes on the host. And that way you will run as close as you can to the bare metal machine because there is no layers of translation in between. That's the second thing that we ask the user to do manually. The third one is huge pages. As you know, memory divided to pages. The page should be managed and in a table, TLB table, as large as the table is, the more processing you need in order to manage that. So we recommend to declare pages with the highest size that you can, huge. If you declare huge pages, the result is that there will be less pages to manage. Less pages to manage, then performance will increase because the TLB table size will be smaller. And the last bullet is disabled the KSM, the kernel same page merging. This is a kernel same merge merging means that you share the same pages from among, if they are identical, among all the VMs and use only one copy of them. Of course that we don't want that because once they are changed, you need to manipulate a change and create copies and the deltas and everything and it should take time. So we disable that. I want to emphasize that the first three, CPU opinion, NUMA and memory, and huge pages, memory backing with huge pages have the biggest influence on performance among all others. So although they are manual, we really recommend to set them. Now few sentences about huge pages. First of all as I said, we need to declare the huge pages size as bigger as we can. But of course it depends on the pages size that the host supports. So if the host supports few pages sizes, then we'll choose the biggest one. But we should consider the three huge pages that is left on the host. I mean if there are few sizes but I need amount of huge pages for the VM to run with then I'll choose the one that have the most number of three huge pages that I can use. And another thing that huge pages are preallocated when the VM starts to run and not dynamically. That's the default mode that we choose anyway. Now I want to drill down a bit to the pinning issues that we described before. We talked about the manual. We talked about the automatic. Now let's drill to the pinning issue. I mentioned before that we automatically set the IO and emulator threads pinning. So the algorithm that we choose for that is that we look on all the physical nomenodes in the host. We'll take one nomenode, choose the first two physical CPUs. Usually it will be 0 and 1 but not always. And pin the IO and emulator threads to those two physical CPUs. If VM spans more than one nomenode, it can be, if the VM is larger, then I will take only one physical nomenode and pin my IO and emulator threads to that nomenode and all others will be used for virtual CPU pinning. So I won't take two from any one of those nomenodes only which is only one. Now let's see everything in a graphic use case so that it will be easier. So here I have a host with two nomenodes, nomenodes 0 and nomenodes 1. Each one of the nomenodes is set with the memory size and the CPUs. And here is my high performance VM which has two virtual nomenodes, virtual nomenode 0, virtual nomenode 1 with this again memory and CPUs. And I also enable an IO and emulator threads as we said before that we should do. And now I want to start the pinning. So let's start by pinning the manual pinning. So let's start by pinning the virtual nomenodes to the physical nomenodes that we said we want to do. So I have two options to do it. That I can pin virtual nomenode 0 to nomenode 1 or virtual nomenode 0 to nomenode 0 in the host. So of course the option that I will choose is pinning virtual nomenode 0 to physical nomenode 0 because each virtual nomenode should be at least contained in the physical nomenode. It's not identical. And if I'll use the other way around then I'll have three physical, three virtual CPUs in my virtual machine while in the host I have only two if I'll do virtual nomenode 0 to nomenode 1. So that is the pinning that I will use for my virtual nomenode. And after that when I complete doing that manually by using the Overt UI I'll start pinning the virtual machines. Sorry, there's the threads, the virtual threads. So the way I'll choose to do that is like that. As you can see the pink arrows are for pinning the virtual CPUs to the physical CPUs 0 to 2, 1 to 3, etc. And we keep on the logic that no cross-nomenode pinning is done because if I will pin virtual CPU 0 to physical CPU 5 then I will pin for a nomenode that I'm not that the virtual nomenode is not pin 2. So there will be cross-nomenode and the whole idea of performance will be lost here. So every virtual CPUs so the pins inside the virtual nomenode that I'm pinning the nomenode 2. And another thing to mention here is that 0 and 1 the physical CPUs 0 and 1 are left for the ion emulator pinning. So I won't pin my virtual CPU to 0 and 1 because that way the physical CPUs will use for both virtual CPUs and ion emulator which is not the case. So I'm leaving them only for that and that the blue line is done automatically. So that's regarding pinning all the all the pinning together. Now future improvements. So first of all we want to improve and change all the automatic pinning to be all the manual pinning sorry to be calculated automatically. So we'll set we want to set the virtual CPU pinning and the virtual num pinning automatically and we want to calculate and set the virtual huge pages automatically that's that's what we want to do in next releases. An additional thing we want to do is a set of finity rules management for managing groups of high performance VMs and host. And last thing is of course continue tuning the high performance VM solution according to future benchmarks because we always found more things to improve on that area. And that's all for me for my side I will be glad to hear questions if there are any. What are your SCSI for the high performance guests? mm-hmm I've heard that but our block is a little bit faster sometimes. So we it depends on a lot of things okay you ask why we choose a SCSI controller for the disks and not and not blocker What are your blocker? and not block because we again it depends on a lot of things but we check that with IOS for the enabled and so that in most cases SCSISI is quicker but it's not yeah but it's not you know it depends on a lot of things so again it depends on you can you do your test and see that for your specific case it's not there regarding the CPU pinning do you have configured the VM in a way that you're focusing on hypo threading CPUs so did you disable hypo threading on those systems or did you configured the VM to have hypo threaded innate it within the VM and pin those cores to the hypo threaded cores on the OS system yeah we okay can you repeat the question again because I'm not sure I am anything so your CPUs you usually have hypo threading in our case we have enabled hypo threading and configured our VMs in a way that they also have hypo threading in the VM okay we try to pin those hypo threaded cores the virtual hypo threaded core to the real hypo threaded core but this is not that easy and I wanted to know how you manage those stuff or if you just disable hypo threading at all we are not disabled in hypo threading but okay he asked if we use hypo threading and now we pin the virtual CPUs to the physical CPUs on the OS so we are using a layer called live view for that and live view does that for us so it's pretty transparent for us we just ask live view please pin that virtual CPU to that physical CPU and live view does everything behind and see if it's possible you know we start having so it's much more easier for us because we have that layer that translated but I hope I answered your question one more yes perhaps I see the question that is there a command line interface to ask please optimize stuff yes we have what it is yes we have the rest API as for everything in Overt we can do it via UI okay he asked if we have a command line for doing that or you need to use the UI right so we have the UI and we also have a rest API for doing everything in Overt in addition to that is the name of right on the board what is the name of I don't remember it now it's something like createVM or addVM but the main issue is that we use the an existing command line it's not a command line it's more a script that you need to create using Java or Python yeah and we didn't add a new we just added a new type so you use I mean you need to see how we use a rest API for all other things and just add a parameter saying the type is a performance that's all no time so no more questions thank you