 Tom here from Warren systems and this is a super micro ultra super server 120 you dash TNR Which is a bit of a mouthful to say but as you're about to learn it really does pack a lot of punch into a 1u server This system is here on loan from super micro So thanks to them for providing this unit for review, but unfortunately I have to send it back I don't get to keep it But let's jump right in the specification This unit supports dual third gen Intel Xeon scalable CPUs has 32 dim slots And can support up to 12 terabytes of memory at the front There are 12 2 and a half inch hot swapable bays that can be configured as SATA or MVME and are connected via PCIe 4.0 by 4 the drive bays are configured in a way that six go to each processor Covering the four total drives then we have four PCIe gen 4 16 slots Two of them are full height and one is a low-profile slot the 8-pin power connectors located on the motherboard Are what powers things and devices such as GPUs? So yes, you can pack some good GPUs into this powers provided by redundant 80 plus titanium powers Supplies that operate at 96% efficiency and the system is set up cool by eight dual four centimeter fans 16 of them in total that are heavy-duty and able to move a really high volume of air But that comes at the expense of making a lot of noise So this thing's not going to be quiet not at idle and of course not at all at full speed But it does do a good job of keeping the system cool And when you're dealing with something as dense as this having 16 fans is going to be really good Now the fans are configured in a redundant array So you have a fan at the front and a fan at the back So you only see the eight here, but actually there's two fans in each one of these modules on here Now this system was designed for high density applications. Like I said, you can put a few GPUs in here And this makes me worried taking this out But I will carefully do so because I'm sure the people at Supermicro are watching But to get this level of density means we have this kind of tight bit for this riser card There we go I mean it comes out But it's really in there good. It's kind of a dual riser card as you can see I have and this was not shipped with it. This is a separate 25 gig card I put in here for some of the testing we're about to do Well, I've been doing and now I'm going to share with you on this device And the other than just being a little bit nervous when I pull them out It's not that bad overall to get this in and out and the high density having the dual PCIe right here and another one over here in the back Does allow you to put quite a few cards in here and the airflow should be adequate For keeping this thing cool, even with the high density applications in there Now there is an option to get this with no networking at all Currently it does have on-board network card But I thought it was interesting they had that as an option So if you wanted no networking except for the IPMI network That is a way you can order the server if you wanted to be air gapped offline only for Processing whatever you want it on there But I do have the 10 gig card installed as shipped from Supermicro But I wanted to try a 25 gig card in here for some of the connectivity Now the good news is this goes in Way easier than it comes up Just give it a quick push goes right in flip these little levers down And it's locked in we're good to go no problem So other than a little bit of struggle getting it out because it's just a nice snug fit But I think that's what you're looking for no real worries overall about it Like so that's the only really complaint on there Now I do have the plenums off on this kind of to make it look a little bit more presentable and less shiny in the lights But the two plenums plastic plenums that redirect the air We'll go ahead and put these back on the system So we can get this thing fired up and jump into the real fun, which is the software testing on it Now before we put this in the rack. I wanted to show the drive base. These are a nice toolless design So easy has these little pins down at the bottom here So you can just set a drive right in and tension from the sides here Just holds drive in place making it really easy to swap out a drive There's all as I said earlier hot swap base easy enough to take them out put them back in and currently these Are loaded up on this side here with these kiauxia NVMe drives these perform really well. We're going to see that in a testing we're about to do now as far as locking the drives I will show the locking mechanism is pretty basic on these. It's just a tiny little Put a flat head in there and turn it Keeps it from popping open Not real secure but enough to keep people from just touching and popping a drive out without having to get in there And it's recessed just enough that this little screwdriver works But your fingernail may not get in there to do that Now taking a quick look at the back. You can see the two hot swap power supplies The little lever to side that slide them out Then you have the two 10 gig ports as provided by super micro with their onboard card that I believe there's another option one that you can get that has sfp ports here instead This is the dedicated motherboard ipmi that we got the usb 3 right over here And then we have the 25 gig card that I installed and because we aren't really going to be using anything for the vga I didn't bother plugging it in but I do like that we have vga and serial And a lot of you are going to be going well tom, isn't it 2021 and we shouldn't have those This is how a lot of enterprise systems are managed with Still the old style connectors and honestly when you have ipmi, you're rarely ever plugging into those type of connectors Anyways, they're kind of just there for now and a lot of the older systems that I said in the data center type Stuff still have if they have any type of connectors for a monitor are still more common to be vga But that's not really anything I would worry about. I haven't seen that many servers yet that have hdmi I think we're going to see this for a little bit longer for those of you wondering But ipmi is the way we are going to manage this and actually that's the next thing to do is go stick this in the rack Plug it in fire up the new ipmi interface because they did some cool revamping on that I kind of like the more modern look it has All right, so we're logged into the super micro interface here One of the things I want to point out is that yes, I'm reusing firefox and yes, it works fine There's a comment a few people had about whether or not it will work in firefox and it will accept You will get this message if you try to launch the console for the html 5 It works fine, but you'll get your current browser doesn't support the capability of video recording Please use chrome for that So you do need to use google chrome for the video recording feature But functionality wise virtual keyboard and everything else. I didn't find anything else that didn't work in firefox So for those of you wondering if it works in other browsers I works except for the video recording feature now We're going to use this in google chrome, but I also want to point out a weird problem I ran into and that is ublock origin I had to disable ublock origin for this site because when it first loads It just kind of spins and not all the elements will load I assume it doesn't like some of the scripting that it's using but turning it off solve the problem A little bit of head scratching at first opening up in firefox I realized it worked at first I thought it didn't support chrome Which led to probably some of the people thinking it didn't because so many people do use this plugin So hopefully that helps someone solve some of those problems Now this is a more modern look of the interface and I do like the new layout I like the you know kind of friendly I am currently power off that we have over here to see that the system's in a power off state I did a power cycle that had it on and turned it back off So you have the power consumption which as this page refreshes it'll go back down to zero We have all the component info cpu info a little green checkbox is to tell us what's good. What's bad All the system information serial numbers part numbers and they give you a great amount of detail in here Now with the system in a power state off, we're not going to have much of the health status Let's actually go ahead and hit power on Hit apply it does this little flashy Retrying a couple times thought that was a little odd I guess it just does that until it gets confirmation if we go back over to the dashboard We'll get a preview again And I can actually hear it don't know the the noise cancellation Probably filters out, but I hear it on the other side in the other room and now it's thinking about the power on cycle You get a couple more of these retries. I just want to do this. This is real time on this part It it does have this like pause when you first power it out But once it's on the interface goes back to working perfectly fine The request configuration has been completed successfully and now it's going through the boot process Back over to the component info everything's green And in the second these will turn green as well. So it's going to get updated readings now that it's powered on So I didn't have any errors in the health status But it does show them red if it's powered off but after it goes to the full boot cycle everything turned green again No problem Curl including once it boots up It'll have the temperature of not only the onboard network card But I thought it was interesting that the add-on 25 gig card Which granted? Yes, it is a super micro card the aoc s 25 g dash m2 s model And that does also give me a temperature reading once it boots up So we'll go ahead and let it boot a little further and get these readings refreshed But while we're waiting we can go through the configuration and account services notifications All the usual things that you have with the previous ipmi are all in this one So you have the ability to push things even to assist log server, which is I really nice the way to have all this set up It's an easy to use interface. I didn't really have any problems The only thing I had overlooked originally was actually over on the component info info Was the fan speed is under advanced here And I if you leave it collapsed you may not notice that you click the three dots to bring it out So you can change a couple different fan speeds Hey here, we can come back to seeing the network temperature on there We don't have any gpu so it doesn't display anything here but I like that it has all these sensor readings and Among the interesting features is when you go to the full sensor reading right here You can export these out to excel, which I think is just kind of novel being able to throw all your sensor readings quickly into excel Uh, pretty neat that it's got that feature Back on just to finish the remote configuration the virtual media So if you would like to completely manage this remotely, which is a very likely scenario So you don't have access to the machine and you may not have access to the necessarily put any USB that has different boot options on there So you can load different things to the system It does have the option to do a shared host and a path to the image So you can essentially load up a ISO server some type of storage server Maybe you have in your rack that could feed this and be able to completely reload it remotely So they have this on here or old school floppy images that can be uploaded to the system itself The last things we have on this part are the network control to set this however you want to manage your lights out management with the certificate ports access limitations Whether you want it to be DHCP or specify the IP address For maintenance, they do have license activation a reset firmware management that can all be done right here So if you want to update the firmware that can pretty easily be done for the BMC the BIOS Or the motherboard cpld. So this is nice that they have all this Right easy thing at your fingertips for remote management. Now, let's see if this thing is booted up We're going to go ahead and launch the Controller. Yes, it is. That's a nice feature is this html5 with the virtual keyboard Made this really easy to manage and the recording feature that does claim not to be there when you use firefox It doesn't work is available though, and you can record what's going on on a screen This is really good for being able to do tech support as soon as you stop it. It has a little Webm file that it creates and can play back for you if needed. So a really simple methodology for doing that I like this feature on there for being able to do the capture or even just do a screenshot And you have your power control set power on power off reset Options hotkeys fullscreen mode How a keyboard mouse hop plug it does have mouse support but the os I have on here is not Support mouse. So nothing I can do there We're going to close this though is let's get to the testing in xcp ng Now i'm running the latest version of xcp ng 8.2 fully patched as of October of 2021 I did try true nas scale on this device because when I was doing some testing and some previous videos I can link to with true nas scale for performance. I was a little disappointed in performance So I tried it on this system and it still didn't perform really well But xcp ng no problem And I figure a lot of people probably want to look at this from a virtualization standpoint And I do plenty of virtualization testing and that's what this server has been doing for the Last 30 days that it's been here of note the kioxia NVMe drives the five of them are set up in a zfs raid z Just raid z1 so not too much redone. See I wanted to see what kind of reforms we get out of there And we're running all these vms local to the system So we're going to run a series of tests and just going to show you the performance I was able to get with this basic load on this So here's seven virtual machines and we'll go ahead and start them all at once Are you sure you want to start seven vms? I feel confident this system can handle seven vms starting simultaneously We have windows server 2019 a windows lab a windows lab attached to the server 2019 domain Just another a bunch of server and a bunch of server here now the bunch of servers I have 64 processors assigned to each of them that way when we do some of the testing We're going to test both of those simultaneously and see how they perform and then over here on the windows server We've got 8x here and then we're not going to do much with this matter of fact Let's have a little fun. So here's windows. That's probably it's already booted We're going to head and shut this windows down and we're going to clone it So we have a couple windows servers running here and each of these will have 32 cores as well So go ahead and stop and clone We'll make three more of them. We'll go ahead and fire up all three of these windows servers here There we go. Now. We should see a little more usage. We look at the host We're using about half of the 256 gigs of memory in this right here But I'm sure it's going to have no problem running all these windows servers or windows 10 labs at the same time and we'll just call them all windows 10 base clone Actually, you probably give them a little better names here clone one So we have all those up and running now. So there's our windows clones and our two one one two lab servers Let's kick these off and I'm just going to run some random pharaonics tests on them But we're going to ssh into each of these so I'll get the IP addresses We're on the pharaonics fio And kick off some io tests here. We don't need to save these results Let's go ahead and get the ip address of the other lab server here So we can run at the same time All right, so both of these are doing a bunch of heavy io operations right now with that Let's see how the disks are holding up. We'll let this run in the background. We'll look at some stats on here They're just getting started right now at about 27 000 iops on the mvme But after they start running, we'll see a little more But why not stop there in why stop there? I should say and let's go ahead and go to the council on here and Let's also go run some tests. We should run at least nine tests all And that's on the windows lab base. Let's go ahead on clone one and run it over here And uh, let's keep going clone two. Let's just see how many of these we can Get the system Only loaded on i o how much i o can it handle and still give us some type of performance numbers out of there So this one's running and that's that one that finally let's go back to three Someone will probably point out. There's a lot of easier ways to do this I know but i'm having fun clicking through all these windows turning this on All right now. Let's go ahead and see how that mvme is holding up Go to our storage All right, we've hit We're at the 175 thousand uh read i ops here It'll once we get to the right side of the test. We're gonna see a little bit Different side. I don't know if it's going to write quite as fast, but that's not bad That kind of iops performance out of these so not bad at all here And is it usable? I guess it's going to be the next question So right here's my kelly linux install and can we do stuff on kelly? kelly seems to load perfectly fine. So let's go ahead and run an update on there. Why not? It's very responsive. I can probably open up Uh different applications Set up a few things get stuff going on here. It's very responsive even while at the same time We go over here to the disc. They're all running on this same local mvme And oh now we're in a 300 thousand in the read iops on here a couple notes though One of the things you're going to get some advantage on is having Zfs and its ability to Read and find common caching and be able to Provide even better results when you're asking for some of the same things And some people say caching may skew the benchmarks But there are a lot of times when caching will actually help your overall performance Well should be all the time it helps whenever there's repetitive file operations So it may skew the performance in some ways But in other ways it is actually providing a pretty real world knowing that it's turned on You know understanding of what's going on of note and for those you that aren't familiar with xcp and g when you set up the Zfs on there you do want to dedicate a little bit of extra memory to the control domain and I've got that set at 32 gigs This allows the zfs caching to Be a little bit more effective now. I do have net data installed on here. So let's actually see what we're doing for zfs And if I go here to zfs file system Here's our arc size and about 16 gigs of arc arc on this the adaptive read cache is caching on there But we're not seeing or seeing some good hits on here So here's where we have the misses and now we're seeing a lot of hits So it's kind of given us some really high numbers for The iops now when a few minutes will speed ahead and go to where we have the read side of these tests going Because when the read side test obviously Versus the right side test you're going to see some different results So once we speed past the read test and we see the right test, let's see how fast it can do there Probably not bad even on the right where I hit in 282,000 Uh iops here. This was our read at 6.2 And our 4.99 wow or 4. Yeah 4.89 for the right throughput So that's that's some pretty impressive numbers we've seen on iow here And we did see a little bit of latency but not anything terrible because as I said, you know We can go back over here like my cali linux, which it looks like that's some updates. We'll go ahead and say yes Everything's still very responsive working perfectly fine. I'm not really seeing any issues with The systems all of them are running fine on here I could probably go through and even do a few things in the windows server lab as well And uh, you can see they'll see Right. Yeah, we logged into this it's completely responsive and working fine while all this other stuff's going on in the background We'll jump back over to the disc And they're running some of the smaller writes which don't need is quite as many doesn't really push it to a higher level of Iops on here Now the last thing we want to test and this is going to be a little bit trickier to test But I'm sure we can pull it off here is let's go ahead and run some stress testing And what we're going to do is log in here And we'll kick off stress And what this is going to do is tell it to consume 64 processors load them up a gibberish for the next 300 seconds So I'll kick that off which will send the server into some angry Pretty sure that I can hear it. I know the staff in the other room are probably wondering what all that noise is We'll run a performance test simultaneously with this virtual machine here And by the way, the other benchmarks the pharaonics ones Are still running in the background here. So we're still doing disk iot tests. We'll continue and we're going to go ahead and run All the cpu tests. So it's now running cpu tests on this particular system here Let's go ahead and do it on clone 2 Do the same thing again and you're seeing the performance numbers are so reasonable. This thing is still very responsive. It's still working Kick off passmark again Continue Run Yes, yes So we're running a couple dual cpu tests while also running stress. We'll go back over here to the host And it's just a wall of processor running right now and it's kicked up the fans I can hear in in the other room bleeding through a little bit and uh, but the system is running perfectly fine It's still handling it and we'll see what the processor results come back to so all that's running and simultaneously How the super micro handles The storage, so let's go over here at the storage stats Yep, still no problems. We're still relatively low IO wait time For the particular runs it's running. Of course now only have the linux VMs really doing the IO I just have cpu tests running out a couple of those windows machines And of course, we still have my cali linux. It's probably still loading a few updates. Yep See if there's any more But overall as you can see this is some very responsive and that was kind of the goal is to Make sure that it doesn't have any problems handling all this It was kind of no doubt that it would but I thought this was kind of a fun way to demonstrate some of that in the video here So this one's completed. Here's my cpu versus the world On clone one. I think we ran it again on clone two That's still almost done almost in here kicked off a few seconds later This one's even better because it finished after that was done And let's go back and see if this is still running over here We'll go ahead and stop stress from running So the system will go back down to normal But even with that running sucking up 64 cores had no problem multitasking all of them And I know some of you that play around with this stuff all the time or use a lot of enterprise hardware Probably aren't surprised at all about the performance of the machine But I wanted to demonstrate, you know, running xcp and g my hypervisor of choice Which by the way, I'll leave a link to all my xcp and g tutorials on here Everything I was running here at the zen orchestra the xcp g is fully open source So you can run all these same tests on whatever hardware you may have But overall, I want to thank super micro for sending us out for review We used it for a handful of little projects we had around here We wanted to do a series of testing and it's just impressive how fast we were able to get some of these things set up on there From loading things some of the operating system testing we did so overall, I do like the server I'll leave links to the super micro website where you can check it out. I have no affiliate codes This was not sponsored by them. This was just a review unit that I do have to send back. All right, and thanks And thank you for making it to the end of this video If you enjoyed this content, please give it a thumbs up If you'd like to see more content from this channel, hit the subscribe button and the bell icon To hire a server project head over to laurancesystems.com and click on the hires button right at the top To help this channel out in other ways, there's a join button here for youtube and a patreon page where your support is greatly appreciated For deals discounts and offers check out our affiliate links in the descriptions of all of our videos Including a link to our shirt store where we have a wide variety of shirts and new designs come out well randomly So check back frequently And finally our forums forums.laurancesystems.com is where you can have a more in-depth discussion about this video and other tech topics covered on this channel Thank you again, and we look forward to hearing from you in the meantime check out some of our other videos So i'm here from laurancesystems, and this is a super micro But let's jump right into the specific haythens