 So we're here at the NAR Connect and who are you? I'm Ed Vilmetti, I'm here from Packet, we do bare metal hosting and I'm here for the works on ARM project. So Packet does servers in Stachegad? Yeah, so we have servers in 15 data centers, four of those data centers have ARM based servers in them, we use Cavium Thunder X right now and the project that I'm working on also has access to server gear from Plowcom and other vendors for tests and continuous integration, continuous development, just getting people access to gear that they wouldn't have typically sitting on their desktop. Sounds really awesome, so in theory I could maybe sign up and run and host ARM devices on a real ARM server? Yeah, so we have a bunch of servers that are available for commercial use. The Thunder X goes for 50 cents an hour and you can rent it by the hour or by the month. The project also has a pool of servers for subsidized use, so some of the reasons that I'm here is to talk to people in various projects who are doing interesting things that maybe have access to some hardware but not enough or they're doing testing on ARM and need access to machines that they wouldn't otherwise have access to. So 50 cents an hour, that's for the whole server? Yeah, that's for the whole server, so one of the differences between packet and other hosting organizations is that we do bare metal hosting, which means you get the whole server, you're not going through a hypervisor, you're not being virtualized. If you want to bring your own hypervisor, that's fine. Or bring your own virtualization tools, but that's up to you. And what we find is that it gives us access to people who are either doing workloads that are sufficiently big that they know that they need the whole machine, or people doing really interesting projects where they need very low level access to hardware. And we don't offer a lot of services above the hardware, we're not opinionated about what people do, we just want to provide people with access to the fundamental infrastructure that they need to get things done. And it includes unlimited bandwidth or... Yeah, so the bandwidth works, you get charged for egress bandwidth, and you get charged for bandwidth between data centers but not within the same data center. So generally the bandwidth charge, I don't remember the exact price, but it turns out to be a modest... Unless you're doing large sort of things, it turns out to be a modest portion of what people actually pay. How does it compare with the price of a Xeon server? So the ARM hardware is interesting, we offer five kinds of Intel machines in one ARM system. So people are using Xeon systems. The Type 2 server, which is the most comparable server, is about $1.50 an hour, so the ARM stuff comes in lower, so it's aggressively priced. People doing systems development on a server that has 96 cores have to think in a different way than people doing stuff on a server with fewer, faster cores, and we're really encouraged by the next generation of server hardware that we're starting to see in the lab where the cores have gotten appreciably faster. So that's the Thunder X1 or Thunder X2? The Thunder X2 has faster cores. We don't have any good benchmark data yet because it's all really new. I mean we have very early production stuff, but definitely the server trajectory is going in the right direction. So the Thunder X2 is going to be available? That's our plan, we don't know when the quantity production is going to be yet. We're still talking with our vendors to make sure that we can understand how to deploy it best, but yeah, that is the plan. And then there might be some solutions from Qualcomm, maybe something from HiSilicon? Yeah, so the Qualcomm gear that we have right now is their early bring-up board. We have some of those, not a whole lot. It's not public, what works, right? So quite a bit of that is not very public. What's public? We actually had at the show in Los Angeles Open Source Summit, we had the exact motherboard that we're running, and they're planning an announcement sometime soon. So hopefully, I don't want to get any secrets out, but it might be quite good. But it's going to be a very interesting machine, it's a very competitive product, and it's just a very interesting, you know, as we see this year's group of servers, it's clear that everyone has made progress on Silicon. It's clear that everyone has made progress on operating systems, but they're easier to boot than they were the first time, easier to bring up. The kernel support has come a long way, just generally making it possible to ingest these systems into our environment and bring them initially to a select group of people, but with the full thought that we're going to be able to expand that out. And have you looked at the socialnext solution or... I was just over at their booth. That's maybe possible. Yeah, so that's one of the things that we've had conversations with. I'm not going to promise anything because I think one of the really interesting opportunities is to have an edge compute device that's arm based that would be in data centers closer to where the workload is. So if you're doing something like self-driving cars or whatever, that your data center for processing that data would be in the same metro as the vehicles that are there. And Huawei, HighSilicon has been developing five generations as far as I know of arm servers and they have the six, kind of like secret one coming out. Have you any access to any of the HighSilicon stuff? We do have access to HighSilicon. It's very limited at this point to people who are doing fundamental work, again the hypervisor and it's not generally available yet, but we've had some really good results with that. So how long has this packet been doing this business? The packet was founded in 2014 and had its first customers in 2015. So still fairly new. Like a startup? It is a startup. The founders had experience at other hosting firms. So they've been in the business 15 plus years. Could be a VH or a host gather or something. Yeah, so it's an interesting group of people. I'm really happy working with this group of coworkers. The team is scattered around. There's a core group in New York City and then there's remote people doing system management and customer support in enough time zones that we have 24-hour coverage. And some people locally in each of the data centers? We have remote hands when we need them, but we don't typically maintain offices everywhere that we have stuff. Our main office is in New York City near World Trade and the data centers in New Jersey. And the remote hands are ready quickly when you need them? Yeah. All right. So there's so many people in the world and everybody needs more and more stuff, like more and more smart apps and websites and stuff, and maybe it's not going to be possible to run everything only on Intel servers. There might be a huge future with this ARM server stuff, right? Yeah. So I think the future is very positive. The challenge is always finding that match of workload to price, performance, and power consumption. ARM servers by their nature have some advantage on the price and power consumption front. Performance is getting better. I won't say that it's better per core, but it's comparable per socket. And if your task lends itself to lots of threads relatively IO bound rather than a few threads relatively CPU bound, then it's very plausible that an existing workload could move over from Intel to ARM without doing a lot of work and without too much risk. So this might be the year things are happening in a big way. Maybe do you see any potential huge wave of adoption into this kind of solution? So part of the challenge has been availability of gear. There's no shortage of people who are interested. The hard part is that some of this equipment is still hard to buy. And just getting the supply chain ramped up so that if someone put in an order for 2,000 nodes that we could fulfill that even is, I guess, a challenge. It's maturing. I think when I got started with this stuff almost a year ago, there was a lot of uncertainty about how well the software worked. And a year's worth of software development has cured most of the obvious defects and really raised the bar so that people can assume fairly easily that an ARM port is an easy port rather than an unknown. And that's been a big difference. Do you have many customers already or is it secret? How many people are trying this out or using it actually, the ARM? So we have, what's the right way of answering that? You don't have to say if it's a secret, it might be a trade secret. Well, yeah, the overall packet platform sees about 50,000 deploys a month, so people deploy their servers, tear them down when they're done or deploy it and keep them. The ARM servers are a fraction of our total workload. They tend right now to be more focused on the development task integration front rather than on the production workloads. But for the people for whom those production workloads work out, it's a big win. It's a big win. I can imagine a huge company like Google and Amazon and all these guys on Facebook, they've been probably testing it out for a while secretly. Not so secretly. Testing everything out, trying, trying, trying. And then at one point, suddenly, maybe they decide to build a whole data center, just ARM. And when that happens, suddenly, you'll be very, very busy with everybody who wants to switch to ARM, maybe, suddenly. It could happen any day. It could happen any day. Yeah, the tipping point is when you have complete parity on the software front and an advantage on the hardware front. The Intel systems, in general, have more mature software just because more people have been using them. So part of the challenge is getting people's level of confidence up. On the hardware front, the Moore's law has changed. You can't just count on the next revision having a process improvement drive everything forward. So what we're seeing, in general, is instead of individual cores getting faster, more and more and more cores in the system. And so I think it's, is it going to be this year, is it going to be next year? There's a lot of work still to be done, but there's a lot of projects that are at parity where if you are running Intel and ARM, you don't know the difference. And then it's just, again, it's price performance and power usage. When you were mentioning before about the single threaded performance might not be as high as Intel, but is there any chance with Thunder X2 and with the new Qualcomm stuff and maybe with the next upcoming Huawei, High City can, maybe they're getting there. Even in the single threaded stuff. Yes, single threaded stuff has definitely seen improvements. Again, I don't have any, numbers are a little bit hard to say because the total system performance, the per core performance is definitely getting better. Now if I had only one core, I would probably be not as happy, but these systems are all many core systems. They all start at 48 or 55 cores or what not, 56 cores. The Thunder X2 has four threads per core, so something like HTOP reports 224 threads. Now how you write software that uses all those effectively is probably the next challenge. Not just for ARM, but for Intel as well, or even for AMD where you have to worry about schedulers, non-uniform memory access, caches get more and more important, trying to make sure that your process doesn't wander between cores and lose all of its cache consistency. So there's a lot of work that you solve one problem and you discover fairly quickly that gives you another opportunity to solve more problems. Has Intel Sion had in the last few years lots of cores or what's Intel Sion kind of core amounts usually? So they've been growing, so no Intel chip that I've seen, no Intel chip that we have has as many cores as the ARM systems have. Usually it's like 16 or 8? 16 or 24. There are some 24 ones? There are 24 ones, yeah. At that number of cores, again, the other things start to become really important. How much L3 cache there is on the chip. How well it does memory access. And I think there's still work to be done at the application layer to really take advantage of these really high core counts. One of the maybe main differentiators for the ARM solutions is that they can customize all kinds of stuff on the SOC. They can do the networking, they can do power management, maybe all kinds of things on the SOC that maybe Intel has kind of woken up and tried to do as well recently. Yeah, so the ARM world is a bunch of fantastic beasts, right? You have systems that have a really high level of integration. You have systems designed for data centers that have some features and don't have some other features. A lot of the data center focused machines are 64 bit only, whereas the chips that were traditionally coming out of the phone world often had 32 bit modes in them. There's GPU integration, just all sorts of stuff. And as hardware gets more diverse, it gets more possible that you're working on systems where the hardware that you have gives you an unfair advantage over a software or any solution. And so I think the ARM world is uniquely poised to take advantage of that hardware diversity. And so how long have you been following or hanging around with the narrow guys and what do you think they're solving some of the many of the issues or many of the challenges and making all this happen? Yeah, so I've been doing internet stuff for about 30 years now. I found the Raspberry Pi, like a lot of people found the Raspberry Pi a couple years back, used it as an industrial controller for a solar energy plant. Got involved with the Cavium stuff about a year ago and immediately found the NARO in that world. And just following, you know, I got involved in ARM a little bit late compared to people who can say, oh yeah, I've been working on these problems for five years. I've got a year into it now, specifically at the server side. I think the challenge that everyone has is for these servers to be completely reliable and completely boring, like you don't even know what you're running on and everything just works, every single problem that gets found and solved has to be upstream. And if you can get to the point where you've upstreamed all of the things, then packages and programs and whatnot just work and at some point it gets very boring, I mean, to be honest. And boring is good. Boring is so good. You do not want excitement in your day to day. So you do some tweeting and you call it works on ARM? Yeah. What are you doing on there and what are you looking for? So works on ARM is a project that is funded by ARM, run by packet. I'm the lead on it. We've got about five people with some level of involvement on it, plus people from various chip makers and from ARM itself and from NARO and a good-sized crew of folks. The strategy is to put together a website that has a comprehensive directory of all the software that works in the ARM platform. Complete with current information about versions and distributions. Complete with honest discussion of bugs that are blockers, potentially for things. Here's two distributions and two containers that run these sorts of things. I put together a weekly newsletter that has a distribution that goes out to a bunch of folks. I tweet because I know how to tweet. And we regularly meet in a group that talks about various issues that have come up trying to be strategic about using the resources that we have. Works on ARM has a few dozen machines that we currently have and we're getting more for people doing CI and for bring up and testing and getting people access to hardware who wouldn't otherwise have the hardware on their desk. It's been good. It's been, you know, it started out as like, hey, we got these servers, what works, right? And has transformed fairly quickly into, hey, we have all these servers and we know that a bunch of things work and not only can we tell you that with some authority, but we can give you code that you can run yourself and prove it to yourself that it's all working. So the challenge I see for the coming year is more and more demos that are already ready to go. More and more CI CD loops so that people can prove out on a development level that everything is working and just general community awareness. So as an example, could you, like, one interesting example that might have been recent of some stuff that works on ARM? Sure. So one of the big computer languages that's relatively new that gets used for a lot of new cloud native workloads is Go or Golang. And when I got to the project, Go was at version 1.6. It had one very small bug that I, anything has bugs, right, but it had one particular bug that caused Docker not to work in certain heavy duty workload characteristics. So over the span of about six months, we found the bug, upstreamed the patch, got Docker to include it in their next release, set up Go, continuous integration on packet so that Go is building their latest version of the language, testing every check-in, making sure everything, every check-in works. And getting it to the point where at Go 1.9, ARM is a supported platform for Go and they deliver binaries for it. So it's one of these things like, you know, is that a great demo? No, it doesn't look super great on the screen, but it's fundamental enabling infrastructure that lets dozens or hundreds of other packages have confidence to what they're doing is to write that. So it's also kind of like a little community where people send in test results or suggestions or questions? Yeah. We have a Slack channel, like everyone has a Slack channel. We have regular communication with people on other projects. There's, I get email from various people. The Twitter following is people noticing things. So, you know, it's generally a useful way of collecting a community together. But it's relatively new, but potentially it's going to get very busy very soon, right? Hopefully. I mean, I've been doing videos about the ARM servers for five, six years now, but this might be it. This might be happening right now. So like I said, if it does happen, then it's going to be huge. Yeah. So as people build out, so the ARM in the IoT space is really well established and really well understood. Single-board computers likewise, I mean, there's some that are very, very popular and some that have a lot of variety to them and there's a lot of ferment there. The ARM desktop, Associate Next has their new product that will be very interesting. I'm hopeful that they won't be the only one selling things in that form factor because it's a really great, great sort of system design. Some laptops are sort of scarce, but it's possible to do that if you're creative. That's what I use. And then ARM in the server room is just the next thing to do. And I'm not going to make any predictions of whether it's this year or next year or five years from now. I don't think it's five years from now. I think it's coming soon. But again, it's a matter of commercial viability. It's a matter of not just individuals standing behind things, but companies standing behind things and the hardware getting to a point where it's price and performance competitive with everything else. Because when you say you compare it with the system number two, type two for the Intel, which you say is the equivalent more or less. More or less. Yeah. And it's three times cheaper. Is it because it consumes three times less power or is it because you're being nice? Are you being subsidized by ARM or something? So are we being nice whenever you introduce something new into a market? Normally it should be high price, no? Because it's like an experimental thing that's scarce. It's not easy to get access to the hardware. So maybe in theory it should actually have been more expensive, but... There is a theory of online pricing everything. You should always be charging more, right? Whatever you're doing, you're probably not charging enough for it. It was... Well, so Packet is interesting because like ARM, Packet has SoftBank as an investor. And so it was a strategic move for us to move out of the purely Intel world into the ARM world. Zach and Jacob, the two of the founders, and Aaron, who's going to be here tomorrow giving a keynote, really have this vision not just of what are we doing this year, but what does the internet look like in 10 years? And Packet's motto is build a better internet. We really think that we're on to something, that the access to bare metal, unopenguinated at the data center and at the edge of the network is really important, and that a variety of hardware to serve a variety of needs is a really good strategy. The hope is that by doing things in this market, we spur on more and more people to say, hey, if we have a system on a chip that can scale up the data center size, we have some idea of what it would take to get things into people's hands and build systems that could be done at scale. But for the equivalent Intel, is it really something like three times cheaper to run? Because you are selling at three times cheaper price? Yeah, so it's not an equivalent machine. No two machines are going to be equivalent. The Cavian box has a lot more cores. Has lots of RAM? Has lots of RAM. So enough storage. We have a block storage solution. So if people need lots of storage, we have ways of doing that. Why is it a third the price? Well, you put your finger in the air and you say, what's the right price for this going into a market? If it's the same price, people will look at it and expect it to be the same thing. We think it's competitively priced. But does it use less power? Does it use less power per core, certainly, but it has more cores. So it doesn't use enough less power to make it fanless, which is a goal. But in the data center you have resources. The Thunder X2 actually, when we first got it, ran a bit hot, and I think we've corralled some of that. The power thing is vendors can make a series of choices. The better the process that you have, generally the more power advantages you have. If you run at a lower clock speed, you consume less power. So if you do a lower clock speed with lots of cores, the arm design is just more efficient. But still you're trying to pack a lot of computing power into the box. So it's a six to one half a dozen, another issue. I guess it's going to be very interesting near future, what's going to happen soon, there's going to be maybe an arm tech con with some kind of announcement, I'm guessing, or maybe not, because they say they want to be 25% of the server market by 2020, and maybe they still mean that. It's getting close. I think the position that I'm in is a pretty good one. If they really are 25% of the server market, that would be awesome, but they're not right now. To get to that scale, you have to learn a lot of things really fast. But I think that by providing this software directory and this community that we can get people there, it's an ambitious target. I wish them well. I wish them all the good things. And I wish you well too. Thank you. It's going to be interesting to follow and see what comes out of it in the near future. Cool.