 Welcome to another edition of RCE. I'm your host Brock Palin. Again, I have with me Jeff Squires from Cisco Systems and one of the primaries on the OpenMPI project. Jeff, thanks for helping me out. Hey, not a problem, Brock. How's it going? All right. Jeff, I hear you have a very popular blog out there now. It's almost like you were prompted to say that, you know? Yes, I have a blog out there on Cisco.com. It's talking about MPI and general HPC things. And just like you have a Twitter account. Yes, I have a Twitter account. And actually, we're trying something new. Before recording a couple days ahead of time, I will tweet who our guest is going to be. And if you could send replies to my Twitter account at BrockPalin, B-R-O-C-K-P-A-L-E-N, we will integrate those questions into when we meet with our next guest. We'll put this information on the show notes or something on the page. So you don't have to memorize exactly what we're saying here. OK, so for today, our guests are we have the largest crew ever. There's a total of five of us talking, including you and I, Jeff. We're talking about the student cluster competition, which is a competition that goes down every year at SC. I've heard about it. I've been interested in maybe getting something going at my own institution. I haven't really looked into it before. So this will be informative for me. Yeah, this is really cool stuff. I see these guys every year. I see the students over at SC. And some of them are pulling their hair out, trying to get the stuff done. And it's always just a great competition. And I'd love to hear more about it. So this is going to be a fun one today. Yeah, it's always interesting seeing the students sleeping there basically on top of their cluster in case something happens. Underneath the cluster, that's right. OK, well, I'll go ahead and introduce our three guests. We have Heia Nam at Oak Ridge National Laboratory and Tiki Suarez-Brown at Florida A&M University. This year, they're SC 2010's co-chairs for this year. So with the cluster competition, the buck stops with those two. And then we also have Doug Smith from the University of Colorado, who has actually advised the team before. And he can tell us a little bit about their experience. So guys, why don't you go ahead and take an opportunity to say your name, introduce yourself, and say a little bit how you got involved with the cluster competition. Sounds good. OK, so my name is Heia Nam. And I work as a computational nuclear physicist at Oak Ridge National Lab in the Scientific Computing Group at the National Center for Computational Sciences at the Oak Ridge Leadership Computing Facility. And if I don't get all of that in there, then someone will come after me. I've been working there for a little over a year. And coming from San Diego State University, I got connected into the cluster challenge by accident. I'm actually, I'm a physicist. So I'm the ultimate end user of HPC. I got pulled in by Brent Gorda, who was the founder of the cluster challenge in 07. And he brought me in when I was a student to help with telling kids how to use an application. So it just kind of snowballed from there. I've been associated with it every year, helping out a little bit here and there. And last year, so my boss is Ricky Kendall, who ran it last year. And he asked me to come in on it this year, because I know all the details that go into it. And he knows that he can torture me as much as he likes since he's my boss. Excellent. OK, good afternoon. I'm Tiki Suarez-Brown. I'm an associate professor in the Information Systems and Operations Management Department at Florida A&M University. And I'm kind of interested in management information systems, collaborative environments, as well as high performance computing. And I've gotten involved in the Supercomputing Conference back in 2001. And kind of been part of the SE Communities Program since 2007, had the opportunity to work with Brent when I was the student volunteer co-chair, and then also had the opportunity to work with, hey, a little bit, as well as Janine, who was last year's community's chair. Excuse me, a student cluster challenge chair when I was the broader engagement chair, and always enjoyed the camaraderie, as well as the competition and rivalry more so between the teams as they're out on the exhibits floor or as last year out in one of the main exhibit hallways. So really interested in trying to get the word out to let others know about the opportunities that the student cluster challenge can provide. And hopefully this podcast will let others know how they can get involved. And we look forward to receiving applications. Oh, yeah, my name is Doug Smith. I actually work for the Laboratory for Atmospheric and Space Physics at the University of Colorado. I am their HPC admin. And we learned about this right after SCO 6 in Tampa and have submitted a proposal as a team from CEU every year to compete and have loved doing so, watching my students pull her hair out at 2 o'clock in the morning. OK, so thanks for the intro. So said they were in the hallway and they're there at 3 AM. And basically any time I was walking around the SC floor, there were students around their clusters. They have actual machines there. What exactly is this competition? Is it software? Is it hardware? So this is Hayya. And I'll go ahead and just give you a briefing of what the challenge is about. Basically the students at the team of six undergraduate or high school students or a combination of the two, they work with the vendor partner to come up with an architecture suitable for the applications that we present to them for the competition. This year there'll be four applications. We haven't decided which high performance applications yet, but they'll be announced shortly. Based on that and wanting to do very well on their HPCC benchmark runs, they come up with an architecture. They hopefully will have time to work on it before the actual conference. They'll bring their clusters to the conference. And when they're there, they'll run the HPCC benchmarks. They'll run application competition data sets that they've never seen before. They'll be given sample data sets prior to the competition. And the team with the highest performance on the benchmarks, along with the highest throughput of application runs, along with impressing interview judges, the team with the highest points is the one that wins. And so basically they're scrambling for the entire duration of the conference between battling with their clusters and also showing people that they have learned what they've learned throughout the process. It's very intense. So I didn't know about the interview. That's interesting. So you're actually seeing how much these students actually have learned and figured out. And in the competition, it's not just the highest LIN-PAC number. It's also how many given sets to push through of a given application. Yes, exactly. So there is, I mean, we do recognize at the top 500 announcement, the general top 500 announcement, the team with the highest LIN-PAC, but that's not necessarily the team that wins the overall cluster competition. I would point out that there's sort of a developing curse in that the team that is posted the highest LIN-PAC has never won the competition. I wonder if that's some kind of a relativistic statement on the state of HPC in itself. LIN-PAC's great, but doesn't necessarily reflect real-world HPC. Exactly. And that's what we want. We want teams to come in knowing that they have to run a real HPC workload, not just get the highest marks on LIN-PAC. So do students have the opportunity to kind of choose, say I'm presented as a competition participant with a given application that can run parallel, but maybe doesn't scale to my entire machine. Can I choose to run one job at a time across the whole thing or maybe run two, but maybe give a little better parallel efficiency on two smaller jobs? Exactly. We want the teams to have played enough with the application so that when they come in, they know how many cores are going to be throwing at a particular application. They have to be able to figure out what's the best way to plan their workload so that they can get the most through. Basically. That's interesting. Everyone's always interested in going absolutely as fast as possible, but this is the time-to-solution problem. You have X amount of stuff that needs to be done, not get one of them done as quickly as possible. Exactly. And since we are sufficiently evil, we give them way more than they could possibly do. And we don't let them know. I mean, they should have known by playing with the data sets for the samples that some, they shouldn't even try. Hopefully, they have some idea of what a data set, how long it's going to take. That's most excellent. I have to say, I was muted when I was chuckling when you said you're sufficiently evil, but at the same time, that's also true, right? I mean, in a real-world HPC cluster, users are never going to have a satiable demand. There's always going to be more stuff that you can pump through a cluster. And it's a very complex decision about exactly what can and should an administrator or set of users allow to run on their hardware. Yes. Yeah, and also, I mean, I'm a sysadmin and understanding the whole idea of, you can't just throw more cores at it. That doesn't necessarily solve your problem. It doesn't necessarily help you. And it may not be an efficient use when we're throwing however many hundreds of thousand dollars in your electric bills at this equipment that's appreciating that thousands of dollars per year too. So it's a good understanding to understand all of that, not just get the highest impact number. It's a great educational experience for the students because they really get psyched and excited about it. And these real-world applications, such that you're not just throwing those cores at them, they have to really think about how they're going to plan together as a team to try to get each of these applications accomplished. So what are the constraints for the students? I mean, can a school just show up with the largest set of equipment out there and set the most amount of kit on the floor is going to have an advantage? Or do they have only like a single 20 amp service or it must fit inside a 42U rack? Is there any type of constraint like that? There are definite constraints on the, we want to make sure it's an even playing field. One of the biggest constraints is that we only give them 26 amps and it's off of two 20 amp circuits. So they have 13 on each that they're allowed to use. So they have to figure out how to balance their cluster, basically. It's not necessarily that we're evil and we want to only give them 26 amps, but we work within the constraints of what a convention center provides. So it's impressive though that they can do it. And part of our flyer that's coming out will show that what is 26 amps? Well, it's about as much power to have three coffee makers going. So they're running an HPC workload off of what three coffee makers require. So that's pretty impressive. Yeah, and giving them as much work as they could possibly consume, they know they're going to be there all night and it's going to be right up till the very end of the time limit. I hope you let them run a supplemental coffee maker to supply the caffeine. We do. Well, actually there was one team last year, ASU who came with an extremely nice, I think espresso maker. They were better decked out than the committee members. So I think that one alone probably was taking the majority of their power. Other constraints, it's a single cluster. So I think the height limitation was 19, but I don't know if people have been very stringent about the height. I think the power is what's the primary limiting component to the challenge. And then they have to fit within all six team members plus their cluster, plus their visualization. We give them a large LCD to show their output. All have to fit within a 10 by 10 booth. Okay, and what other, so that power envelope there, does that include the LCD? Does it include any monitors? It includes the network or is it just the servers or is that everything, everything, everything? No, so it's for the cluster itself. The LCD monitor, we give them one extra one for their laptops and their coffee maker and for the LCD monitor. Well, I wanna go back and touch on the evilness bit again, how do you design the problem sets? How do you come up with sufficiently evil yet sufficiently realistic stuff to have them do? So I'm an application scientist. I create nuclear physics codes and for all the applications that we choose, we find a domain specialist who has worked closely with the codes as well as the data sets. And we ask them to provide something easy, something hard, give them, let them know that the competition for running the applications is on the order of 48 hours. So what would take longer, give us enough data sets to pass 48 hours basically. Okay, what kind of things have you run in the past? What kind of applications and whatnot? The first year, we ran a quantum chemistry model called GAMES. We ran an embarrassingly parallel model called Povray or POVray, I don't know how it's pronounced. Yes, Povray. Last year we ran WARF, another model and the past two years we've run NWCAM. We've also run, I think some biosciences models, Rax ML, I believe is one, Open Foam, which was a finite element, yeah. Those are the ones I can pick up off the top of my head. So those are really spanning the gamut of quite a few disciplines there. So there's a lot of involvement here from the entire community is what I'm hearing. Yes, so what we highly recommend of the student teams is that they enlist domain specialists. And you'd be surprised if you send an email to someone and say, I wanna know more about your application, how willing they are to explain it. And so we've seen teams like Purdue consistently, they'll, you know, throughout their, you know, throughout Purdue, they'll go around finding people who understand these applications and bring them in and get them to help the students or the students themselves will be a domain specialist and then they have to learn the cluster portion of it. So it's a really nice, you know, education and HPC across the board. And I just have to say, I think it's, there was a student from Jugg School, so Dustin Lieberman, who said he didn't really know that much about HPC, he was a contestant during the first year. And now he's working for Oak Ridge National Lab. So do you constrain any software? So you have the target application, but do the students have to pick which compiler they're gonna use, which MPI library they're gonna use, any supporting libraries they're gonna use? Like, say the application you're using can be built with, say, two different IO libraries or two different FFT libraries. Do you dictate which one they're supposed to use or do they basically have to figure out the better one to use on their system? We let them figure it out. We don't constrain any of the software that they're supposed to come up with. Just a target application, you must solve an NWCAM model. Exactly. So if they, I mean, if they can get the application running, then they'll know what software they need to come up, come with, basically. And we give them sufficient time, well, we will be giving them sufficient time this year to really get to know the applications. We should be announcing them within the month, probably before the end of February, so that they have sufficient time to understand them and figure out what kind of hardware and software configuration they need to come with to be competitive. Okay, so where do they get this hardware support from vendors? Do you set them up, you being the facilitator, set them up with vendors, or do they, as a team, have to go out and kind of recruit vendors to provide them with equipment? In the past, it's kind of worked both ways, depending on if there's already a relationship that a university or a team from a university has with their vending company or vendor from their university, then they might pair up that way. We do try to assist with partnering or trying to get additional donations or donors to sponsor a particular team so that we can get as many teams involved if they have not already provided or found a relationship with a vendor, an existing vendor. We've used vendors that we've had already on campus in the past, but this year, I think we're gonna expand the field and try to examine as many vendors as possible and make a selection. Well, as I said, we have a lot of vendors who are very interested to be paired up with teams. They don't know, you know, they need help to get to a team and the teams sometimes need help to get to a vendor, so hopefully that won't be, you know, it won't dissuade a team or a vendor to become interested in the competition. Okay, so you mentioned that high school teams are allowed. What's, how many high school teams do you have in a given year? Well, we haven't had a full high school team, I believe one year, I think it was an O8, that there was a high school student associated with the team, but actually this year, we are trying to foster a team of purely high school students. Yeah, in O7, Alberta, University of Alberta, I think it was, had a high school student on their team, and I believe he returned last year with their team as well. I think he was 16 when he started. Wow. I think Alberta that year had the youngest team member and the oldest team member on their team, didn't they? Yes. Going for all kinds of records that year. Yeah. So all right, well, let's talk about the past. Who's won in the past and what have their achievements been in the past? So this competition's only been going on since O7 and Reno. That year, the University of Alberta won the overall competition. I don't recall who the Limpac winner was actually. I know that we were kind of, do you remember who it was Doug? Yeah, it was Taiwan. Taiwan. Yeah, I'm gonna probably pronounce it wrong, but it was like the National Tsinghua University. You got it perfectly, right. But they also won the Limpac in 08 as well. Yes. Right, so, and the numbers, they're roughly at about 700 gigaflops in 08. And if you think about it, so I think a lot of the teams that are trying to configure their hardware so that they can break a terraflop on 26 amps, which is pretty impressive, because if you think like the first terraflop machine in 97, the ASCII Red from Sandia needed 850 kilowatts. So to be able to do it on the order of three is pretty amazing. And it tells you how far cluster technology has come. That's very cool. That's HPC technology. You wouldn't happen to remember, what have the Limpac numbers been over the past couple of years? Even if we've only been doing this a couple of years, but it would just be interesting to see what the trend is. The same power envelope every year? I mean, is this something we could make an apples to apples kind of comparison and see an interesting trend on just what the students are able to do in the same amount of power with newer, better hardware, assumedly every year? Well, I could say, I think Taiwan's winning Limpac in 07 was like 420 gigaflops and ours, the University of Colorado's in 09 was 658. I don't know about 08, what Taiwan did there. I think 08 was 703. Yeah. So we kind of peaked out over the last two years. It was 703 and then 692. And the thing is that the teams come in with the back of the envelope calculation saying they're all gonna break a tariff flop with what they bring. Unfortunately, the reality is it doesn't, they're so afraid of hitting that power mark that some people don't run as hard as they could. And even, I was gonna say, a side competition, a lot of the students really get friendly with each other and build up these little side competitions. And one of them is before the competition starts or after the competition's over is to run our clusters full bore, right? Without any power restrictions. You're still only on those two circuits. And we still, we come really close to a tariff flop, but it's really hard to get over that. Okay, that's all pretty cool. How, on a different vein here, how exactly did the schools attract interest in this? I mean, how do you create a team? How do you find undergraduate students who are HPC savvy enough or interested in HPC to be able to field a good team with a reasonable showing and whatnot? I think that's the answer to the question, yeah. Yeah, the way we've been able to do it is, of course, we put up flyers everywhere. We announced it in all of the undergraduate CS classes. We try to put some advertisement out in other departments like astronomy, physics, chemistry, since the models are used in those domains. A lot of word of mouth. We have the ACM student chapter. Those are the main places that we really advertise and draw from. And we get students all over the skill level. Some of our students have never touched any clustering. Others have played with it a little bit. So that's what we've tried to do. So I think Doug is leaving out one of the important components, and that is him. You have to find a strong enough supervisor or an advisor, someone who will spearhead the initiative and really bring these students along and find out what are their weaknesses and where do they need to be augmented and bring in that information and really keep the kids motivated. The supervisor is the key component to getting them ready, bringing them to the show floor. They're responsible for them on the show floor. For us, I think the biggest way to build a team, the way that I've been dealing with building teams is finding someone who will spearhead a team, basically. And then from our side of the view from the committee side, we're really trying to, applications actually are gonna be opened on February 1st. So we want everyone to go to the website, the sc10.supercomputing.org website and take a look either at the Participate link or the Technical Program link, and we'll find additional information concerning rules and guidelines and information about the applications that we're running and general information about how to submit for we're really trying to, we'll have a flyer that will be up that everyone will be able to download and just like Doug as well as Hayat stated, we wanna get the word out to all the different departments, as many departments as possible, mailing lists, Twitter, definitely using this podcast. We thank you very much, Jeff and Brock for having us on as well as to just try to disseminate it through a lot of the other communities from their mailing lists as well. So if anyone is interested in participating, definitely go to the sc10.supercomputing.org website. The Student Cluster Challenge is also part of what's called SC Communities, which brings together programs that are designed to support merging leaders and groups that traditionally have been underrepresented in community and computing, excuse me. And hey, I had the opportunity to talk to a lot of these individuals from last year's a broader engagement program as well as the student volunteers program and education program. And so a lot of those individuals are now going out to their various universities and departments trying to let them know and hopefully we'll have some new additional teams this year that are applying. So we definitely want you to go onto the website, take a look and please do submit the applications open this February 1st. And really from a faculty perspective, this is really an excellent additional thing to your SC courses, or I'm sorry, your scientific computing courses and high performance computing courses in the CS programs. This is all hands on, ours is all hands on. So it's a great addition to curriculum that an institution may already have. I know some other institutions have actually built curriculum around the competition. So it really isn't a great opportunity. So Tiki, you were talking about the application process there and what exactly do you have to submit? Is there a lot of paperwork? Is it easy to apply? What are the requirements for it? And how do you decide on which teams will be accepted? Is there a limit on the number of teams? I assume there's a physical amount of space that you are limited to at the convention center and things like that. Yes, well, as I stated before, the applications do open February 1st, 2010. So we want you to go onto the sc10.supercomputing.org website and you'll of course have to submit your information about your team members as well as advisor and that's up to six members and they either need to be high school or undergraduates have not received a four year degree as of yet, as of the competition. The application deadline is April 16th, 2010 and we're trying to get notifications to be sent out the week of the first week in May, around May 3rd or so. So the information will be submitted electronically. Basic information, contact information about each of the participants and also we're looking for each of the teams to provide final architectural proposals around September 16th and we'll be able to, as Hayah had stated, provide the actual application such that they can run their tests. Initially we would love for them to be able to get started as early as possible with contacting, communicating with possible gender partners. That should be done as she stated before by the end of February, if not earlier. So I can just round up that information a little bit. We're looking for six to eight teams to be competing and in each team there needs to be six team members, undergraduate or high school. They need to come equipped with a supervisor and advisor who is affiliated with their educational institution and for their initial application, they need to submit a proposal that we've actually modified this year. I'm hoping it will be successful but originally they needed to submit a complete proposal with their entire architecture as well as their hardware and their software configurations and we found that we needed to give the schools more time to figure out their vendor relationship and figure out their hardware but they also needed to know in advance whether they would be participating. So we've broken up the proposal into two components. One is a team proposal where they get together with their team, they explain why this team is going to be a winning team. What kind of diversity of skills do they provide? What kind of experience do they have? Hopefully they're not coming in completely clueless and they've done some sort of research on system configurations. So we'd like for them to give us a proposal that doesn't have to be very specific but a general proposal of what kind of system they might be coming with as well as what kind of demonstrations they might do to outreach to the judges, to impress the judges, to outreach to the participants. So that's the initial team proposal and then once they've been chosen we will ask for a final architecture proposal that they worked with their vendor and their advisor on so that we know if they're coming with something completely outrageous we might be able to give them some advice. And there isn't a small additional requirement that your hardware must be, and correct me if I'm wrong, hey, it has to be commercially available by the competition. Exactly. Yeah, so you can't come with unreleased product. Well, that takes all the fun out of it, doesn't it? Yeah. Yeah, it can definitely, I mean, depending on who your vendor partner is, that can be a tricky, tricky thing. But we've modified it to be, it has to be commercially available on the day that the challenge starts. Some people want it to be that close. Well, everybody times things to be released at supercomputing, so that's probably very, very helpful, I would assume. So I wanna circle back to the team composition question here, a related question a little bit. Where do universities get the expertise to actually train the team members, particularly in some of the nuts and bolts issues, of setting up a cluster and setting up a competitive environment and things like, these are pretty specialized skill sets that good HBC administrators are difficult enough to come by, but then you wanna go and throw it in a competitive environment that's gonna span so many different disciplines and fields. You talked about getting subject matter experts in all the different fields, but what about the actual administration side of it itself? Now, Doug, you might be a bit of a ringer in this area because this is what you do all day and every day, but what do other universities and organizations do? Yeah, I mean, we've tried to mentor students both from the vendor side, as well as from, in my case, the faculty representative side. And this year, we leaned on the application specialist a lot more than we had in the past, so that students could get a much more in-depth familiarity with given applications. So really, it's a collaborative effort from the entire HBC community. We're training the next level of us, so the vendors have been great with participating, you have your faculty sponsors, a lot of the domain specialists are out there. You ask for help and a lot of people step up to actually help you. So what's the benefits of being on a team provided by ASC? Do the participants get free admission to the conference or free shipping for this 400-pound cluster, they're shipping halfway across the country? What's benefit now? It's an honor just to compete. It is, jeez. So all the team members, including their faculty advisor, do receive free registration into the conference. We hope that the vendor partner will be, I believe traditionally the vendor partner has shipped the cluster to the show floor because a lot of times they're already there and they send it with their booths or sometimes they just take up the cost on that. Other benefits are, it seems to be, one of the students said one of the biggest benefits was an unlimited supply of Mountain Dew. We feed them, but we work them really hard and the camaraderie of being there on the show floor with a bunch of other miserable students who are very sleep deprived. What other, students love that, they live for those moments. But I think one of the biggest benefits I've seen is the exposure. I mean, again, I go back to Dustin Lieberman or to some of the other students and a lot of the SC participants are coming and talking to the students and these are really bright kids and they know that it's an excellent employee base, people to exploit for their use and labor. We also hope that the vendor partners may be able to assist with travel support but we do try to work with each of the teams in terms of, of course, hotel arrangements even though they'll probably be up under or next to their clusters working on them. But in addition to registration, you want to be able to make sure that in case they do need a bed in a place to stay, we work on those logistics and help out with that as well. And in addition, as just as, as well as Doug was stating, the educational experience and being able to meet with individuals at the largest international supercomputing conference that is known. Just the opportunities besides the networking, the students can also, if they can even get away from their clusters for just a couple of minutes, can participate in a student job fair. They can also participate in a mentor and protege program if they have or so inclined to. So there are many, it's a tremendous number of opportunities that the students will receive as benefits for participating in the student cluster challenge this year. That's cool. So how does one become a sponsor? You know, if, you know, someone works for a big name organization that, I don't know, say Rhymes with Misko or something. They have no real university contacts or contacts with a team. How would you get matchmakered up with, with somebody who's looking for a sponsor? So Jeff, you can call me. I'm looking for a university of Colorado. There you go. Well, how about a general answer for any other founder or potential sponsor out there? Go ahead. You can contact us through the website, through student-cluster-challenge-info.supercomputing.org or go to, as I stated before, the sc10.supercomputing.org website. Let us know if you have any interest in being a donor, also known as a sponsor in some other communities and we'd love to get you involved. We'll also have some information literature flyers that we'll have coming out, definitely the beginning of February, which is just next week. So if you have any interest at all, please do contact us and we would love to have you participate and support some of these wonderful students from these teams that are gonna be really from throughout the world. So what's in it for the winner? Gotta ask about what's the winner gonna get? You gotta ask that. So in the previous years, there's been a plaque with your name on it, really nifty that says that you were the winner, but we are actually fostering some new ideas this year. Actually, I don't wanna downgrade that. Not only, I think one of the things the students have also done is, they've gone to conferences afterwards, explaining their experience. So that kind of exposure has been really great, but the overall winner receives a plaque as well as this year, we're talking with a couple of different vendor partners and sponsors to come up and up the ante a little bit. I know there's been some talk about perhaps donation of a small cluster or travel grants to conferences and things of that sort. So we're still working on that. I think we'll know more about prizes as the year progresses. Well, that's cool. I mean, I think that bragging rights actually in itself is still a huge thing because way back in the day when dinosaurs were walking around with abacus is I took my hand at the old ACM programming contest thing and that was just tremendously fun and it doesn't have nearly the same level of exposure that you do to vendors and other HPC luminaries and stuff that this competition does. So I would think that, I made a joke about it earlier in the podcast that it's an honor just to compete, but I have to believe that that's actually true. It's gotta be tremendously fun and tremendously cool just to do it. And I'm not saying that just for my own job security. I say that just because I can imagine that it would just be a tremendously fantastic learning experience and just life experience to do it. Sounds corny when I say it, but I still believe it. No, I absolutely agree. I mean, you know, after last year's show, I talked with all the students and it was just gushing how great the opportunity was. I think their only complaint was, that they didn't have sleeping bags at the show for, but other than that, you know, they just love being there and they love the competition. Doug, Tiki and Heya, thank you very much for your time, for doing this and this will actually go up after February 1st, when this goes out, the actual applications that are listed will, I believe will be listed, right, February 1st. The applications? The applications open. The applications open February 1st. We'll have additional information on the website at that time. We will also definitely update the SC10 website under the technical program and communities program concerning once we finalize the applications. So I think just to clarify for the listeners, let's make sure, I think we're overloading the word applications here in two definitions. Tiki, could you explain? Yes, definitely. The application, the process to submit an application, a team of up to six members and an advisor will open on February 1st. So we want all interested individuals and teams to go on to the sc10.supercomputing.org website and we'll provide you with the link that will allow you to submit your information about your team, to answer some of the general questions and to provide an initial overview of your architecture proposal. Now, as time progresses, as we continue on, we have not finalized the actual applications that the students will probably be run on to give them some test data that we will update on the SC10 website so that you can be informed and see what types of applications that you can run your cluster on to prepare yourself for the upcoming challenge. This will be out soon and thank you for your time. Thank you. Thanks, everybody.