 Welcome to another edition of RCE. Again, this is Brock Palin. You can find us online at rce-cast.com. You can find links to all the usual Jeff's blog, my blog, our Twitter accounts, and all the usual social media things out there in the world. Jeff, once again, is here to help me out. Jeff, thanks again for your time. Hey, Brock. It's been a little while since we've been on. We took kind of a summer hiatus, but we're back. We got a bunch of shows lined up, and let's just jump right into this first one. Okay. Our guest today is Steve Tiki from the University of Chicago. He's going to be talking to us about Globus Online. So, Steve, why don't you take a moment to introduce yourself? Yeah, hi. Thanks for having me on. Yeah, so Steve Tiki, I'm with the University of Chicago and Argonne National Laboratory, and I'm one of the leads of the Globus Project, which is really about providing software to help researchers get their research done. Okay. We've actually had the Globus toolkit on the show before. We'll include links to that to kind of talk about what the Globus toolkit software package is. So, can you give us a little bit of an overview of what Globus Online is and what it aims to accomplish? Yeah, so maybe as a starting point, I can do that as a comparison even to Globus toolkits, and some of you may have familiarity with that. You know, we had spent starting in the mid-90s, a lot of time building really plumbing software, Globus toolkit, things for helping scientists move around big data, deal with their security information services, stuff like that, but we really focused at the plumbing layer and expected other groups like the high-energy physicists or climatologists or other groups doing their big science to build their own custom solution over top of that underlying plumbing that we provided in the toolkit. After I went off and spent a little time in industry and then came back to the university about five years ago, Ian Foster and I, who's my partner in crime and all this, decided that we really wanted to generalize what we were doing. So, I'd take a lot of the lessons we had come out over the last decade and a half in helping scientists deal with big data, but produce tools that would be applicable to the rest of the science community, not just to the few big, rarefied science communities. So, that's really our focus with Globus Online is taking a software as a service approach, you know, things like Netflix, Gmail, sort of online web-based approaches for managing your big data transfer and sharing and longer term getting into rich or richer research data management functions for the nonprofit research community. Now, can you explain that a little bit more? Because, I mean, to me, when you say web-based transfer of terabytes worth of data, to me, that's like trying to upload a video to YouTube and getting really annoyed because it fails halfway through. Yeah, because that's in my mind with the typical state of web-based transfer is. Clearly, you're talking about something different, though. Yeah, you know, I use the analogy a lot, right, that my seven-year-old son every day streams gigabytes of video to our house with Netflix, right? And why isn't it that easy for us as scientists to move our data around? You know, and think about some of the use cases. Maybe that's even a place to start, right? Is that, you know, as a scientist, you know, there's data coming at us from all sorts of different directions. You know, maybe I'm using scientific instruments like NextGen sequencers or MRI machines or light sources or things like that. Or maybe I'm running simulations on an HPC cluster so I need to, you know, I got big output files with my simulation results or I've got big data coming that I want to do analytics on from various, you know, population sources and the such. You know, that's data that I need to deal with and get to the right places, right? Whether that's to my HPC processing cluster or to my cloud running in Amazon, running Hadoop jobs or, you know, to my desktop or my server sitting in my closet in my lab, right? There's all these places I need my data, right? And, you know, as I sort of analogy with my son in Netflix, it should be that easy for me as a scientist to get that data wherever I need it to be, right? And to be able to do that by just simply sort of point and click, you know, get to the various end point systems, say what I want to move from A to B, if there's failures, have that just taken care of, right? When Netflix, you know, they've got their Simeon army that goes around killing servers and even hold data centers randomly to make sure their reliability works, you know, my son doesn't have to debug that when those things fail, right? And the same should be true for us, right? That it should just work. We should, as the service provider, be taking care of that. With one big difference, of course, which is in this case, the data is sitting on your own storage systems, right? So we have to be able to orchestrate and manage these transfer and sharing activities between these existing systems, leveraging the high performance networks, all that sort of stuff and act as an overall kind of mediator or manager of these activities so that you don't have to do that yourself. So you're using the name Globus for this, but like we had the Globus toolkit on here before, which is the underlying, you can say the heavyweight software part of this. And it actually does a lot of things besides just file transfer, but we're just focusing on file transfer right now. Why did you select to go with the name Globus? Why not just like wet, grid FTP or something like that? Like why? Yeah, that's artifacts of history, I suppose. We went back in, what had been in 1995 when we wrote the first DARPA grant for the software, there was, we had a previous project that I was also a lead developer on called Nexus. And it was a communication library for clusters that we're starting to use in some interesting ways for more distributed environments. And so we, when we were writing that paper, I remember we put in a latex macro for the name of the project and stuck in Globus as sort of the global version of Nexus and never really went back and changed it. So kind of stuck. And then when we started up Globus online, it really was, we had debated a lot, do we keep the sort of Globus name at the heart of it or do we go off with some other whole new name? But I don't know, we like the name as a good generic name. And it kind of conveys helping scientists with distributed big data stuff. So, that as much as anything is why we just kept with it and kept moving. And so we really then talk about Globus toolkit as that plumbing layer toolkit that we and many others use. And then Globus online as the software as a service that uses the toolkit but provides this real end-to-end experience for the users as software as a service. All right, now what, you outlined a couple of these already, but let's go a little deeper. You said we're to some of the advantages over say a web-based upload, but let's go to something that's a little more robust in a web-based upload like an SCP or an SFTP or the other traditional file sharing things that seem to work pretty well or even better, a sneaker net. If I just FedEx myself a couple of DVDs worth of data, what's the advantage of what you're offering? Yeah, so good timing. So I just actually got off giving a webinar with the ESnet folks who are one of the national high-speed research network providers for Department of Energy. And they actually had some great slides on this. I would definitely refer your listeners and you guys might even wanna interview some of the ESnet folks in a future podcast. They talk a lot about the networks and the high-speed networks. And so they had a great slide about exactly what you're asking, which is that we have networks now to campuses that are gigabit going to 10 gigabit, even 100 gigabit networks. And 10 years ago when we were only talking 10 to 50 megabit network sort of things, schlepping around USB drive, shipping and FedEx was probably the fastest way, but they actually had showing the numbers of it's just not the fastest way anymore. And so they went through these various tools. So for example, SCP, it was not designed for high-speed, high-latency networks like you hit when you're moving a lot of data around. It does well sort of very local area environments and the like, but they showed some experiments where on a 10 gig network, they could only get about 150 megabit per second out of SCP, it's just not tuned for that. There's patches to SCP that allow you to do higher performance, maybe get up to the gigabit range. Traditional tools like FTP, similar sort of things, you can get to the gigabit range, but they're not really tuned for lots of files and smaller files and exploiting parallel streams and all this sort of thing. So now there's just this whole set of tricks, I guess, as much as anything that we and many others in the community have developed out over the last couple of decades for how do you really move data at sustained rates of gigabits per second, even into 10s of gigabits per second. And to first sort of grid FTP as the underlying Globus toolkit tool and Globus online as a client to that, it just plays all these tricks. It knows all the games to play to tune your networks for parallelism and for buffer sizes and pipelining and all the things you need to do to make it work so that you really can sustain gigabits to 10s of gigabits per second on real transfers on real systems. So you've been talking about making this easy to do online and stuff. So it was a transfer actually happening through a web browser on the client end and there's a server that an admin would install on the local cluster. Like what's actually going on here? What's the actual interface look like? Yeah, the model that we've got is, I can actually use an analogy of Dropbox. You know, and I'm a big Dropbox user for my little data, for my Word files and stuff like that, right? And the model there is I stick this agent on my laptop which connects my laptop to the Dropbox cloud, right? And the analogy is very similar to what we do in Globus online is we have a set of products we call Globus Connect which are really just fancy easy installers around Grid FTP and a few of the other Globus toolkit tools. But with Globus Connect, the idea is you take your storage system that you want to make available and manageable within the Globus online universe, put this Globus Connect software on and it connects it to the Globus online cloud, right? And we have versions of Globus Connect for everything from high performance servers. So if you have got a big parallel file system like a GPFS or Luster on a high performance computer or super computer, you know, you can have a bunch of these that can sustain huge data rates. On the other end of the spectrum, we have versions that are designed to work on laptops. You know, sit on my Mac menu bar, operate in the background, work behind nets and firewalls, right? With the Globus Connect tool, the whole point is, you know, get whatever storage system matters to you connected to the Globus online cloud. So then Globus online can do its thing. And so then as the user, once I've got the storage systems connected, I just log in via the browser. We also have command line interface and REST APIs behind all this if you want to do that. But the typical mode is I just log in with the browser, connect to the two, what we call end points, those two Globus connected storage systems, log in as necessary to each of them, because they may be in different security domains. And then point and click, you know, say transfer this folder or set of folders from one side to the other and walk away and it'll take care of it. I can come back at any point, check on it. And Globus online will keep working. If there's failures, it'll retry, you know, all the sort of stuff there just to make it work. I see. So the web browser is not really the transfer agent. It's the command and control. And that's kind of a major difference between the scenario that I said before of uploading to YouTube and getting annoyed that it died in the middle. That's right. It's not an upload download to the browser, but really, you know, the very common case in our community is what we sort of call third-party transfers, right? The data is sitting on my lab server and I need to get it out to my HPC cluster or super computing center or Amazon, but I'm doing that all from my desktop or my laptop, right? So we're able to orchestrate from a third-party location from my browser all of these data transfers and synchronizations. And I, you know, I've mentioned sharing very briefly, but also sharing, you know, not just transferring amongst my machines, but saying, here, Brock, I'm gonna share this folder with my research data to you and you can pull it down onto your systems just fine, all with the same mechanisms, all orchestrated, you know, via the browser. So you mentioned this kind of earlier, but I'm curious how it actually does it. So if I use the Globus toolkit directly, I can specify parallel TCP streams, striped transfers, concurrent transfers, if I'm moving a lot of small files, but I notice with the Globus online, I don't have a lot of those options, but you alluded to that it knows all the tricks. How does it know it? Does it like run a test? Like, how's it do this and understand that? Magic. It's basically, it's a set of heuristics that we've built up over the years that, you know, so for example, you know, there's a few main kind of tuning options that you can do within the sort of grid FTP Globus universe. You can talk about concurrency, that is how many simultaneous sessions of transfers are we running at a single time? Can talk about parallelism, which is between any two grid FTP servers, how many individual TCP streams are we keeping active at any given time? You know, about pipelining or sort of keeping the request flowing. So if you have lots of small files, you're not stalling out your TCP tree streams with all the control stuff, right? So there's things you can tune on all these axes and as well as a few others. And so we've come up with some heuristics where we, you know, we look at the file sizes involved and, you know, where you're transferring to and setting set these parameters based on, you know, just having done this too much over the years, you know, and we do it, I'd say the sort of 90% good job, right? For the typical scenarios of transferring, you know, within either on campus or, you know, within say the US, you know, our tuning will get you 90% or more of the way there than what you could do even if you knew what you were doing down to the mundane details. You know, there's some scenarios where like you get into really high latency or high loss networks where you might want to do a little more custom tuning and you can do a little bit about the command line interface, but the first order, nobody does. And that stuff over time will just keep improving those heuristics, you know, looking at, you know, what we're seeing to, to keep sort of modulating these parameters as necessary to make it work well. Now, when I, let's say I'm a random researcher and I have, you know, several terabytes of data at, you know, Brock's university, right? University of Michigan. And I need to migrate them to some other institution. What's another institution that uses this? No, University of Chicago. There you go. I need to move to University of Chicago to run them on the machines over there. And I initiate a transfer. Does this mean, I want to dive in a little bit about how this works here is where I'm going. This means that both Brock and CIS admin at University of Chicago needs to have some agents running between their file systems, right? And I'm effectively issuing commands to that saying, hey, copy some of Jeff's files from Michigan to Chicago. Is that how it works? That's right. Each of the two store systems, the one in Michigan, one in Chicago would be running one of the versions of Globis Connect. You know, there's desktop versions, there's server versions, you know, depending on, you know, who's installing and all that. But, you know, in this case, let's say, you know, you're using the server version. So the admins are installing what we call Globis Connect multi-user. And that installs the necessary software at the local system, both to allow you to log into the system or log Globis Online to kind of log in as you temporarily to, you know, where you provide your password, Globis Online gets sort of a short-term security ticket that it can use to interact with the system to do things. All right, so basically log into both sides in that service and then it's got the Grid FTP servers to talk to those local storage systems. And so then Globis Online, what it's doing is, you know, when you, you know, it has functions for doing directory listings and stuff like that. So that's what you're using. It's going out to those Grid FTP servers to when you're doing the browsing to pick your files. And then when you fire up a transfer, you say, here's this folder that I want to transfer from A to B. You know, that basically starts a workflow of Globis Online talking to each of those Grid FTP servers at the two sites and just orchestrating the activities, right? So it'll do things like use the directory listing functions in Grid FTP to walk your folder to figure out what files you have that need to be transferred. And maybe if you've turned on the synchronization options so that you only want to transfer files that's changed, it may, for example, do checksums on those files and then look on the destination directory and do directory listings and checksums there to see what's there and decide what it needs to transfer. And then it'll talk to both points and tell them you guys set up a direct connection between yourselves. And on the sending side, you send this file on the receiving side, you receive the file and write it out locally, right? So it's sort of orchestrating these sorts of workflows throughout. Verification is a big feature. There's a lot of history in our community and even a lot in various papers showing that things like TCP are not reliable from a bit reliability perspective so that bits can get corrupted in TCP undetected. And so we've built in extra levels of verification on the back end of your transfers or reread your files to check sums as necessary just to make sure everything's working correctly. So it's really sort of at Globus Online managing this workflow of these two grid FTP servers at each end talking to each other and talking to Globus Online. Okay, now you mentioned something interesting in there, the server version, multi-user version. Does that come with some kind of, you know, say policy support so that Brock can say, oh, I only want, you know, of all the transfers going on right now, I want, you know, a maximum of this many megabytes going upstream at a time and this many megabytes going downstream and possibly even specifying specific network interfaces to use and all these kinds of things. Not yet. We actually just, we had written a proposal on that that didn't get funded. I mean, I should have found you earlier to support that proposal, but it was a bandwidth manager to do precisely what you said. There are some ways that you can modulate this depending on what you're trying to do. So one thing a few sites have done, for example, is use like the Linux kernel parameters where you can set, you know, network sort of artificial network speeds on particular interfaces. If you wanna sort of tune down a server, you know, we'll tend to use whatever you can give us at this point. The other thing that's happening a lot in our community is these science DMZs, you know, the stuff again out of the ESNet, folks I mentioned earlier, where it's sort of a particular way of architecting your networks and research data environments on campus to make sure you really have the high band with paths but also that are protecting your other flows, right? That you're not squashing all of your other normal web traffic that your students around campus are using. So there's various techniques more at the network architecture level for doing that. You know, we don't yet have the capabilities to say, you know, I want group A to get three quarters of my bandwidth and group B to get one quarter of my bandwidth. That's what we proposed, but we don't yet have the funding to implement that. We'd love to at some point though. Okay, so you had alluded to sharing a little bit. Can you explain a little bit about what sharing inside Globus looks like? Yeah, the model in Globus online was sharing or maybe step back and start with the use case is that every researcher I know is using Dropbox, right? They're using it for the doc files and PowerPoints and the small stuff. And that sort of centralized cloud synchronization model works pretty well for that small stuff, right? I can copy everything up to the Dropbox storage and synchronize it all to each of my machines. Wonderful for my day-to-day files. But that model falls over when you start talking about big data, terabyte data sets and the like, right? Because one, you're paying a lot of money for that cloud storage. Two, the performance to and from is not necessarily all that great. And three, I often don't even want all that data on all of my different machines, right? I often, when I start talking about scale of data, I wanna be more selective about what I'm putting where and when and sort of the workflows around that. So what we've built with our sharing capability is our mantra there is file sharing, big file sharing and transfer with Dropbox-like simplicity from your own storage systems, right? And so the model is that as a user at a site that's turned on the ability to do sharing at a global connect endpoint, I can log into my normal file system, my normal endpoint as myself, but say, here's a folder that I wanna share out to other users and I give it a name, sort of think almost like a mount pointer, virtual name that I can just share with other users. And at that point, those other users can use that sort of virtual endpoint just like any other system name I use, any other place they can log into and do transfers from directly. But the key real feature of it is that, if I'm sitting at University of Chicago, I have my data's on the University of Chicago machine, but I wanna share it to you, Brock at University of Michigan, you don't need an account on my University of Chicago machine. Only I need the account there. By virtue of sort of exporting it, sharing it out to you, Globus Online can take care of the security across the institutions for these sharing purposes. And then Globus Online can do fine-grained access control. So I can say, I wanna give Brock access to these files read, write, but Jeff, I only wanna give you read access to them or only give you certain folders. And we also have group capabilities. So I can say, I wanna give rights to this whole group, my whole project team. And if I add a member to that group, they automatically get the access that's been given to them on these shared file systems. So we've tried to make the model really as easy as Dropbox users expect for their little files, but make this work for the big files from your own storage systems. So what about like read-only anonymous? Yeah, we have read-only, I'll be careful, I'm gonna be a little pedantic. We have read-only to all users, all Globus Online logged in users. We don't have reading, sharing yet for unauthenticated users. That's on the plan, we just have not yet finished this off. The sharing functionality is in beta testing at the moment. So it's relatively new and we have lots of great ideas for where we wanna take this, including that one. So do you see yourself having Globus Online be a universal scientific identity manager, Clearinghouse, kind of similar to almost like some of the stuff that Incommon or CI Logon are doing? Sort of. So the work that, for example, Incommon in terms of providing sort of basic infrastructure where I as a University of Chicago employee can use my University of Chicago login to log into various services, we have support for that in Globus Online. We can leverage that very well so that we can actually provide single sign-on using my campus University of Chicago ID in the Globus Online and back out to University of Chicago resources, right? That's a standard feature in what we do. But we've also built up all sorts of great capabilities. I mean, if you think about fundamentally what we've done, right? We've created this nice web application that plays really well with research infrastructure and security domains and Incommons and all this sort of stuff so that we could use that to build transfer and sharing. But what we realized some while back was that everything we were doing there, maybe others could get used from that and then maybe we should make that available as platform as a service as well. So we've actually now got a few other groups, for example, the KBase project out of the Department of Energy. It's a systems biology project building a web application, softwares a service for that particular science community is leveraging our platform, all of the Globus identities and the ability to do federated identity with Incom and in groups and all of that for their application. So all those same groups you use for sharing in Globus you can use for access control in KBase, right? And so this is part of our strategy. So it's not really to replace those systems as much as provide the tools that make it very easy for other groups who want to create their own web applications for this community to leverage all this rather than have them have to reinvent all this stuff themselves like we had to. Okay, now taking a slightly different direction here and someone going back earlier in the conversation, you mentioned Globus Toolkit and we've talked a lot about Globus Online. So Globus Toolkit is open source, right? And you can just get that and do whatever you want with it. But Globus Online is a service. What is the relationship between these two? How do these work together? What is the differences? What are the commonalities? Yeah, so Globus Toolkit as you say, it's open source. It's a patchy license, pretty typical open source project. You can do whatever you want with. Globus Online, we made a conscious decision to say that the software is a service. The part that we run at University of Chicago that it's not open source, that this is something that we're keeping. And there's a variety of reasons for this ranging from that, one of the lessons I learned in my stint out in the commercial world was that the whole software as a service development and delivery model is very different and has different efficiencies to it than traditional like open source software development. And so for example, by not actually distributing our server side software, I probably am saving 30% of my development costs because I'm not making it work in different environments and porting it places and trying to make it so that other administrators, other than our own people can actually install and use it. So there are actually a lot of efficiencies on the creation of the software itself. And so the result of that is I'm able to give a lot more functionality in Globus Online by doing it as software as a service with us being the only ones who actually operate it and use it. Other reasons is that we also, and we've been in this game a long time and recognize there's this big problem in the research world around sustainability. And there's been lots of hand wringing around that term of sustainability in our world for a number of years now. And so one of the things we are actually doing with Globus Online is starting to introduce some of the capabilities like the sharing as an actual paid service. As what looks to the user, a lot like commercial software as a service. Now we're doing this as a nonprofit from the University of Chicago for the nonprofit research community, but we're doing this so that we'll be around 10 years from now still providing better and better capabilities. And so part of the strategy is that the software as a service is something that we're creating, we're owning, and we're providing for payment to the community and that's what partly sustains us. So essentially our model is that the software as a service that we run is ours, but all the Globus Connect bits, all the pieces that you would install out on the periphery that would connect to that, that's all open source and part of the Globus toolkit, including all the Globus Connect pieces. All right, now another question, kind of tying into that, but going back to how does this work here on the homepage for globusonline.org, which I amusingly note has a picture of Brock right on the front page. There is also a counter in the top right. You're a poster child. You really are a poster child. There's a counter in the top right here that says how many megabytes have transferred and I can't wrap my brain around it. Is that exabytes? How many bytes is that? That's almost 21 petabytes at this point. Almost 21 petabytes, I agree. Now, where are you getting that number from? Because a lot of these transfers are going site to site and they're not going through the University of Chicago. So I mean, are you sending these kind of statistics? I'm just curious as to how that number is assembled, I guess. So remember, these numbers are only for transfers using Globusonline, not all grid FTP servers out there that may be using other clients, right? Because there's a lot of grid FTP servers out there, for example, in the energy physics community that are using other clients, whether they've developed them or some command line clients that are in Globus Toolkit to do transfers. And actually on that, we're seeing about a petabyte a day of traffic based on, we have sort of this anonymous usage collector, usage logs that come back from grid FTP servers that turn it on. And so it's actually probably more than that because we know a lot of people don't turn it on. So that's at the grid FTP level, the Globusonline level and what you're seeing on our homepage. Remember, that's all requests coming through Globusonline, through the Globusonline service where we're mediating all of that, right? We know there's this file of this size being transferred from site A to site B. We don't see the data itself, but we're, because we're managing it, we know what's being transferred around by Globusonline and then that number is actually just a query against our database that's within like 30 seconds accurate. Okay, so you've touched on a couple of things there and so I wanna clarify a little bit, like if you wanna try this out, what can you get for free and what do you get if you pay? Yep, so what you get for free is you can do as many transfers as you want, as many as synchronization. The sharing is really the first feature for which you're doing paying. So you can create your own endpoints, you can put your laptops on, admins can put your servers on, right? All of that's free, getting your systems connected, doing transfers amongst those systems, that all that is and will remain free for the nonprofit community. As if you want to start sharing, right? If you wanna create what we call the shared endpoints, you'll take some data from my endpoint and share it out to you. You need to have what's called our plus subscription, that's our sort of our subscription plan that gives you access to some of these additional capabilities and we'll be adding other features to that plus subscription over time. And so that's what you need and that's the one that you can either get for $7 a month or 70 bucks a year and we'll have credit card processing literally in a few weeks here for that. And then we also have versions that campuses can purchase that you get a bundle of plus subscriptions you can give out to whoever you want as well as getting some management tools. So you can sort of manage and see all the transfers happening to and from your endpoints, your Globus Connect servers via Globus Online and see what's happening, if there's issues you can see all that, suspend, resume, all that kind of stuff. Okay, so to clarify a little bit that $7, that's for the Globus Online user. So me as an admin, I install the software I can enable, disable whether I'm gonna allow sharing on my machine but I don't have to pay anything I can just install the software and then the user using it can pay the seven bucks or me representing the University of Michigan can go and get this site license basically and then all my users can get it. Yep, you got it. And that's on purpose because we recognize some of the campuses and even some of the more get down to individual research groups, they just wanna get up something up and going and they may not need the sharing and so the transfer's sufficient. So we've purposely come up with this model where you can get going for free and you can do a lot of stuff for free but as you start moving up into some of the more advanced functionality we have ways for the sort of different parties within our community and users and our campuses to pay for that advanced functionality. Now, one of the hot button topics today is social. So are you gonna be integrating social services in here like Facebook logins and Facebook likes of certain files and things like that? We actually have some more to pay. I'm totally kidding. There is no answer to this question. That was a joke. We should probably just edit this straight out. We have some of those over beer but actually we do allow login using Google open ID today. So we don't know. Really? Okay. I was kidding but that's actually nice. Yep and we have considered Facebook but we haven't had the demand for that one with Globus online yet. As you probably know right in our the research and university community there's actually a lot of universities starting to integrate more closely with Google and Google apps and stuff like that. And so that was actually there was real demand for that capability. Okay. Well, if you ever add Facebook liking for individual files then you have jumped a shark. So what's coming for the future of Globus online? So we've talked about a couple things that are coming up you've mentioned incidentally but what are some of your other bigger plans that you can talk about? Well, I mean we're still in sort of late beta testing on the sharing functionality. So we've got our hands more than full just getting this out the door and we've got a huge laundry list of stuff we can do to improve little features around that. So certainly you'll see for the next six to 12 months most of our activity will be just happening around making that service better and better and easier and easier with the features for the users. But beyond that, I mean there's a few big things. For example, we're getting a lot of interest from campuses in Amazon web services and being able to transfer files to S3 into Glacier, stuff like that whether it's for long-term archive and a service like Glacier or to get my data up into S3 so that I can use burst processing in EC2 whether it's high performance computing or Hadoop type jobs. So we're getting a fair amount of demand for that. So we're starting actually some of the design activities around what would it look like to do really high quality integration of Globus online sort of native deep integration with S3 so that sort of all the stuff, the transfer sync sharing, all the verification everything that we do just works with that sort of environment. Longer term, we're also really starting to look hard at sort of the next layer up the stack, right? Today it's about files, right? It's about transferring your files, synchronizing your files, sharing your files. But of course there's a lot of other research data behind besides files. And so our real overall mission is not file transfer. It's research data management and across all of that data across the life cycle of the data. So we're starting to do a number of pilots with various groups right now to start getting into the whole metadata arena so I can start tagging my files, searching for things, sharing both of the files and the metadata. Really having a way to capture more and more of that research data and be able to manage it and share it through its life cycle. So a random question and this might sound like I'm continuing the Facebook joke but I'm not. Do you offer, what kind of notification do you offer? Because the whole point of this is to transfer very large things that assumedly take some time, right? You got to start the transfer and then go off and do some other things and then oh, okay, now the transfer's finished and I can actually do the research that I'm trying to do. How do you notify people this? Like, can I get a push notification on my phone? Not currently, actually. We were just discussing that one last week also. It said we're, right now what I get is that you can get an email notification of completion or of certain fault conditions that you may need to respond to. For example, maybe you need to re-authenticate, re-login to a particular system to allow the transfer to continue right where it sort of left off. We're actually, I mentioned the long backlog of items or to-do list, to-do, that's one of them is to allow these multi-channel mechanisms for notification, whether it's email, text messages, as you say, push notifications to iOS or Android apps. Even we've talked about things like instant messenger. So I could send a notification to my G-Talk or I guess it's now Hangouts account. So that's definitely something that's on our roadmap at some point here. Okay, so something we'd like to ask people is what's the most unusual or strangest you've ever seen Globis Online used for? I don't know, I suppose strange. Strange as compared to some of the research stuff. I guess one I would, a couple maybe I'd point at. One that was sort of a funny one was back at Globis World in April, which is our annual conference. We were having a bunch of the sessions videotaped and right after I did my part of the keynote where I showed off all the sharing functionality at the break after that, the fellow who was videotaping everything came up to me afterwards and said, do I understand this right that I could use this to allow me to get these videos I'm doing out to my customers? Yes, why yes you could. So we're seeing things like that and we've also seen actually some interest even interestingly in the film industry of groups that need to transfer movies and stuff like that around within particular industries. So we're getting some of those sort of more commercial and use cases that we certainly hadn't thought of when we were building this showing up. I'm not sure if they quite qualify as strange but they're certainly not what we expected. Okay, well Steve, thank you very much for your time. Where can we find more information on Globis Online and get started using it? Yeah, just go to www.globisonline.org and you'll see there lots of information, sign in, sign up. Go give it a try. Okay Steve, thank you very much. Yeah, thank you. Thanks for your time.