 Welcome to another edition of RCE. Again, this is Brock Palin. You can find us online at rce-cast.com. You can find a link to all of our blogs, all of our Twitter, and our entire back catalogue of over 100 episodes on scientific computing, engineering, and other things. I have again here Jeff Squires from Cisco Systems and one of the authors of OpenMPI. Jeff, thanks for your time. Hey Brock, how's it going? Good, good, good. Before we roll into this, I do want to make a small announcement. I have a little bit of a different job opening available right now in Ann Arbor at the University of Michigan, developing using containers and container orchestration software for big data. So if somebody has experience with Docker, Mezos, Kubernetes, or anything like that, hit me up. You can find contact information on the website at rce-cast.com and let us know if you're interested in that position. All right, with that administrative note, let's jump right into it today. Today we've actually got three guests on here and we're going to be talking about the Sage 2 project. So gentlemen, I wonder if you could introduce yourselves. Hi, my name is Jason Lee from the Laboratory for Advanced Visualization and Applications at the University of Hawaii. Hi, my name is Dylan Kobayashi. I'm a student founder of Jason Lee. I'm a working force modification in computer sciences and I've been working with the Sage 2 project for about two years now. And I'm Luc Ranambou, a research associate professor at the University of Illinois at Chicago at the Electronic Visualization Lab. Okay, so we're here to talk about Sage. So I've been following Sage for a while. Can you guys give me an overview of what Sage 2 is? So Sage 2 as the name implies is the second generation of Sage. And what Sage was was essentially a piece of middleware that allows you to drive large display walls as if for one contiguous canvas allowing you to post visualizations and information onto those walls. Whereas Sage was all written in C++, the new Sage 2 architecture is entirely redone in Node.js and JavaScript technologies. So what motivated you to switch from C and C++ to Node.js? Because off the top of my head it would seem that C and C++ would be a much higher performing language and have better optimization possibilities than the other technologies you cited. So part of it was actually an experiment. We had started noticing that computer graphics on browsers were starting to get powerful enough to be able to do something real as opposed to just little video games. And in a little experiment we decided to write just a small piece of test code to see how well it performed for Sage-like tasks. And then we said, well, let's go full tilt and see what would happen if we tried to port the entire Sage architecture to it. And surprisingly it worked very well. And on top of that we get the benefit of leveraging a lot of the JavaScript infrastructure and a lot of the tools that this large community has already been developing. So what are some use cases then for Sage or Sage 2? Wow, let me do that again entirely. Let's pretend we're still on the break and I'm going to just start again. So what are some of the use cases for Sage 2? I mean, why would you buy a video wall and what are some things that you would want to display on it? The community of users that we work with typically have large amounts of data and they produce visualizations of them and just one picture doesn't give you the answer to the research question they're looking at. Usually it's a bunch of different images from different data sets and they're trying to put it all together like a giant jigsaw puzzle so that they can come up with whatever story is hiding behind all this data. It's very typical for users to come to a meeting room with the large state walls and start throwing information onto the walls and talking about them. Or they would form a little meeting where as part of a research meeting people would come in and deposit information onto the walls and have discussions. Okay, so most of the time when I see display walls being touted I'm always talking about how many megapixel it is and they always have one giant image of the cosmic background radiation or something like that but you're saying the most common use case is actually to generally have multiple images up there through time or different variables for the same simulation. What's the goal here? Is this a productivity boost for researchers or just a necessity? Yeah, it's both a productivity boost but also a creativity boost. When I teach classes for example using SAGE I often give my students a data set and then when they come back I ask them what have you been able to visualize with this data set and then they can throw up all the different approaches that they've come up with to visualize the data like a giant pinup board and then we can discuss them like why did you choose to use color to represent this particular attribute versus size? And you're absolutely right, historically large display walls were typically used to drive one giant picture at very high resolution and that made sense in the past because part of it was pushed by the HPC world where we had enormous amounts of data and we wanted to try to produce the best visualization possible using the highest resolution lens possible. But as people start becoming more productive and trying to use these displays they thought well why don't we have our cake and eat it too, throw both high resolution content but also lots of it because frankly there is just so much different types of data that they have to look at. Okay so let's jump back a little bit here. What exactly is the relationship between the University of Illinois in Chicago and the University of Hawaii at Manoa where you guys are working because clearly we have two groups of you on today. What is the breakout here? How do you guys collaborate? So I was previously the director of the lab in Chicago and I was professor at UIC before I moved to Hawaii just because the weather is so much more awesome here. And so this project was something that we began back in Chicago as far back as 2004 and so this is just a continuation of that project. Okay so you said in the past people were just driving large megapixel displays. There was another project that appears to be dead now called Chromium. Is there any relationship between Sage and Chromium, different philosophy, two people trying to do the same thing, what's going on? Yeah so Chromium was really an implementation of OpenGL over the network if you want to see it. The goal was kind of the same. You wanted to drive large display wall with real-time graphics and one approach was to basically send the OpenGL commands to the display nodes directly. But that's a model that Jason described where people were putting large amount of effort to drive the wall with one application, displaying a large simulation result for instance. And if you wanted to see the second simulation result, you have to stop the application and start another one or something like that. And when you compare to how you really work, that's not the way you work on your laptop. For instance, how many applications do you have open when you try to solve a problem? You have email, you have an office suite, you have a simulation package, you have statistics and so on and so on. So people do not work with just looking at one image or one representation. You want to see multiple images, multiple views of the same dataset or multiple dataset from last week or the previous run of the simulation and so on. So that was one goal of Sage was trying to let you show multiple dataset at once and leveraging those high resolution walls, not just for demonstration but for actual work. So how does this work? What is the architecture here? Do you have to have a dedicated server set up and you send the output of each application to the server who then projects it to the wall? So yes. So the Sage 1 model was really pure streaming. So an application was rendered on one machine and we were streaming pixel to the display nodes. And then when we saw the power afforded by the browsers, we changed that model and said let's try to do the rendering inside the browser. So now we have some kind of web server in the middle that manages all the resources but the rendering is done on the display side. So you talked about having multiple applications and the way you work on a normal machine. I'm sitting at my desk here and I've got my laptop with two 27-inch displays hooked up to it but I'm out of ports to be able to connect stuff. If I wanted to have more, could I use Sage just with my normal machine and have more email and more terminals and more things, does that work? Or you just talked about having a web server, does everything have to be a web app? Yes, I mean yes. One trader was to say let's try to use only web development tools to drive those display wall because we gain a lot by using the browser as a runtime system. So we leverage the audio API, the 3D rendering API and all the web APIs available and the tradeoff is you have to convert your application to web application. And when that's not possible, we use a lot of desktop sharing where you just push your laptop to the wall and using the same technology you see in Hangout or Skype desktop sharing and so on. Okay, so you have to be using some middleware that makes your desktop show up in a web browser to make it work. So I can have Matlab plot some giant thing on this but I'm doing it via screen sharing and then the wall is just scaling it up. Yes, that's the first step to show just the immediate content but then if Matlab generates high resolution images you take that image and drag and drop it onto the wall and see it in full resolution. Yeah, what we found is that sort of many science based applications have been moving towards a portal interface for their end users. In fact, even things like Google Docs or Microsoft Online where you're actually typing in your word processor or your spreadsheet directly into the browser itself within the Sage 2 framework we can actually bring those documents directly onto the wall now as part of the applications that you can show. Even many commercial vendors are producing historically very powerful and well used visualization tools like Arc or VTK. They've also jumped on the browser bandwagon and because of that we're able to take advantage of it and very quickly put those capabilities into Sage as well. So is Sage 2 then pure software or are there specific hardware requirements for the server side or the video wall side? There's no strict set of requirements. So currently we have a gaming PC actually that you can buy just from almost any of the major sites. For example, we use an LLMware and it's just a regular consumer PC. There's no special parts and it can run on that perfectly fine. If anything, the limiting aspect in this particular setup is the network because we need to transmit large amounts of data so the network has to support it. But if you don't have to worry about the network usage being particularly high then it'll work on a regular home network as well. Okay, hold on. I work for Cisco so I got to nail down what you're saying about the network here. When you're talking about a large amount of data are you talking about like 802.11G has a problem or does 802.11ac be good or are you better wired in for one or ten gig connectivity? What kind of bandwidth are you talking about here? So that actually depends on the wall and how it's used as well. So I can give an example of the summer I worked on a 40 gig Melanox network and the particular data they were using were high resolution they were peaking over 15,000 pixels by 15,000 on a small image. The really big images were close to I think a gig in nature but those had a very difficult time being updated at constant rate especially when they had two streams going and they sort of gave up on that idea but for the sake of having a constant stream of updates yes it really depends on what you're trying to show. Okay, so from what I just heard from you there 40 gig we're definitely talking a wired network for any kind of reasonable type of visualization. There's a bit of difference between different steps so if you are within the room between your laptop and the wall a wireless network is perfectly fine. It's when your application wants to access a large amount of resources and that communication is mostly between the stage two server and external storage for instance and that you can leverage whatever infrastructure you have and leverage high speed networks if you need to. So does the amount of data being transmitted to the wall then depend on the number of panels in the wall? No, it depends on your application. So if you're just showing pictures and PDF documents it's going to be using very little network. If you want to show 4k movies at uncompressed resolution for instance or very high resolution animation that's going to be using a lot of bandwidth. So it's really dependent on your application. It's like asking what bandwidth Windows is using interestingly there is no bandwidth used it depends on what you are trying to do. Could you answer that question? Yes, certain applications will actually be multiplied by the number of displays connected in terms of bandwidth usage. Is there a concept of developing your application to be Sage Native? Like one of the large scale visualization tools like visit or pair view where you could have the parallel visit server talking to the Sage cluster and not just be sharing your desktop from your laptop to Sage. Like does that concept exist? So we provide an API but it's up to you to port your application to do that partitioning of the data or the pixels. So we have a couple of examples. Natively Sage doesn't impose anything of that sort but we have explored both scenarios basically what Dylan did this summer we're trying to port five view applications being rendered on a cluster or just pushing pixel final images out of a simulation or something like that. Let me elaborate on that Dylan. So actually the pair view example is one that was occupying the 40 gig network. So what it did was Sage had a native app interface that would connect with a pair view web server that would then connect with a pair view server for data access and actual computation. So between the pair view and the pair view web servers they would do actual computation manipulation and then send it back to Sage through the app on the native resolution of whatever the user specified to view the results. And that one was example one that used large amounts of bandwidth because separate and distinct displays that need to be updated. And for the, yeah, that was the one with the high pixel revolutions. Okay so if you're doing everything in web and you're primarily really just letting multiple people screen share to one giant virtual screen it injects me if I'm doing the wrong thing. Do I have full support for things like DirectX or OpenGL or does that not even make sense because I'm really just sending images and that's happening on my laptop where I'm actually like rendering my app? So you can render whatever the browser can render. So the browser expose WebGL as a 3D graphics API that's one and otherwise what the browser can display are images like web formats specifically. So JPEG, PNG and GIF and then for video it's mostly MPEG4. So anything you can render to the browser can be supported in some way in Sage too. So 3D graphics is WebGL and otherwise images, PDF, movies and so on. Now as I understand it, Sage too also supports multi-location interaction. Is that correct? Yes. So can you describe that a little bit? So Sage too, so collaboration has been a big part of the effort in Sage too from the beginning. So as soon as you have a wall set up in one room you want to collaborate with people across campus and across a country or across continent and so on. So within Sage too we have support to share application and share assets between display wall of various size and resolution. So you can share documents, you can share movies and videos and PDF documents and on top of that we support sharing application where the application are synchronized between the different sites. So you can imagine the PDF document what you want to synchronize is a page number for instance. So when I flip to page three on my local document every site is going to see the page three of that same document. Okay, and so what we're really talking about here is two people with their own video walls and one person is driving the application that shows up on both walls. That's what we mean by collaboration, right? Yeah, and there is no master or no, not one person is driving the application. Once it's shared everybody has control of the application. Oh, so it really is like full blown screen sharing not just I need to use Google Hangouts because Google Hangout runs inside the browser and Google Hangouts has the ability for me to also give control. Sage actually has a little bit more going on. Yes, the application are synchronized through the server and there is server to server communication to ensure that we maintain the state of the application across this several walls. So you can synchronize right out of view point inside this really application. You can synchronize the page number or the time inside a movie player for instance. So does this have full blown video conferencing? I guess that means just start up Skype, start up Google Hangouts, start up whatever application you want and everybody will see everybody. Not really. That's not how we view it. It's always been fairly hard to support video conferencing or to trying to invest in developing video conferencing system. So basically we leverage whatever people want to use and whatever infrastructure they have locally. So when we work with Jason we use Skype or Hangout if people have life size or video conferencing unit they use that too. But at the end it's more often pushed to the wall through desktop sharing. So we do not enforce any video conferencing solution inside Sage2. Now that's interesting. You cited several different types of systems there. How do you handle the portability between those systems? So for example, if I'm rendering a movie to my wall locally and my browser knows how to render a movie type X but does that mean that the remote browser also has to know how to render movie type X or is there some kind of magic mojo translation that happens? The mojo is a standard that people... It's a W3C standard for browsers basically. So people have to agree that movies are mostly MPEG4 these days and WebGL is standardized and so on. So that's where we gain a lot using the browser as a middle way basically to do all the computing is that there's a lot of standards in that domain and so we can be sure that we can play, for instance, a movie MPEG4 in all the browsers. For communication, for desktop sharing for instance we use WebRTC as a standard to do the communication and so on. We want to do audio as a standard for the audio API and that's one of the main goals of using the web browser as a rendering engine is that there's a lot of standards and so you have a good chance that something runs here on my machine and across browsers and across systems. Do you guys have a video online that kind of demonstrates a lot of functionality of Sage and Sage 2? Yes, we have a series of videos on YouTube either on the VL site or Jason's Lava YouTube channels and we have a series of videos and most of them are published on our website sagecomments.org website. Okay, we'll include links to those in the show notes also so people can find those at rce-cast.com on the show notes to go along with this episode. Alright, so with the browser compatibility issues do you then have like minimum requirements for browsers because in the bad old days browsers were not all entirely compatible and I'm not really in the web area of things but I understand that things have gotten much, much more portable recently so are there minimum versions that you require or do you only work with Chrome or how does that go? We mostly use Chrome but we try to make sure everything works across all the system and indeed the browser has been getting much better these days so it's mostly HTML5 basically and the minimum requirement is to be able to run HTML5 with most of the standard. We don't track especially the version numbers of Chrome or Firefox or Microsoft Edge for instance but the core functionality of Sage 2 works across all the main browsers these days. Okay, so let's get into the scale of how Big Desync can actually go. What would you call the largest Sage deployment out there I'm guessing probably by the total number of tiles being driven or pick the metric of your choice? Okay, so I don't know exactly what's off hand but the thing behind us is 32 displays and that is for all of them are 4Ks so 3840 by 2160. It's a 256 megapixel wall driven by eight computers each display of the 32 displays is 4K in resolution. Jason has probably the highest number of pixels so over 200 million. At UIC we have a system with maybe more machines and less pixels or around 100 million pixels and 36 machines but more and more systems are driven by a single machine we're trying to push the definition of a Sage wall mostly around consumer technology meaning powerful gaming PCs and as many 4K screens you can plug into a single machine. That's what we see is the most deployable configuration to end users like scientists like a biologist or geoscientist and so on. Maintaining cluster is very hard and it's not in domain science-based tasks to do that or to allocate a grad student to maintain just a cluster so we are going mostly 4K TVs and gaming PCs. So if you're putting everything onto one PC and hooking as many displays as you can why even use Sage? Why not just open up your application there rather than share it up to there? It seems like you're introducing complexity so what's the benefit of having this machine in your lab with a bunch of displays hooked up to it but only having one? The benefit of that is the web server because the Sage deployment also includes the web server itself where you turn your system into inaccessible through the network and assuming that if you need to go through public IP or rather if you have if you have a public IP or public host name that routes to the computer then other people outside of your network is able to access it as well. Another way of also saying that is whereas a desktop computer or even if you've had a large wall running just native Windows you can indeed run many apps on it however the typical model is that you have one mouse and one keyboard that controls the whole thing. In Sage it's completely multi-user. In fact any number of people can simultaneously launch apps, drag and drop documents onto the wall simultaneously and they all have their own mice. You can actually see many little mice pointers running around and they can all move content at the same time. What we found was that we had to do that because as the windows grew bigger and bigger to cover the wall it was just impractical to use Windows because Windows was just not designed to support apps for extremely high resolution. You would open up dialogue boxes in Windows and it would appear on the far right-hand side of the screen, you click on it and another dialogue box would appear on the far left-hand side of the screen and the mouse is flying back and forth trying to deal with it. We completely redesigned our Sage user interface so when you click on the dialogue box the dialogue box actually pops near where your mouse is. All the apps have their own menus that are close to them so you can quickly access it as opposed to having your mouse back and forth across this 256 million pixel wall. Here's a question that we ask a lot of our guests. What is the strangest use of your project that you've heard of? Like somebody is using Sage in a way that you hadn't really anticipated or planned on but it works for them. Sage has a web browser aspect now that's fairly recent it's only been in since the last part of 2016. One of the things I've been doing with it is because I need to test cross-site usability I'll actually open a Sage display client within the Sage display on a different site so I can see what the other site is doing while I'm doing the remote session synchronization. Naturally it's supposed to be able to mirror mode but it's very rare to have a mirror mode active within a natural Sage display of a different site. Another thing I've seen done is that professors one predictor one had a Sage server running in his office and it wasn't high resolution but the reason he did that was because for his lectures rather than using the standard AV equipment in the room while he would use the projector he would open the browser, connect to his own server so that all the students would then be able to connect to his server back in his building rather than just have a static lecture where he would be stuck with screen sharing literally from the cord video output of his laptop. Otherwise as a funnier project I run Sage 2 on a Raspberry Pi at home just as a picture frame so I have my pictures being pulled off, something like Flickr and so on and I just rotate pictures maybe changing picture every 30 seconds and so on and in the lab we use basically Sage 2 as an information display so we show the weather outside we show the webcams and so on. I think Jason was showing the surf on the beach so as an information display it's a great utility. What you do is you have the Sage wall setup and then you can connect to it with your phone as long as you have internet connection so depending on what you're doing you have the Sage wall setup and then someone goes out in the field takes pictures and then sends them back to the wall especially depending on you're trying to find something like which beach you're trying to find the seals or something. So what's coming into future for Sage? What's coming down to Pipeline? So now that we've got this infrastructure up one of the things that we put up towards the end of last year was an app store you know people because they're starting to develop apps for it we like them to have the opportunity to share it with the Sage community. Sage is only as strong as the apps that can enter it and because everything now is in JavaScript it's become easier for people to contribute. We want to see initially most of the apps that people are going to be posting are essentially open source because you know this was an NSF funded project everything we distribute is open source. If the community decides that hey maybe we can start doing monetization and individual app developers might want to charge some money for their apps you know we're actually open to that possibility too. If it sort of encourages people to develop more apps for it. So you mentioned the commercialization aspect of it. Are you looking to do any commercialization of the core of Sage itself like deployments and support and things like that? Yeah we've actually licensed Sage already to a couple of companies. One of them is in Brazil I believe and the other one. I forget which one it is. The UIC technology manager knows the details of that. They are actually applying it to their own very specific niche markets. One of them is like monitoring of energy power plant basically operations. But yeah it's available for licensing certainly. We don't have the time because we're spending so much time developing Sage already. We don't have the time to say spin off a company to drive this. But it's encouraging that companies are coming to us now saying could we potentially license it to turn this into a product. So what you mentioned that this is open source do you have some kind of free for non-commercial usage kind of license or what license do you distribute it under? Yeah it's a license from the University of Illinois at Chicago. So it's basically free to use as long as you don't make money out of it. And then if you want to sell Sage to installation for instance you have to get a license. But otherwise it's open source. And the source code is on a Bitbucket site or something like GitHub. So it's open you can go and download the source code and send full request and bug reports and so on. Okay well Luke, Jason, Dylan thank you very much for your time. Where can people find more information about Sage and download it and get involved? So the main site is sagecomments.org and everything is available from there. Okay Thank you very much guys. Thank you. Okay All right