 So welcome everyone. This is a talk from the University of Oklahoma Libraries on a pilot we're running on leveraging the National Research Platform to build a scalable research and education environment. My name is Tyler Pearson. I'm the Director of Informatics at the Library and my colleague. I'm Mark Loverswine, I'm the Research Data Specialist. So I want to take a quick moment and just recognize the National Research Platform and their partners along with the Great Plains Network and their former Executive Director James Deaton that without them we wouldn't be doing this presentation today. For those of you that are not familiar with the National Research Platform, it is a partnership of more than 50 institutions led by the researchers at UC San Diego University of Nebraska-Lincoln and UC Berkeley and includes the National Science Foundation Department of Energy and multiple research universities in the U.S. and around the world. And additionally the Great Plains Network is a nonprofit consortium advocating research on behalf of universities and community innovators across the Midwest and the Great Plains who seek collaboration cyber infrastructure and support for big data and big ideas. So the Nautilus project kicked off right around, I believe it was 2015 and to date they've had over six grants tolling over $27 million. From a technical perspective, Nautilus is a hypercluster for running containerized applications utilizing Kubernetes for managing and scaling applications. The partners also provide the hardware behind what's running Kubernetes. Additionally, a benefit of Nautilus that we've utilized is their community. They're shared resources both in compute resources but also documentation that's available through GitLab. There's an online chat that we've utilized quite a bit both providing feedback to the National Research Platform but also to troubleshoot when we run into problems. Additionally, there's a lot of workshops that are held for building some of those skill sets. So for those of you that are not familiar with Kubernetes, it's an open source system originally developed by Google for orchestrating the life cycle of containerized applications. And one of the things, well, let me go back. So I want to emphasize that we are not experts in Kubernetes. We are just users of this platform and the National Research Platform and their system administrators make it much more easier for us to deploy on and we approach this from a perspective of being users. And specifically for our pilot, we've deployed a Jupyter Hub instance and another great thing about the community is there is a configuration where we only had a right just under a couple of hundred lines of configuration as opposed to a thousand or multiple thousands of lines of YAML if we were to want to deploy this ourselves. I'm going to hand it over to my colleague to talk a little bit more about why, what led us up to pursuing using Jupyter Hub within Nautilus. So for us here at the University Libraries, we offer a lot of workshops for researcher tools around the things that will make things like data management, better practices around code development and those elements much easier. The problem is that a lot of these software need to be installed locally and depending upon the type of machine and how that machine is being managed, not all faculty have brute administration rights or they have to go through a software process to get their software approved. Also we run into the fact that there is issues around the variety of types of hardware, right? Faculty are famous for driving their computers till the last bit of magic smoke comes out. So we have various ages of equipment. Also research complaints had been a lack of a common coding environment. So when a new graduate student comes in, they load the current version of Python and all the supportive libraries, then two years later a new graduate student comes in, comes in with a new machine, they get all the latest stuff in the meantime. The old machine may not necessarily have been keeping up with their patches and updates and so what you start to see is this coding environment in a research group that might have most recent version of Python but be two versions behind in their specific libraries and so now the question around reproducibility starts to come in and have a big play because suddenly code that runs on a machine that's running in a library that's two versions behind and now that same software doesn't run on a current version, right? It turns out they've deprecated a command. These become an issue for us and faculty then are starting to lose time. So one of the things, as we pointed out, varying user environments, various policies, inconsistence of versioning, conflicting software, not all the software when it comes on the machine will work with that particular version of the OS or it may require additional packages that will affect the OS. These are all things that faculty do not want to have to be thinking about when they're doing their research and especially for any faculty they're also wanting to teach these technology tools in a classroom environment. It runs into a huge waste of time and this common framework that we're going to be offering up and discussing in a second addresses those issues and of course the whole reproducibility challenges around research that's starting to become more and more important can be addressed with this. Hardware, especially in the student environment as learners, they're going to come in with instruments that aren't going to necessarily allow for the installation of some of these tools. Legacy laptops, you know, hey, you know, older brother just graduated, guess who got the hand-me-down machine that's now four years old. So that type of instance, recent architecture change around the arm chip with Apple and then with researchers working with various types of GPUs as AI machine learning starts to become more prevalent in a lot of research domains, there are a lot of choices around GPU cards and so this method will also allow for us to take care of some of that. What we did was we started working with an initial installation and as Tyler alluded, we did not have to be Kubernetes experts. In fact, we didn't even have to be Jupyter Hub experts. We built upon the community. We took examples of running infrastructures that were already out there. These are all shared environments. We tailored it to our particular needs and working with our central IT group in terms of supporting resources such as a DNS and everything else and within two months we had a working piloting type of operation to go. We presented our local academic tech expo in that January and we got an interested faculty member who wanted to use it for teaching. We then since expanded out to a full semester's worth of a class in a learning mode as well as working with one or two researchers on research projects. We finally decided that we needed to do metrics and that's something that this system is not necessarily designed to do. So we had to kind of learn to do that and we were more than willing to share that. And now we're on the end of our first year moving into our second year. So we'd like to discuss a little bit of the speed bumps and how we're going to move forward with the next phase of this project. So the project timeline in specifics to show you just how fast we got this up and running from our resources. It was December of 2021 that we started working with this. We presented at the end of January the expo by February. We had a faculty member wanting us to run for a meteorology course for upperclassmen. It was a professional meteorology course that had a segment on programming and he wanted to use this for their programming session. We then offered our first workshop, which was our primary goal was to facilitate working with workshops. We teach software carpentry and data carpentry, which has in software installed. And we went what we would find is that that initial hour of the workshop where we ended up having to solve that issue of everyone's install went away. Our biggest problem was getting people logged onto the system. And that and that was, you know, trivial in terms of time and effort on that. We then expanded the test with an R.E.U. research experience for undergrads. And this is a program where students from around the country come into the country. They are paired with a mentor. In this case, it was meteorology. They work with a meteorologist for 10 weeks on a small scaled research project learning about research as undergrads to decide if they want to go on and do a career in graduate studies. This platform, because it does offer persistent storage and continuation, it's not like some of the things that you would maybe pay money to fire up into the cloud and have it go run. But then at the end of the workshop, you have to set it down. These things can continue to run. And so we did have undergraduates who continued working with their mentors and who then presented at the American Meteorological Student Conference in the annual meeting. So the work continues. But what this did, it was gave the researcher a common framework so that they were working on the same versions of the Python with the meteorological software. With their data, they could share code back and forth and have this common framework so that they didn't have to worry about if the student had resources once they got back to their home institution. The faculty member didn't have to worry about that they might be too far advanced in terms of versions of code. And that went very, very well. We then expanded to our first full semester under just a single class. It was an introductory programming course in meteorology. And then that was very well. It went very well to where we expanded at the next semester for that same course to both sections. So we went from about 20 users in a learning environment to almost 50 users in our learning environment. And that's where we stand today in terms of our pilot and who's been working with this, along with some of our researchers. I don't know how big this is showing up, but this is our initial login to the Jupyter. So it was based on a federated login called CI login. That's another advantage because as a faculty member teaching a course, you don't have to create new accounts on the system. If the student has a valid student ID, it authenticates off the federated login of CI login. And what they'll have is their choice of these pods or containers. These are Docker containers that are basically designed to bring in a working environment. We know a little bit about Docker containers, but that's one of the services of the library we can offer our researchers or our instructors is that we can help them create these if they wish to know them or they can use the ones that we set up. We just need to know what their software requirements are for their particular course so we can build them in. So we have kind of a generic science one in sci-pi. That's the standard Python install with most of the libraries. Meteorology, as I mentioned, has some very specific libraries associated with weather. And so we put those in into its own container. And so that is allowed to run. We've got one for Consulate. That was our research project. We had a faculty member who was doing research on text analysis. And so one of the things that we did was we made sure that this Python version had all the natural language toolkit tools in addition to a library that was been created by JSTOR called Consulate that also facilitates. And they have a very nice tutorial section so that if a faculty member is wanting to learn more about natural language, we can we have it so that those those workbooks are in. Then we have one for TensorFlow AI machine learning that works with the TensorFlow. And then we've got the R in RStudio. So not only does this run a standard notebook stack that you would expect in Python or Jupyter, it also can run applications. And we have a pod that will run either an R as a notebook or RStudio as an integrated developer environment that's pretty typical for our users. And then we have the Kitchen Sync one for Data Science just to show that can be done where it has all the programming languages that are typically used. And that's a big container and takes a little while to run on that. From where they select the container that they want, they are given the typical Jupyter. This is the one for R in RStudio. You can see along the top they can they can instigate or start a session using RStudio or R. They can also access the console of those tools along with some editors and then the terminal or the command line. And as much as students may not like that, there are still certain practices and research such as version control that work very, very well in the command line interface. So we have that ability to connect a terminal window to any of these sessions that they fire up. This is an example then that of an RStudio lesson that was brought up where you've got the R code in the typical code window. You have a figures plot window. You have access to the file system. So any of you who are familiar with working with RStudio, this is a typical RStudio deployment. This is an application that's running the R framework. And the other nice thing we talk about the common framework is that all the libraries that are available in the RStudio are the same libraries that are available in the R notebook, right? So everything is saying as they work through this. This is an example of another application where we can run OpenRefine as an application. So OpenRefine is a great tool for cleaning data and working of that. It's a standalone application that actually presents through the browser when you run it locally. But here it's also presenting through the browser, but it's just being managed on this Jupiter cluster that we're running on this. Here's a typical meteorology. So we talk about meteorology. This particular library has the ability to do the graphics and visualizations that meteorologists are used to using. This is a four panel multi-level, multi-variable type of state. I can tell you this will help with forecasting. I'm a meteorologist by training, so this is great that I'm able to do this all with the bringing in of one one library and gives me the access and control to be able to work with these. This is an example of one of the things that we think is an advantage to this is that especially with we have a very active open educational resource community within our university. But one of the things that you find a lot of times is that while you can bring together a lot of text to create for a great classroom, there is not much in the way of homework problem sets or tutorials. This is an example of a tutorial that I built to showcase the idea. This is a blog on R using R for linear regression and whether or not it's a single variable or multi-variant type of model. And so the layout is such that it's a stand-alone with hyperlinks going out to additional resources with snippets of code that then highlight the topic that you're trying to do. The example on the right is to do is my data following the assumptions that are needed and this is showing them doing a histogram fit of the data to show that you've got kind of this bi-modal or modal single mode whether or not it's going to be multi-variant or not. So this is an example of some of the metrics that we started gathering and we we put together some tooling right around November of last year and started capturing active sessions and we can see as this progresses through the semester that the the the lines on the left are that first initial meteorology class and we could see that at the peak we had 25 active sessions and as we went through the winter break the next set of peaks there is actually a software carpentry session that we had and then we added the additional meteorology classes and this auto-scaled for us once we had deployed this we didn't have to do anything extra to add these additional users on. I will say that we for our instance we aren't limiting those who can log in but the configurations are there that we could enforce that only those approved users are on here but since we've opened this up our max usage prior to this conference we had 38 active users using this and we can see this slide upward trend in our usage. So since we started grabbing our metrics we've had over 2,500 unique sessions by 335 users and on average so the median session length for each of these sessions is just under two hours and we can see in this table here what our popular pods are are our docker containers and it makes sense we've we've onboarded three meteorology classes that this would be our most popular pod and these others are probably tied into our software carpentry especially the RStudio the SciPy which which houses our Python environment so just prior to here this workshop I was a guest lecturer for one of those meter up both meteorology sections I was teaching Git and GitHub as part of the version control this just shows you one of the monitoring tools that we can see all the login sessions that are occurring you can see mine I didn't fuzz out for you but of this all but about three or four of the connections were students in the class and you can see the utilization that was going on at this time. One of the things I did not mention but that is also makes this really important is the fact that the storage that is assigned to your login that's associated with the login is what we call persistent storage so it stays there a lot of times when you're dealing with systems like this in the cloud you get storage for the time that the cloud is active but then as soon as you log off you lose all access to that this storage stays and is attached to the next time you log in with that username and we do have the capability to grow that we limited right now to about five gigabytes because we want to be good community people we don't want to be carving out a ton of storage that we may or may not be using but it also gives us an opportunity to talk to the students about what makes for good file management on computing systems I don't know if you have ever worked with students and you ask well let's go to your project folder and you go right to the downloads directory on their laptop and then the downloads directory you see eight downloads at the same data set right so you give you an opportunity as an instructor to start bringing in good data management practices and things like that the key point is is that the students will have access what we're really excited to do about this is to follow this cohort as it starts to migrate through the meteorology program this is an introductory course for late freshman early sophomores this account will be available to them through their entire career through our program so when they run into programming exercises or classes that will require them to do programming they will still have this environment so anything that they built in previous classes can then still be available to them within this information system through the through the kubernetes this just just to show the various institutions we've not totally locked down our access because we've been wanting to promote it but also because of the fact that our reu student program are coming from other universities and this is just some of the major ones but if you're anything in the meteorology you know that wichita state valpo for valparaiso um um v vermont um our meteorology places and so these are places that we had students come from and work with our mentors but this also from a researcher standpoint shows you that we have the capability to allow for researchers across institutions to log into this environment and do their work together in a collaborative way again working in the same framework with everyone's operating in the same version of python or var and all the libraries are up to date as well i'll add on to that that uh looking at software carmetry another uh struggle we had is when we kick off a session so prior to this we we had the struggle of installing software and helping learners install this and we wanted to minimize uh their barrier of entry to this so we we left it open so they just needed to know what the url was to get into our environment log into it and they would have access to um to that environment so lowering that that barrier of entry so this is a work in progress and we are still learning but we we are leveraging the community to um to lessen some of that that burden on us uh another thing i want to mention is during our pilot uh we have not purchased any additional hardware all of this is hardware that uh the partners have have shared with us and that has both pros and cons so we don't own that hardware so we've we've learned some ways of troubleshooting when something stops working um and another thing we've done is is reached out through the chat software and provided feedback and we've we've also provided areas where we discovered signals of well this note is down and shared that back with with the Nautilus folks and contributing to make their system even more robust so the other way is that we can help in working with the researcher who used our AI module is now interested in using it in his own research he's already gotten the vpr to purchase him some additional storage so our job this summer will be to add that storage to the kubernetes system so that he can access this he will be living it to his own research group which is one of the other things that you can do within this Nautilus system you can carve out a little bit for yourself but because he believes in this and he likes the fact that he's going to be using other people's GPUs he wants to share some of that so a percentage about I think you said about maybe 15 20 percent of the storage he wants to put out into the general pool to share with the community so the idea then again is this in this distributive system is that every institution has a way to maybe just contribute a little bit University of Oklahoma does this through a Fiona node that's associated with networking work that we do out of the high performance computing center that's contributing to the nodes for that so the idea then is that you know as a shared resource you can contribute what you can you use what you want just be a good user and be smart about it we have been sharing out this system that we've been using and it's worked out pretty pretty good for us up to this point so we're finishing up our first phase of our pilot and we're going to start writing up some some both internal and external documentation and look at kicking off our second phase where we're going to be looking more at sustainability and building partnerships on our campus and again strengthen our partnerships with the community scaling in terms of library support we don't necessarily want to be managing the day today or when the network starts to go bad and people are having problems connecting that's that we just don't have the time and the resources to do that that would actually probably be a better job for our central IT to maybe weigh in on we would like to be working with the faculty members on on container development and library management for whatever they're trying to do within the common framework on this but we want to expand beyond just meteorology I'm excited that we have a potential to have a digital humanities course it is what I refer to as my programming classicist he's a latin professor who self taught in python digital humanities he's teaching a digital humanities course this fall and he's looking at using this framework he is a good example of where a lot of the students are coming in on Chromebooks so this is this is one of these things where I think the library is starting to also realize about this equity of computing not every group on campus that you may support from a research or training an educational standpoint has access to high performance type of computing or just even regular type of computing that would allow for you to run some of these tools this now starts to leverage that in level of the plain field so to speak it still requires a network connection it doesn't have to be broadband it just has to be stable that's the only gotcha in this and that stability goes along for the rest of us yeah not a less framework for researchers we just successfully got our first AI running with GPUs so again we'd like to see some of our containers be focused more on the needs of researchers and we were very concerned about making sure that that we have local partners that both can help us with some of the work but also helping to promote this and work with us on that I think we left a few minutes for questions maybe not we're done put a fork in it let's go ahead yeah there is some ways that you can have shared storage that the people who log in will have that common shared storage within their accounts allows for that type of collaboration so any code that you might have and that's in that shared space as it stands as when a person logs in that's private so that does not get shared out but there are configurations that do allow for that sharing and creating those environments we have just not learned yet how to do that for ourselves I know there are some groups that not all this has helped we'll be going to get lab repository and stealing their YAML files I mean that's really what it comes down to because everyone in this community shares their stuff it's it's not that hard to reconstruct we have the advantage I'm going to take care of the expert bias I'm a former systems operator and installing software and things like that Tyler is an informaticist and is very much familiar with working with APIs but YAML especially so we do have a little bit of an advantage there but our resources were available we partnered with the libraries at the University of Kansas where they were teaching a library course on text recognition using tesseract using tesseract and that is a real beast to install if you're familiar with that coming from the libraries and this allowed for a workshop of 25 librarians from 25 different institutions to all work in the same common environment and have that workshop be successful we're working on that it's there's been starts and stops and whatever as a librarian within our own offering of researcher support services digital humanities is always on our mind I may be classically trained in meteorology but I was a music not want to be and my favorite courses from college weren't my science classes but my humanities classes so there's an affinity for us just as individuals to not forget humanities on this but to me and I'm very much of the ilk that you raised the tide of the whole ship and so these are resources and as the digital humanists start to realize some of these tools have some purpose with what they do there's groups on campus that have been using these tools I'm hoping to start to foster that type of collaboration so that people can learn from each other on that but for us we do have a digital humanities tilt into the library we do have a section that does that it's we're hiring I'll put out that much so it's always been part of this I mean our researcher services is about the technologies and tools not the domains because these tools really do cross numerous numerous of these of these domains go ahead yeah we've we've thought about a couple of ideas and our first go around was looking at cloud services and we dropped that pretty quickly but that sounds very interesting open grid we weren't we were concerned a little bit about the the NSF thing on science and we knew we wanted to be able to expand this to the digital humanities and so we weren't clear if we would be able to get access to the allocations for that working with this in this framework because we were approaching it from the instructional standpoint around a common framework it becomes a little bit domain agnostic and we're not worried about it if it's being used for science versus everything else on that in terms of some of the stuff that we've done with the supercomputing around digital humanities it's data and data doesn't have boundaries either so we kind of get around that NSF requirement that it's moving science forward and that's the issue really with the with the science grid yeah yeah yeah yeah I think that's uh time any maybe one more question nope all right thank you thank you