 All right, great. And I should introduce Supertapped Mary Medical College. I'm the director of bioinformatics and I'm a co-chair of the annual ECC. My favorite band is Nirvana. And I think I've cut into some time. So we'll get some presentations from Brian O'Connor from the road and then from Fred Tan from Carnegie, who's also the one of the senior project managers on handle. So why don't we just go ahead and cut to that. So, Brian, Brian, you. Yeah, actually, I think that that was a cover slide from the previous present previous session. So it's. Yes, yes, yes, please. Yeah, no worries. No worries. So yes, I'm very happy to tell you about outreach and training. It's a particular passion of us on the project. And I am excited to tell you about what we've been up to. As you heard earlier, there's a lot of sort of excitement around the anvil. And then there's a lot of questions around how do we get it into people's hands. So the next slide please. So a little outline. I'm going to talk about our mission and then I'm going to hand it over to our colleague, my colleague Tiffany from Broad Institute to talk about awareness and asynchronous support. And then I'll come back and tell you a little bit about some of the other support we're doing and the goals for our outreach group for the future. So next slide. So our mission is really to develop scalable training, support and incentives to make it easier for the scientific and community to adopt and use the anvil platform. I mean, the real incentive is to try to get people to use a cloud based tool that's really helpful for democratizing access, but can be a little intimidating when you're moving from using computers on your laptop to using computers on the cloud. Next slide. So the hope is if we build it, then people will come and use this tool and get excited about it and use it. But we know that there's going to be some challenges. We know that it's actually kind of not just that we can build it and that people will easily show up. Next slide. There are many amazing things you've heard about a lot of them earlier today. There's data sets you can't get anywhere else. There's computing power to do everything from R to Python to Galaxy. You can share analysis quite easily. And you can scale up analysis. I'm showing a graph here of Mike renting more computers than it makes me sweat just looking at that graph. And so those are all incredible aspects of it. Next slide. So yes, it's extremely exciting. We're super pumped up about it. But then there's some challenges too. So next slide. There's some barriers to adoption. So making people aware of the anvil and its capabilities, overcoming hurdles like nervousness about billing and security and really a new way of thinking when you're working on the cloud, training people to use the cloud, how to manage people on this infrastructure, how to manage data. It's a lot for people to learn. And then providing support to diverse communities because we think the real power of a tool like this is that its ability to democratize access to computing and genomics for a much larger cross-section of researchers around the U.S. and new researchers and students and everyone else. Next slide. So we don't want people thinking like this frustrated at their computer when they're trying to figure out how to use these new cloud computing tools. And so a lot of our work is to try to make it easy for people to use. Anvil is renting computers. It's sort of, you're out there trying to sort of select, you know, use computers in a way that's very different than what people usually do. They buy a computer and then they just use it with cloud computing. You rent a computer over a period of time. And so it's kind of a stressful thing for people who aren't used to that to start. Next slide. So our team for outreach so far has been consistent of two groups. A centralized group that does user research, content development, some funding support, things like that. I'm going to tell you about a lot of that too. And then a decentralized team that incorporates the training groups of Bioconductor Galaxy, the Broad Institute, and Terra to basically support a really broad cross-section of users who are trying to move on to this cloud computing platform and get excited about it. Next slide. So I'm going to now go a little bit to our approach and I'm going to hand it off here, I think on the next slide to Tiffany, who's going to take over to talk about a little bit of the first part of our activities and I'll hand it over to you, Tiffany. Great. Thanks so much, Jeff. Okay, so yeah, I will go ahead and start talking about the activities for training. So they are modeled after the Caesar acquisition funnel on the slide going from awareness to loyalty. And so I'll start with awareness and then asynchronous support and then I'm going to pass it on to Jeff to finish off. Next slide please. Okay, so awareness. So we need to make sure that people can find out about Anvil and the ecosystem. Next slide please. So there's a few approaches to doing this and one approach obviously is through blogs and pushing that content out to people. So this is just a quick table of blog posts that were specifically about Anvil and highlighting different features of Anvil from Terra.Bio. And basically the Terra team has this blog and it reaches a wide community of folks from the Terra communities and Anvil communities. And just wanted to provide some view numbers here too so you can kind of see what that looks like. And I'll just point out that the most popular one was about RStudio and within that blog post there was a demo of using RStudio with Bioconductor and Anvil powered by Terra. And then you can see also two other blog posts announcing Galaxy. The last blog post was actually a guest blog post from Dr. Martin Morgan about Bioconductor and the Anvil R package and why we need the Anvil R package. So they're great posts for announcing what's happening. Next slide please. Another awareness tool is using social media. So the Anvil has an active Twitter presence and the Twitter is used to announce training events and to make people aware of awesome work that's being done on the platform. Next slide. Okay, now I'm going to switch over to asynchronous support. Next slide please. Thank you. So now, you know, you make users aware of Anvil. They're getting started using Anvil. They have questions. And so to kind of react to how to support the ecosystem, the Anvil discourse page which you can see here is a forum was created to help, you know, get answers to users in a scalable way. So basically it's like other forums where, you know, users can connect with each other. They can see each other using Anvil. They can answer each other's questions. And the idea here is to help scale support with a small team. Next slide please. So there's also actually when you're using Anvil powered by Terra, there's also other channels that you can use too for help. There's an in-app widget that you can use to contact us module essentially where you can send questions about the workspace you're working on right there in a screenshot. And so there's also a forum available from the portal Anvil powered by Terra for very Terra-specific questions. And there's also email if people are shy and don't want to post their questions publicly. Next slide please. Okay. So we just talked about kind of how people can get support, but also asynchronous support means, you know, people want to onboard, you know, service. They don't want to have to necessarily talk to somebody to learn how to do something. So through providing persona-oriented training materials, that is a way to kind of minimize the noise among all the training materials and develop things that are specific for each persona. So we see PIs, analysts, consortia, teachers. Next slide please. And one really great example of a persona-driven pieces of documentation. You can see here are the budget justification templates and the Anvil cost estimator. So those are made available in the investigators section. And investigators can, you know, adapt these materials and actually people really like them and the NIH cloud platforms are interested in making these more accessible as well. And Jeff is going to actually talk more about the Anvil Getting Started Guide 1.0 and some other upcoming materials that are getting planned for self-service. So with that, I will, next slide, pass it on to Jeff. Awesome. Great. Thanks very much. Next slide. So we have been developing an educational technology platform for a while at Johns Hopkins in part supported by Anvil that basically allows us to create open source content in the same way that you do sort of agile software development and then publish it in multiple places on the web and through various different massive online open course platforms so that people can, we get the maximum impact for the course content that we develop for asynchronous learning. Next slide please. So we started off by developing an Anvil Getting Started Guide 1.0. It's available right now at this URL. It's also being integrated into the portal with collaboration with the portal team that basically helps people get started in all the hard stuff, you know, getting your billing account set up, how to work with workspaces and share data and things like that. So it's a really exciting resource that's really targeted to specific users that in their personas so that they can figure out how to get up to speed and get started as quickly as possible. Next slide please. So then we also have been providing synchronous support. Next slide. We've supported a large number of events. It's been a despite our relatively small like centralized team. We have a pretty big and distributed decentralized outreach group. And so we presented it a wide variety of conferences and events and provided synchronous sort of activities so that people can participate. Including the Magic Jamboree, which we hosted, which was really successful and had a number of consortium members participate. We have a bunch of events planned for the upcoming year as well. So we're really exciting to get really interactive with people. Next slide. We also have Anvil Office Hours, which we've initiated, which basically gives people an opportunity to show up and talk to the Anvil outreach chain directly, ask their questions, get them responded to with a bunch of experts on a Zoom call all at once. So it's a lot more quick and easy response if you're sort of a little nervous about asynchronous sort of support. So this is a great opportunity and we're highlighting it on help.anvilproject.org, which is the community that we've been building to try to get people excited about Anvil. And so you can, if you sort of go to that website, you'll be able to find out when we announce these Anvil office hours. Next slide. And we also give some tailored support to very specific communities in order to try to build bridges and sort of support a more democratic access to cloud computing. So one of those is through this AC2 program. The AC2 program is a cloud computing credits program that's been administered by the outreach team with some external reviewers who have provided a lot of assistance. And they selected a number of pilot projects that got cloud credits so that they could get onto the Anvil and not have to put their own credit card down and could actually start trying it out and doing some really cool science. These are some of the examples of the projects, but that program is able to sort of scale up and provide us more opportunities to get people onto the platform. Since billing is usually one of those sort of big barriers for people joining onto a cloud platform. This is a cool way to sort of move people onto the platform without having to worry about that sort of stress. And we plan to think of creative ways to support that going forward. Next slide. We also have been working with this genomic data science community network which you've heard a little bit about. This is something that we're really excited about. This is I think really illustrates in a real way the power of something like the Anvil. We have a number of colleagues and collaborators at institutions all over the U.S. We have outstanding scientists and biologists and computer scientists who are working with to develop curricula from them, you know, developed by them that we can share and support students as they grow genomic data science at a variety of institutions, not just the Hopkins and the Broads of the world. And so it's really exciting to see what's going on with that. Next slide, please. That has had its kickoff meeting in its first symposium and we're working on the second symposium now and it's been a super high energy event. So there's some people on call here today that are participating in that and it's just been super fun, just a really great group of people. Next slide. Oops, is it advancing for other people? Okay. And then we're doing user research. Next slide. So we're collecting data and analyzing the data about who's using the Anvil on Anvil. And so that's kind of a meta approach to user research. But it's giving us opportunities to discover what are the different ways that tools are being used and how to support them. Next slide. And we're also doing what we call deep pilots, which is working individually with individual researchers and educators to figure out what are their pain points as they try the platform out. What are the things that are stressing them out? What are the things that make it easy or hard for them to use the platform and collecting data on that qualitatively? Next slide. And so our goal in the future going forward is to build a big, happy, diverse, vibrant Anvil user community that helps each other out, supports each other through help.anvil.project.org. And also can use the platform to do a whole host of things that you couldn't do before. Next slide. Moving a little bit towards the future, we're trying to think of how do we actually develop this sort of community of people. We've built this incredible resource. It's still getting developed as it goes forward. So we're thinking about how do we scale up our asynchronous support? How do we scale up our synchronous deep training efforts? How do we continue to develop content and tools that can support platform changes? The platform changes over time. And so, you know, we need to be able to adapt our resources, our training resources to an adapting platform, which is obviously a big challenge. And so that's something we're thinking a lot about with our EdTech platform. And then build the Anvil community through GESN, through our outreaches to clinical collaborations and working groups, through Anvil discourse, and through bringing these different communities together, the Galaxy community, the bioconductor community, the Terra community. And then really one of the things that we're most excited about for the future is to really leverage our user research, the things we're identifying that are pain points with the platform, and really feed that back in to improving the process for people, the user experience for people as they adapt this platform. You know, it's an incredible tool. It has a lot of power to it. It's really an amazing accomplishment of what we've built. But if we can't make it really easy for users to use, to adopt and to use and to be supported on, then it will be a real challenge to get for people and people be frustrated as they get on. So we're really excited about trying to find ways to leverage the things that we're learning from our users to improve the platform. And with that, I think I will, I think we've done with our presentation. Looking forward to the discussion. Great. Thank you, Jeff and Tiffany. I think we're actually a little bit ahead of time. So just let me. Most of us were in the previous session so I think rules are fairly well outlined, but we'll spend about 10 minutes on each of the SWAT discussion topics and certainly we can, you know, mix and match as we need it. If you go ahead and just raise your hand, you know, as a primary and then everyone has the ability to commute as well. So I think that said, we will probably just get into, you know, 10 minutes to do strengths, 10 for weaknesses and so on. So I think we'll just open up for that unless Chris, there's anything else to do before we get into SWAT. No, we can maybe ask the discussants in particular to provide their thoughts. Great. And I apologize. Did we miss any discussants previously or some changes from from before? If so, please, yeah. Let me know that. Otherwise, we will go ahead and go. So why don't we start with strengths. And I think obviously there will be some overlap from the previous session because there's a lot of overlap going on with this, you know, with these different functions. But can we can we start with the strengths of and will proceed or experience from from this for you at risk of me. Good. Andrew, go ahead and then and then we'll get to Bill. Very quickly at risk of being repetitive. I was in a different group prior and one of the big things that's rather obvious you don't have to have big outlays of money. It's certainly easy to just use a cloud resource and the democratization you can reach the Midwest. The theme that came up in the last session was the silos on the coasts and a flyover country maybe that's overlooked and I'm going to can reach those. Awesome and by flyover country like you're being you for mystic but I am in St. Louis and yeah so that's me. We're going to talk about that quite a bit. Bill, go ahead please. I think that the final study correctly the deep pilot program is a real plus, I think, or we can get that intimate interaction with the users and get them involved directly in using the tool. So, let me ask a question about that is that, I don't know if I picked this up and was that primarily aimed at faculty students postdocs of what the deep pilots program. That's a good question so it's actually we're trying to hit a few different communities with that though so we're trying to hit the the researchers sure but we also have grad students who are actually using the platform day to day and trying to figure out what their pain points are. We sort of that persona model is sort of a way to think about it we've got grad students we want to talk to some PIs and how do they set up their lab and how do they sort of manage the costs and then educators how do we run this in a classroom and how do we make sure that it works well for them. So sort of thinking through that that deep pilot program is to collect information from all of those different communities try to figure out what are the pain points for each one. Yeah, I think that's the deep pilot for I think it's a really good idea. I think it has the potential to demystify. But while it's being demystified, it's also exposing its strengths to the people. So I think that's something should be exploded. One other question I didn't know what this meant. What do you mean by and bill discourse what is that. Yeah, absolutely. We can paste the link here help dot and will project or if you go there it's a community it's sort of a place where you can post questions and get answers. Okay, so one of the big challenges of the anvil is because it's Mike is actually the one that pointed this out. The anvil is you can think of it more like a computer. There's lots of apps on it right you can launch are you can launch galaxy you can launch Python. You can do things on Tara, each of those has their own support community each of them has their own sort of development of places where you can get answers to questions. We're trying to build a community around this website where people can go and they post their questions and then our outreach team often ends up like shuttling that question. And Fred posting questions to the to the Tara help support ticketing system for example. So we try to like be the, you know, to use the analogy where it looks all smooth on the surface you asked your question, regardless of what it's about you get an answer on help dot anvil project that organ and the where the duck with the feet underwater running really fast trying to figure out where that's supposed to go and getting you your answer back so it's sort of a, our goal is to build a one stop shop around all the different tools and get as we get more users as if they continue to join this. The hope is the galaxy group in the bio conductor group have done this incredibly well they've developed these communities where people will answer each other's questions you know we stop answering the questions because somebody else already knows the answer and they they post it so we're hoping to get there eventually. Thank you. John, please go ahead. Yeah, hi. Again, as I said earlier. Excuse me. I run a virtual data science training program over the spring and then Jeff and Mike they were ordered to support us and then I mean it was a very successful program. We had an average of over 70 participating each week, and then of course the onboard team exposed the participant to the various tools that they had and then another strand that I see is they also give all the participants credit to be able to use the resource I don't know if they did that to only as if they do you to know one but I think that is a great strength to expose students and researchers to to. Great and I want to just want to be sure like we parking lot that discussion about the credits because that'll also pretty much figure into the weaknesses. That's if you get it for free then that's a strength. Of course if you're paying for it then that will be a week and that's why I said if they did it for us, or they did for everyone. Yeah, we'll have a discussion of costs in later sections for sure. I did ask a question to you like so what you got what you worked on with with with Jeff and Mike is that part of how words. Yeah, yes. Yes. Can you tell us a little bit about that and and. Well, it was an eight week data science training. So in this year we're going to do it only for Howard students or RCMI or institutions with RCMI but we extended you to other ABC use and then once we advertise it we're getting. Yeah, so there were actually people coming from Europe from Africa for from everywhere so it was more like an overview of data science. We were writing some basics of statistics near machine learning, deep learning and then we had a special section for Andrew. That is where Jeff and Mike and his team the whole team were there for over two days so we're very appreciative of that. Yeah, I definitely want to talk about that some more and especially like maybe from Harry, you know, another, you know, historically black college where there's a need just to hold on to that thought we don't have to bring it up. Rob, your hand is up. Yeah, I was curious about two questions. First of all, what types of institutions are using and bill do we have data on that yet because as Andrew Lee said you know this the strength of an bill is supposedly in the ability of everyone to use it. And so my big question then is okay so who is actually using is it still those in the now we're really going to be firsthand. You know, linked to people who are involved in the humble project or are we starting to really see it starting to blossom outside of that. Galaxy for example I think that's quite well to catch on and I'm just curious to see how and bulls doing in comparison something like the galaxy project. And I was also interested in the persona right there that you have for different end users. I think one category that maybe we should add to that is retraining of the workforce so I'm a h and h era in particular keeps on talking about retraining in genomics and data science. And so maybe that should be another persona that we should be trying to actively bring on to the humble campus we call it to help aid in retraining the faculty for genomics and data science type use. I think one of the things we hope with the persona framework is that we can identify new personas I think the retraining one is a is a brilliant sort of suggestion and so the question is, how do we distinguish the retrain what what would we do different if we're retraining somebody versus if we're training somebody from from scratch and so I think that that's a cool idea and we have to think how to do it but I think it's a good one. And typically someone who's going to be retraining they're going to be probably more cautious they're probably going to be more stuck in certain ways and maybe I'm just talking about myself I don't know. But, you know, everyone knows okay if you imagine you were taken out of a current comfort zone and something has to do or be expected to start learning something else. What would be the impediment to learning that what would stop you learning that like, I keep on thinking I should learn twice and but I don't point. I have to stick with trying to spread everything out or he's in one of so what what's the what's the typical sort of impediments or challenges that researchers would have to learn a skill like that. You can then learn from that but to then how, how then we should think about maybe training modules, specifically targeting maybe older faculty or faculty maybe are not from genomics and areas. So, this is related I guess to the retraining but one of the things I'm seeing more and more in new faculty postings is the request especially teaching universities to be implementing things like cures where you're bringing research into the classroom. And I think that's one of the things that we've been talking about a lot as groups both in the GDS and internally is just, how can we bring this kind of research to those kinds of institutions. And I think one strategy is targeting the faculty because of the faculty don't actually know how to do the research, and it's hard for them to advise and mentor the students. So I think exactly, you know, in line with what you're saying Robert about the retraining I think is just getting people accustomed to speaking language to finding the collaborations and doing the research, you know on the platform like anvil. So, you know, I think that retraining in that, or even just the first time training, you know, is a very good point. Yeah, the physical hand up just to quickly add that one of the notes I was making in preparation for this was as much as we're talking about modules where students are going to get their hands wet and get in. I was struck to riff off Fred's point which is maybe we need a tour of somebody using anvil how do you go from recruiting patients to then collecting dated crunching numbers to seeing everything come up in the cloud because that way it would naturally, then maybe fit for retraining of faculty and also just as an introduction even to students about what does this actually do when you go from beginning to end. Yeah, that was actually very close to the common I just put in the documentation which is, even though the documentation looks really nice. But you're starting off with, you know how to create an account in a workspace before you've even shown somebody that sort of walk through of. Here's an example of analysis you can do that you don't have to sign up for you don't have to worry about cloud credits you don't have to use a push a button and somebody will say, Well, you know, here's the data set and here's how you get it here's the analysis I did and here's the buttons I pushed. I mean that sort of high level walk through before people have to invest in setting up an account and cloning workspaces and stuff. So the energy of activation to get into using anvil is still too high. Because it's still, it still requires sort of a trust that if I spend the time to register and get workspaces set up that it's going to be something I can use before there's any anything I saw. So there's a lot of work spaces that you would could be walked through but you had to first clone the workspace right so there's a lot of upfront stuff you still had to do I think that lowering that that energy of activation for understanding what anvil can do for you, like was just mentioned, I think would would be very, very useful and I think it's a weakness now that there's still sort of that high level of energy of activation. I also wondered, it's a question a comment. How users are finding anvil, like do you pull how people learn about anvil and can you learn from that. How to better, maybe target groups that you're wanting to reach but are missing. That whole user acquisition strategy, I'm just wondering if you could say a few more words on that. Yeah, I think maybe I can respond to a couple of those. So there's two or three things there. One is, I think your point is well taken around billing and activation energy. You know, this is a deep across deep pilots and we'll help that animal project across all these platforms. The thing that intimidates new users is billing they get, you know, like that's that's the scariest part of this whole thing for a lot of people. And so we've been thinking, you know, actively and one of the hopes for the future is really thinking through how do we support that ease of transition for people onto the platform so they can try something without feeling about billing and it's a tricky but it's a heavy lift to sort of create a fear free tier or something like that but it's something we would love to be able to do because you know just thinking back to the model that Tiffany showed and sort of user acquisition. You know a choke point for us is really around getting your credit card in there and stuff like that. There are examples as Mike link to one of different walkthroughs and we can show people features workspaces. Unfortunately at the moment if you want to start doing stuff on angle yourself, the very first step has to be you got to get your credit card in there. And we all know that that's a huge barrier to access and so that's a infrastructure problem that that outreach can only do so much to overcome but but we would, but we agree that that is a key barrier we need to get people over the hump. I would say that when galaxy was first coming out one of the reasons that I started using galaxy is because you go to the site and you could click on this five minute video and Anton basically walked you through here's how you do an RNA seek analysis like you didn't have to do anything. What I had to do is watch him create the workflow get the data in push the button and then watch the results come in, and just that simple five minute. Like here's why here's how it works. You know I immediately set up an account for that but I didn't have to go through any of that because there was this nice nugget of walkthrough about how to do a very specific analysis that met my needs. So I was able to jump right in I, I don't see that in the current and the book getting started book it's like introduction and then what does it cost. And I think that's that if there was something to there that was more. Look, here's, here's before you think about signing up or paying money. Here's some examples of what I can actually do for you start with that rather than start with the what does it cost. We've moved into the, the weaknesses and even threats, of course, but great sorrow. Yeah, please. So, since we're so I was going to say that since we were talking about strength, the annual documentation is very nice, but I agree to all of the people I had in my notes for that to that I think it would be really helpful if instead of that much documentation if we have short videos. So, like the previous speaker mentioned about galaxy. I would also like to mention us about a software that I use frequently which is IPA ingenuity pathway analysis. So if you, you don't need to subscribe or do anything to look at what it does. So if you go to their website they have multiple videos that you can go and learn, which gives you an idea of what you need it for and whether you need it. So which is what people would like to know before they register or do something else. And I think that would be very helpful. I think the point is well taken that we can throw a couple of we have several like really awesome video walkthroughs that we can put on the front page and put at the beginning of this book. The thing to really keep in mind though is that, you know, a difference between us and say galaxy is there is a galaxy main which is sort of a free cloud based service. And so once you've been excited about doing something you can immediately like press a button and start doing work for free that currently is impossible with with Anvil and would take an engineering change to sort of make that happen. And I think it's something we should absolutely explore and I'm with everybody on that I would love to see that but it's sort of something that's hard for us to do is the outreach team. So, so Jeff, that is why I mentioned about IP IP is a very expensive software like registration $12,000 $13,000 per year. But you can go to the website and see like multiple videos to find out what it actually does, and whether it would help you or not even before you do anything. So I was, that's the reason I mentioned. Just because I think this was being communicated in the zoom chat but the getting started guide I think hits at the people who have been convinced already that they want to start using the platform. And so it's trying to walk you through those step by steps. This is part of the larger outreach material. And if you go to the actual portal. We do have three videos that do highlight use cases highlighting on variant calling using GTK doing GWAS analysis and getting into what goes into eq to analysis. So, you know, I think one point that's well taken is that we are continually trying to find the proper way to organize this material so that it's easier for people to understand. But you know, these are definitely all great comments that we need to, you know, try and make sure that the message is getting across clearly so places that you have for suggestions on redesigns are very welcome there. And for it might be interesting to know like, who is asking these questions, you know something coming from sorrow, would you know, given you know that he's at your utap right quite smart. You know, just have yourself pass over it. That might be a very different question, or it might be a very different answer from you than you know someone who's from somewhere else. Levi sorry you've had your head up for a while, please. Levi, go ahead. Sorry about that. I just wanted to chime in a little bit more about the issues of cost and security which have been raised and only to point out that these are not just issues at like the individual investigator level that in my experience there are policies codified at the university level around those things that are that for me were were like really prohibitively discouraging to using cloud resources and I feel like I mean either my university is like worse than everywhere else for that which sometimes I felt like or we need to recognize that there are also institutional hurdles that that we need to think about how to overcome or else whatever is done at the at the individual investigator level is going to take a long time to percolate up to like central it's and purchasing departments. Which school are you at and the reason I ask is because it support is very different at very different places. Yeah, I'm at I'm at City University of New York CUNY. So it's a pretty big college I mean it has like 20 campuses and, you know, I don't know some hundreds of thousands of students but I can tell you that I was the first person to use a cloud servicing rebuilder. In my case, Onyx to pay for for Google cloud computing and to set up a strides account. And I know that because I had to broker negotiations between the office of legal counsel and Onyx in order to get a contract that both sides could agree on so that I could use a third party rebuilder, you know, it's, it was not it was easy and I also had to like fill out security review forms where I had to pretend that I knew like what kind of encryption and will used and what kind of authorization system it used and still in the end I think I got away with that part because I said well I'm using publicly available data I'm not using any, you know, data collected at the university. So, just just to say I mean I went through like a substantial amount of effort just to be able to pay for and analyze data on fun and bill that there's no way I would have done if I weren't funded to work on the project. And it'll be easier now for anyone else at CUNY who does it and I imagine at the larger universities. It's already been done but there's going to be a long tail of places where it's just, I mean I think that was basically their policies that are meant to actively discourage faculty from using cloud computing and just to chime in on Andrew, please comment in the text. I wonder if Levi's problem would be similar to the counter to other public institutions. Yes, and private institutions as well. It's not so. Yeah, just to chime in there that we had the same problem that Johns Hopkins frankly with some of these things which is sort of embarrassing to say. But one of the things we've thought about on our end and I'm curious what the group thinks about this is templated documents you know every everybody's going to have to do at least I did at some level if they want to use cloud I think unfortunately that's sort of like an entitis and other people, you know, lots of people mentioned this. Are there things that we could create security documents templates around what the anvil is you know like is there a packet that we could give to faculty that would make it easier for them to have broker those deals. You know I don't think we can eliminate that step for universities I just don't see that happening but I wonder if we can make it easier. Jeff, I mean briefly I'm sorry if there's a video lag or a text like I'm halfway across the planet, but this particular problem right now is actually something that's being acknowledged at the NIH wide level, and not just with anvil or not just with our institute but with NIH as a whole through the Office of Data Science Strategy. I mean imagine if Johns Hopkins or Spoonie or UC Davis is having a problem getting a credit card to use the cloud. Let's consider the smaller places that are part of the GTSC and let's consider the places we haven't even reached out to yet. So really I think part of the entropy that Carol and others mentioned is not just scientific but also administrative and here at NIH we are giving lots of thought to how to work with purchasing agents or administrative officers to to enable them to let the scientists or the researchers or the postdocs at their institutions use the cloud. So this is something we are taking very seriously it's still early days yet but there's a lot of conversations that NIH shaping up around this topic. We're going to categorize that one as an opportunity short job is that is that a. Frankly, you know, as the program person I'm going to insist that my categorization not be taken as the one but I see it as a weakness frankly and an opportunity but maybe a weakness and an opportunity at the same time. Definitely not a strength. Seems like there's a definite opportunity here for an bill to do something to help researchers at institutions negotiate these challenges and talking the chat is about boilerplate documents but I think maybe it will take more than that. I think if we're relying on documents going back and forth between legal there's going to be severe delays. And that will impact projects and maybe what we actually need is like a security officer is part of the ample program who can actually assist and help researchers with these sorts of issues at their institutions, with the immediate access to those elements of security protocols I'm sorry my researchers don't really know those things. So it seems like there's a definite opportunity there for a need for a position that to assist with research. I think there's always going to be a clash conflicts of interest between institutions wanting to keep hold of data for the idea that maybe they can monetize it. We see this being a HPC a lot. And the concept of open access depend on his pain for the data. Good point. And yeah I don't know how common that would put would be at a different landscape but in general that sounds like an issue that we all have. John, please go ahead. Yes, again, as I said around this data science training workshop, and then one of the feedback or comment that we got from the student of some of them felt they were left out that they were not advanced enough to participate in some of the sections. My question is this step by step, getting started with and where does it have prerequisite considering that some students or researchers may come from biology, computer science, mathematics or statistics. So, are they prerequisites for using on the special for the self service individuals. Mike I think that one's that one's for you prerequisites for and prerequisites to get to get to become a user. I mean I'm happy to do it. Yeah. So like the prerequisites in terms of the prerequisites at the university level or the prerequisites for like a individual student or faculty member. No, I would say for for student offering the one who wants to get started using. Yeah, so I think that there's you know there's two kinds of prerequisites right there sort of the sort of technical and bureaucratic prerequisites like at the moment you kind of have to have somebody who's going to set you up on the system with a credit card and billing account and all that kind of stuff. Assuming you have that the prerequisites in terms of the ability to use it. You know it's it's the three biggest tools that I think I see students immediately trying to adopt would be galaxy is a first point because it's sort of it tends to be a tool that gets used in early bioinformatics courses. Then are in Python depending on, you know, like most stats courses will kind of adopt an arm model and maybe computer science a little bit more Python sort of being like, you know, generalistic there but but there's sort of one of those three main tools and the main tools to be used. There are other tools that people can do they tend to be a little bit more advanced though and so one of the things we're working on with the GCN and other people is a collection of introductory examples that we can give to people. That focus on those tools so that people can try them out, but sort of we're thinking like the main target audiences would be sort of people in a by entry level sort of biology bioinformatics class or entry level stats class or an entry level computer science class could, you know, do that work on the does that answer the question I hope. Yes. I would also comment that a lot of this is sort of more of a cultural shift than a technology shift. You know a lot of people, you know, are really, you know, quite excellent and just as a sort of personal example I'm teaching undergraduate course right now in genomics. And I've been, you know, I've been making assignments where they write their own workflows and, you know, they've been incredibly successful at that it's, you know, maybe that's not for everyone but to me the bigger barrier is just the cultural shift, you know people are comfortable using their own laptops or their own computers, and I'm just, you know, under something new is a big barrier in itself. So it's, you know, it's, and it is a big shift and we need to help them to do so but it's, but there are ways to make a progress there that I think basically anyone could overcome. Anyone can overcome and I'm really, I was really pleased to see my undergraduates, you know, kind of figure out how to work through widows and work through workflows. It is, it is doable to do so. But it seems like, like kind of the user research could be done right on like testing out the onboarding materials with different people from different institutions with different backgrounds to see, you know, where people tripping up in, in the course at this point or does this need to be explained a little bit and be expounded upon because there's some assumption that we don't even see on how much knowledge somebody has coming in. I know one thing that we're doing for like anvil powered by Tara is we're sort of trying to replace our workshops that we that we usually do and take a lot of time to prepare for percent. We're trying to replace those with with courses with with MOOCs. And the idea would be that you could have these courses represent all of your one on one on one workshop basics. And, you know, usually that can take like a while to like present and go through. But if we have them already in courses, they can be, you know, reused by the actual people that are going to the workshops to like train other people because they can use the courses. And then it also helps scale the number of workshops we can do so we can go to, you know, actually build relationships with different, you know, groups and we can we can go to places and be build relationships and actually like, not only like teach but also leave back, you know, something that someone else can take and use going forward. So that's something that we're experimenting. We're going to be doing next year. And yeah, if anybody has any thoughts on on that or ways you think we should test the materials. I'll be thinking about that at least so. I saw Titus and Mark's hands both go up. Is this like, if you guys have a comment about what were the current discussion please go ahead and just unmute and go for it. If it's a new question then just hang on for one sec, but Titus mark current discussion or new question. Okay, great. Bill, go ahead and then sort of. Thank you. Thank you. I guess, as I've listened to the discussion seems like outreach is really taking two buckets. One is where you have people that already know about an bill and outreach has taken on the role of making it easier for those folks to get involved and get engaged and to know that they already see value in and they know about already. And another bucket of outreach is really to people who may only have nominal knowledge of what and it is and how it can be useful to their work of their research. And to combine that I really like the comments about reaching out to faculty because I was thinking, I work at Howard University. So, how, what does it take to get a sophomore biology student at Howard University interested in using and what does that process look like? What do you have to do to get to make that sophomore biology student Howard say I want to use and bill for my project for my thesis for my summer work. And I was thinking that the probably best pathway maybe through the faculty, for example, suppose you gave faculty instructional credits. Such that give them access to use and bill in their classes. And by giving them access to use and bill in their classes, then they're a sophomore biology student with one getting exposed to it. And then see some of the aspects that he or she could use. And then it would kind of open up a window of potential opportunity to get those students involved. Another aspect. Also, just because you're a PhD student or postdoc, a graduate student or postdoc, it doesn't mean that you know all about and know how to use and for example suppose you are a graduate student or postdoc in an experimental lab doing wet chemistry. But an aspect of your work could really be enhanced or benefited by some of the resources that Andrew can provide. How do you get that person knowledgeable enough about Andrew to say that's a tool I need to use in my work. So are there ways to reach out to the undergraduate level and also to the graduate and postdoc level. I know there are different pathways, but but to reach out to, for example, if there's a postdoc says there's a graduate student working with a PhD in in minor parts of biochemistry. That's traditionally a wet chemistry type work. So, so, so say the studying the studying mechanism diabetes. So, so is there a way that this student can understand that Anvil is a way of understanding biochemical or genetic basis of disease processes. And that would turn on the lights and maybe I can use it to understand the disease process that I'm studying. So those are two ways. I don't have real good suggestions of how to solve those. But I think solving those but broaden the footprint of access and interest and and bill. Those are amazing suggestions and I would say that our initial effort we didn't get to talk about it too much because we're sort of blazing through lots of things the outreach team is doing but the closest to what you're talking about that we're doing right now is this ESEN effort. And so there we're kind of getting faculty that are at these institutions and trying to build with them shareable open source curriculum. So say you're a faculty at Howard and you want to deliver this course to your, you know, teach your course about biochemistry, but you want to include a little bit of Anvil in there. If there's already a pre prepped lesson plan and some ankle credits and you can take it off the shelf and share it and you know then you tweak it how you like it and then share it back to the group and maybe a biochemistry professor somewhere else can use it so. So we're just on the beginning of building those curriculum we have our first example which we can probably pay the link to here on galaxy using galaxy like this but our goal is to build more curriculum like that. And then provide cloud credits again the building is always the place where it gets a little challenging but we have some cloud credit programs that we could definitely use to support that kind of educational effort so that's that's great. Lots of it is sorrow of Titus Mark and Levi actually wanted to thank Mike for sharing the document for his undergraduate course it was very helpful. As Mike knows it is more probably related to GDSM I teach both undergraduate and graduate bioinformatics and a lot of tools are used in those programs so I can easily incorporate and will into those and students can start using that it is meeting. If you ever want to discuss. Yeah, sure Mike I would, we need to do this offline. I mean we, I had shared my course document with you probably syllabus before when should you have introduced but then we can do this offline of this discussion. And really good participation are there any discussants who haven't had a chance to talk or write in the chat yet that that would take this opportunity. I hope we've got everybody. Titus go ahead please. Sid, as a as a strategy you need to let the silence become uncomfortable if you want people who haven't spoken to speak up. Well here we go 15 seconds. I'm just, I'm just kidding I'm just not kidding but but you know. So, so so we have lots of I have lots of experience in my group and my various efforts with the various modalities of cloud sign up we've been doing this for over a decade we've been through this with a Amazon and Google compute. We now go, we now go through a progression. For example, we start with binder which which offers a free our studio server, no sign up, literally click a link, boom, it's there. And that's for up to three hour workshops we use Amazon classes we use Amazon web services I am for like three hour ish things where we can create accounts and delete them easily and assume the billing responsibilities no credit card needed. SSH key is sometimes needed which can be a problem. And then we do we support group accounts and individual sign up for for more than three hour, you know, week long two week long and longer. And the thing is you want all of these working right because they legitimize prefer engagement and provide a pathway to while providing a pathway to increase engagement and the cool thing and I think this fits for Anvil as well transition between them isn't actually that hard right. You have our studio across all of them you have the command line across all of them. And so it's actually a great learning experience for the user they get, Oh, I can do something in our studio. And now I can log into an account and have a persistent our studio for a bit. Yay. And then, Oh, now I can like log out and log back in a week later and it's all still there. The challenge right now seems to be that Anvil doesn't support up many of those modes of engagement around new accounts and so on and that sounds like something that this group the training group wants would like to see, but that isn't supported on the infrastructure So I just want to make sure clear that that's both an opportunity and a threat. You can align technology you can align the training side with the technology side of Anvil across all these training modalities and potentially transform both and those user base and the common practice of data science. The threat is that if this doesn't happen and then if you don't have that alignment where you can do easy and cheap sign up then then that transformation may not occur. This also ties into this other thing that I think I threw into the both the threat and the opportunity side which is the sort of threat that Anvil becomes a dead end. What I mean by that is none of us want to be in a situation where we put a semester of training into a student, for example, and then they walk out and they can no longer use what we've trained them in this happens a lot with technology. So the fact that Anvil is built on a bunch of really standard technologies, right, like the workflow systems the data analysis systems are Python bio conductor, all is wonderful. I think a long term threat for Anvil would be to make sure, or a long term opportunity for Anvil would be to double down on that like we're doing everything with standard platforms as much as possible. Very little, only the necessary stuff around data use and privacy and so on is really specific to Anvil and Tara and Gen three and the other other platforms you're using. So I put all that into the document that that's my biggest sort of threat opportunity dichotomy for for Anvil at the moment. Great. And Tiffany is advising us. Yeah, we're so we're down to 10 minutes so we'll probably spend some time doing the summarization of the summation of this slide, but Mark and Levi please go ahead you have your hands up so. So my apologies if I missed this but I don't think I've heard anything about the NHGRI training grants during this discussion. And it seems like that the trainees on those students right or that the next generation of leading genomics researchers. And so I'm wondering if Anvil's done anything to try to reach out to enlist those trainees as annual users. You know, for example, I know they have an annual meeting is anything presented at that annual meeting to try to get them involved. Mark I'll take that one that is that that is a wonderful insightful comment I happen to be the person that is part of both the Anvil training and outreach group and a member of the extramural training team here at the genome Institute. So this Mark is the first year that we feel that Anvil is in a position to be put in the hands of trainees. So literally, and Teddy or Carolyn or others on the call please feel free to chime in but literally starting this year. We are in the process of integrating and bill into what could be used by our trainees you know for example at the trainee meeting next year. There will be presentations on the anvil there will be use cases. So you're making this comment right as we internally here said now I think we are ready to put anvil in the hands of the trainees so watch out by this time next year hopefully we have many trainees using the great glad to hear thank you. My pleasure Mark thank you. And it helps if I need to last comment from Levi and then we'll move on to for another spot. I just wanted to respond to Bill's comment about getting students using anvil and and say that thanks to this deep pilots program from the outreach team I'm doing that with my class this fall and they're not advanced computational students their epidemiology and biostatistics students, but they need to use our studio for for their analysis. And it's, I think it's been really successful I've had high uptake of it and really no complaints at all about it and I've had more and more students start using it as the semester goes on just because as soon as they have some problem on their laptop I say well use and bill then and you know it's it's, I think it's not that hard to do. And I'm working on trying to make an easy set of instructions for other, for other professors to follow. But the big thing is there just yet being some some money to put on a workspace so that all students have to do is sign up and give me the email address they used to sign up and then they're a member of the workspace and it tells them how to get started. So these are all really, really useful comments. I think we're, when we switch over to Chris if you can put up the supply. If you're ready for that and we'll just kind of decide as group. If we need to change anything. Great. Yes, just a second here. They've also been really helpful thoughts going into notes document so I was not able to pull those in yet. Is that sharing. Yes, I see your slides. eligible. Yes. It was. I just kind of run through some of what is captured and then we can see what's missing or, and if you know if we have something has lost in translation please anybody chime in and not to raise your hand just. So, you have the low activation energy relative to purchasing computing resources was one of the strengths, the ability to really get more geographic diversity of users. So, I would just add in the quality of just engaged participants, you know to see so many people coming back for different meetings and being engaged in all the other outreach events so supportive network supportive engaged participants. Should we qualify more about the credits for new users, because I was just thinking that's that this is referring to the program, the new program. Oh, provided by Google. Okay. Thanks. I have AC to do that comment weaknesses. The opposite here the high activation energy compared to using something you already have. The fact that there are so many different tools and capabilities that supporting them all becomes challenging. The cost the actual dollars is spending to spend and then the difficulty and actually being able to pay those dollars. It's the lack of a free usage here you need to create an account the lack of video documentation. The difficulty and platform integration. Like a lower slope entry point. And like an advertising awareness gdsc and was called out. Chris, I would, I would. Oh, I'm sorry Robert, go ahead. I'm just going to add in one point and that is the, the, the fear that it might be a dead end utility. You know, if say NIH decided in five years it didn't want support Anvil. People would then be very worried that they've invested all this time and energy into switch and go over to using the the Anvil ecosystem. And I think that's the long term support plan is by NIH so that needs to be better communicated as well. So the thing that I wanted to add is, I think there's there's, I wanted to expand on the lack of free tier. One is credit cards specifically are really big barrier it's not just creating an account it's credit cards, and the other is the need for security review. It's usually tied to putting in a credit and institutional credit card number. Creating accounts everybody's happy to make a throwaway account on a new service, I mean, approximately. It's the, yeah. Okay. The opportunities. I heard a lot about retraining faculty and data science and Anvil potentially being a place for that. And I think that's what I'm trying to emphasize in their specific perspectives, just that everyone's encountering these barriers to using the cloud and maybe Anvil can help provide some materials that are standard, and just awareness, normalizing all this opportunity to kind of conduct research on how people are discovering and using Anvil opportunity to normalize the use of the cloud for the next generation of researchers to reach faculty and provide resources to really integrate. And especially undergraduate coursework. Some discussion of like Titus's comment of different scales for signing up user engagement course documentation. Looks like we are about to exit this group. There's retraining opportunities for retraining current workforces is really was really strong in my view, my memory. And correct you guys are about to leave here. So any last words you want to wrap up. No, I think this is, this is awesome. We'll see you guys back in the main session and this is really helpful. I look forward to hearing from this group again in the future. Thanks everyone for the comments in the Google doc you can keep them coming briefly after this. Thanks everyone. Thank you.