 Myself, I'm Nate Angel, I work here at Hypothesis, which as you know, is the organization dedicated to stewarding and developing annotation technologies and practices, or something like that. That's our tagline. So these slides are available at that bit.ly link that you see down there in the corner. In case you want to get to them, there's not really much here though because this today is really all about a conversation, as opposed to any kind of presentation. And so one of the things is we actually would really like to involve you all in the conversation, but we're in this webinar mode that Zoom offers. And it doesn't really allow for as much participation as we might want. And so one of the things that we would really like to encourage people to do because you can't, you know, as an attendee, you can't see who else is here. If you feel comfortable making your presence known, go ahead and use the chat to introduce yourself and say a little about it. And please feel free to keep a lively discussion going in the chat as we go along. Note that in Zoom, you have to make sure who you're chatting with. And so there's that little pull down menu to pull it down to make sure that you're chatting with all panelists and attendees. Otherwise, it will just go to the panelists up here and, you know, you don't want to talk to just us probably. I mean, if you do, that's fine, but definitely feel free to take it out a little bit wider. And then we're going to use, you know, Zoom's usual tools for communication like the chat and the Q&A if you come up with questions that you might want to pose to the panelists. And we also can turn on people's mics and cameras. Like if you come have a question and you really like to come on stage if it, you know, as it were and say that out loud and be on camera, we'd happily invite you up when we get to the Q&A part. I really like how my colleague Jeremy Dean talked about these features of Zoom as themselves kind of forms of annotation, kind of annotation on top of the meeting itself, right? And so chat is sort of like the informal annotation that you might do kind of the little asides back channeling kind of stuff and Q&A is the formal. I'm actually submitting this annotation to the teacher kind of thing. So I think that's a nice way to think about these tools in the context of annotation. So I also like to announce that you're at a very special meeting today because we decided at Hypothesis we just have so many interesting things happening in our community that we feel that we need to do a better job of getting them in front of people more quickly. And so we've decided that we're going to try to do a weekly show about all things annotation, if you will, and the social learning around that and all sorts of different topics that come up in our community. And so we were just chatting about this before we hit broadcast here with the panelists and we coming up with a name for something is one of the hardest things that you can do. And so we went through a long process of trying to figure out what to call this show. And here's a few of the names that didn't quite make the grade. I was, I was pretty, pretty psyched for a couple of them. I, when margins last on the door yard bloom, I thought was a pretty good title, or maybe even puff the magic margin. I'm actually decided that I'm pretty much going to be known now as the sheriff of Nottingham, which isn't really a great show name, but I thought it would be a cool title. But anyway, none of these, none of these names for the show made the cut. And so we ended up with Tada. You are at episode one of liquid margins, and that is the name of our show. It's going to be every Friday at 9am Pacific time. You might have some special shows at other times or something, but that's, that's the plan. And so welcome to being a part of the inaugural group that kicked off this liquid margins experience. I would then also like to invite you to perhaps come and participate in episode two, which is going to be next Friday, and we're going to focus in on annotation in the composition classroom. And you, there's already an RSVP link, which we will also share in the chat for you. This will be an every week kind of thing you can come live, or you'll be able to of course, come view recordings, either audio or video after the fact, if you can't make it, and please show that out as widely as you want with with other colleagues or folks. So that's really all the housekeeping kind of stuff I had today. And I wanted to take a moment just to introduce who's who's all here kind of on stage, if you will. And so in addition to the actual guests that we've gathered to talk about piloting today, I've got a couple of my colleagues here from hypothesis. And so I wanted to give them a chance to introduce themselves. And so I'll start with Franny who's going to actually serve in the role of the producer of liquid margins. And so Franny, why don't you tell folks where you're where you're from and why you're here today. Hi there everyone. First welcome to our guest and welcome to the many people here in our, I guess I'll call her our audience but sounds too formal, but thank you all for being here. I just started with hypothesis a couple of months ago I'm on the marketing team. I have a background in education and I was an English major undergrad and I got a master's in English so you know I can fly across the country for like $800. So that's good. So I got a lot of benefit from that. I live in Portland, Oregon. And yeah, that's it. I'm so, so happy to have Franny here with us. And then also, Holla Smith Hughes, I think I have pronounced her last name right for the first time in my life is also part of our team. And Holla does a lot of work at hypothesis on but she'll be focusing in a lot of the post production work for the show for liquid margins so Holla, tell us about yourself. My name is Holla I live in Portland as well with Franny and Nate we actually all happen to live in Portland even though the rest of the team is kind of all over the place. I have a background in graphic design and social media strategy I've been a freelance social media strategist for about 11 years. I have two little boys. And I also if you see behind me, I am a textile designer and I design blankets that are inspired by places I've traveled so that's what this office is kind of a shared office for my work with hypothesis and my other business. Yeah, and I, it always looks a little bit like maybe Holla's calling in from like the laundry room where all her socks or something. So it's good to have a little explanation. Also we're the little team that's going to be bringing you liquid margins from now on me the sheriff of Nottingham and the other musketeers here. So, thank you, thank you all for helping to make this possible so now without further ado, let's turn to the actual topic and conversation of today's show and if you, you might have seen the memo. Some of you who registered early might have been expecting a half hour long show, because we wanted to keep this kind of brief and casual, but we actually realize we weren't going to be able to do it in half an hour so we extended the show to 45 minutes if you want to take off early. Go for it don't worry about that that's great. You'll be able to, you'll will email you when the recording's ready you'll be able to come back and be the rest of stuff. So, I'm really really pleased and honored to have the three folks here today that we have as guests. And I won't, I won't throw the slide back up with their names on it because you see all their names there and soon. But Aaron Barker joined hypothesis fairly recently and her title doesn't really describe very well what I think that she's, she's here to do. And she is really leading the work that we do with people at schools and different kinds of colleges and universities and some k12 institutions as well to help them be successful with annotation collaborative annotation and hypothesis getting it up and running, kind of on on both a pilot and in an ongoing way. And so she is really at the center of that work. And so, Aaron, do you want to come in and tell us a little bit about yourself. Yeah, thanks Nate. I'm Aaron and I'm the customer success specialist for hypothesis and as such, I am in charge of all of your pilots and I'm really excited to see so many of my partners attending today show. If I didn't give you a shout out or I haven't given you one yet the know that I do see you and I'm excited that you're here. And I actually come at you from Carbondale, Colorado on the Western slope and I'm a former Portland resident so I don't know what it is about Portland for all the attendees on this show. We must just have like be connected in a previous life or something so I'm very excited to be here and to talk about piloting today. I'm glad to have you not only here today but on the team air near your godsend. So, Kyle, you're up. I think you're up next for in my mind and I, you know, Kyle, I don't know Kyle well actually but Kyle is at Wake Forest University and Wake Forest is really just kind of blown off the charts with collaborative annotation just doing so much interesting work and activity there. And so I wanted to make sure Kyle was here to kind of bring that perspective. Kyle, where are you calling in from. Yeah, everyone I'm Kyle Denlinger. I'm actually in Winston Salem, North Carolina. So on the other side of the country from most of you all here, I've been to Portland once. So I don't know if that counts. I'm the digital pedagogy and open education librarian at Wake Forest and have been at Wake since 2012. As a librarian, I both teach and do faculty support. So, have been very, very busy since March and everything. Since everything hit the fan with coven, and I'll probably get into this into this later but we have, like Nate said, been doing a ton of work with documentation with our hypothesis pilot in Canvas through our faculty training institute that we've spun up. So I'm excited to talk more about that. Thanks for having me. Great. Thank you, Kyle. And I think Winston Salem is the Portland of North Carolina, isn't it? That's what they say, yes. Yeah, yeah. So you're good. You're part of the group. And then Michael McGarry is somebody that we've worked with for a long time. He's at Cal State Channel Islands. And, you know, I think Cal State Channel Islands was the first school to really formally pilot hypothesis in the learning management system. And I'll apologize in advance that both of these schools use Canvas, which means that we're going to be a little Canvas focus today, maybe when we when we reference hypothesis but we are talking about kind of piloting in general. So Michael, do you want to jump on screen and say hi? Sure. My name is Michael McGarry. Thanks, Nate, for the intro. I'm the instructional technology lead at Cal State Channel Islands, part of the teaching and learning innovations team. So we really focus on working with faculty and enhancing the online instruction experience for not only them but mainly for our students. So getting them on boarded with different tools and stuff that can really help elevate that online base, which, you know, that we used to have kind of our little core group that we met with and worked with and then since March, as I think we can all agree with, it's exploded. So now we're supporting the entire campus. But it's been, it's been a great opportunity for us to, you know, get into the trenches and work with a lot of folks that we don't typically get to work or hadn't previously got to work with. So, you know, the weird crazy time but it's been, it's been pretty good. So we're, yeah. A weird crazy time for everyone, right? And maybe that's a, that's a great way to kind of key up the conversation we want to have. I'm just going to, you guys have probably all seen it because you went to this blog post to RSVP for the show today. In that blog post, we kind of tried to outline some kind of best piloting practices that we sort of leaned along the way. A lot of them came from Michael's thinking, for instance, and the experience of Channel Islands. But, you know, we could, we could talk about piloting in the normal world, if we wanted to. But as everybody's made clear, like we're not in the normal world now, we're in an extraordinary world. Plus that there's a global pandemic going on. It's also that that global pandemic caused a huge shift in what was happening in education, right. And I think, you know, it's a time when we are kind of faced with the possibility of trying out some new tools. And, you know, people, whether it's people individuals trying them out for the first time on their own, or whether it's institutions adopting and so this question of piloting things is maybe substantively different today. Sorry, it's still relatively early for me here in Portland. So I thought I might kick things off by handing the baton to Aaron, you know, and in your work, in your work piloting so many different, piloting with so many different institutions, not only with hypothesis but in other work that you've done. So I'm wondering if you know how you're thinking about piloting now in the time of COVID and then maybe you can use that as a bridge to bring Kyle and Michael into the conversation around that same question. Yeah, from the hypothesis perspective, I think I'm frequently thinking about how do we get the message out to faculty during this weird time when we're not meeting in person necessarily. Because I have ideas about what makes a successful pilot from the hypothesis perspective. That's, I think, the biggest barrier for me. And within a piloting scenario, but I'm curious for Michael and Kyle, what on your end are some attributes that make for a successful pilot right now. I think so. piloting stuff, we try to always approach it at CI from, as you guys noted in the blog post really looking at it from an angle of trying to solve a problem, not just hey this is a new shiny tool. Let's try it because it's new and it's there. We really try to get ones that are going to solve some sort of pedagogical issue. We've tried to do that from the get go when we're piloting new stuff, which you know, sometimes is seen as with some of our faculty can be seen as kind of like annoying in a sense because it's like, well there's this new thing that I want to use why can't we use it it's like, well, we have to explore it a little bit. But we're fortunate on our campus to have a lot of faculty that are really into looking at new things and looking at it so we're in a fortunate space in that sense, but I think having having that sort of focus on doing it with some care and not just jumping at the new things just for the sake of them being new, because at the same time we don't want to pilot a bunch of stuff that we're not going to look at eventually adopting to I think longevity is part of something we got to look at when we're looking at new tools like if we're going to try this thing out. Let's look a couple steps ahead and look what eventual adoption would look like is it feasible, otherwise you end up running the risk of getting a bunch of people hyped on something. And they had it for a semester and then you can't support it moving forward which doesn't help anybody. Yeah, so I'll kick it to the Kyle to chime in. Yeah, Michael's last point is pretty relevant at Wake. So our, I want to say our organization around IT adoption is pretty loose and community oriented. There's not a whole lot of top down hierarchy. So, when someone's interested in the tool, just trying it out integrating into Canvas or whatever. If they have the right relationships as I'm sure this is, I'm sure this is the same way at many other institutions. If you have the right relationships you can basically find out a way to get a plugin or something installed in like a short individual private pilot. But we do run that risk like Michael said of getting people really hyped about something and then not having the resources to sustain it long term. But with hypothesis, that was something that I personally had been using for a number of years, both in my own teaching and some faculty support. This was prior to any elements integration. And when COVID hit. I knew that my director for our teaching and learning center center for the advancement of teaching rather was also a hypothesis enthusiast, and we were faced with this scenario where we had to figure out a way to train all of our faculty to teach online. So what happened for us was we had so we had basically 1000 faculty that we had to train to teach online, and there were maybe 12 people on this loose team of it adjacent support. And because I had that buy in from our teaching center director, because I had such expensive experience using it, we were able to basically make a very easy sell to the powers that be to integrate with hypothesis and get that the pilot kicked off. And in doing so we were able to integrate it into the course that we built for a small cohort of 65 faculty trainers from their departments. So they applied to participate in this faculty learning community. They spent two weeks intensively learning about the nuances and interest intricacies of online education. And then they're currently engaged right now in developing their own faculty learning communities that they will then turn around and deliver to their departments. So, in one way it's kind of like 12 people are training 65 people who are training 1000 people. So, because we were able to integrate hypothesis into that initial cohort, we were able to expose people to this new practice of digital annotation of social collaborative annotation. Hypothesis is the tool that we chose because it was the one that we had the most experience with, but it got people thinking it got the faculty thinking that, you know, online education doesn't have to be discussion boards. They can engage with text more deeply. And just this idea that practices online can be substantially different they don't have to be a transition or just a shift of modality. There are different affordances that are available when we when we move to online and that gets them thinking differently about how they teach. And now, because they're designing those cohorts that those learning communities for their fellow faculty members. Some of them are integrating that into their own communities, and thus exposing more people to it. So it's been a tremendously successful thing for us. I have overwhelmed myself with support requests now, but that is kind of the nature of the beast is, it seems like every day I'm answering so many support questions for hypothesis. But it's a great opportunity for outreach and I think our students are really going to benefit in the fall from this new experience of engaging with text. So I really like that you talked about this idea of creating these learning communities for faculty and then essentially those learning communities creating their own or those faculty creating their own learning communities and passing that knowledge on, especially in terms of thinking about pilots and how do we get faculty to try these different tools, right that we are implementing and that we found pedagogical value in. What do you Michael and Kyle think is the most difficult thing about piloting during this time. As you think about the context of trying to get faculty to try so many different tools in such a weird time when they're all they're learning so many other things too. I think one of the most difficult things really is now everyone wants a different tool. There's so many tools out there and because we know we're going to be teaching online, or at least in some capacity, people are making requests for every tool under the sun. And we know we don't have capacity for it we don't have the funding for it we don't have the support infrastructure in place. So we're having to say no a lot and you know, to be clear I don't have any decision making power. I'm not the person who controls the budget or anything like that but they come to me, a lot of them do because they have a relationship with me. And, you know I'm able to temper some of that expectation sometimes and point them to tools we already have existing relationships with. But that's our biggest challenge right now it's like just the overwhelming demand for new and various things that we just don't have the capacity to support. Yeah, I'd second that for sure I don't think I've had any single semester with as many integration requests I've had these past couple months and it's, it's on the one hand it's great to see because people we know that our faculty are going out there finding new stuff but it does. It sucks to have to say no. And we're not typically in the business of saying no but there's, you can only do what you can do right and we're, we have a really small but mighty team at CI it's like six of us really that are driving this whole thing and now with the support for the whole campus it's really just trying to make sure that we can best support the tools that we have, while still hopefully being able to adopt new stuff. I think the other. One of the other big issues, as you mentioned Aaron is just the fact that people are already starting to learn so much that's already been there, but you know we had like canvas for instance is a new ball game for some of our faculty because they were doing face to face classes they didn't rely on an LMS for very much if anything at all. Now all of a sudden they're in a situation where they're having to learn that and the basics of it they're having to learn the basics of, you know, some of the tools that we've had for years, and sort of throw like all these things at once it's it's it's a delicate balance right between use these things and then also we have all this stuff and it's like it can become overwhelming really quickly. And so it's finding that balance of really trying to meet the faculty where they're at and meet their needs and then so listening to what they're trying to do first, and then figure out what tools are available to help them do that as opposed to just saying, here's the open toolbox and all these things we teach how to use all of them, because then again it can be overwhelming and then you know just you want to avoid burnout from all this switch. I think another complicated thing, at least from my perspective on campus I've always been the squeaky wheel with the ethics of the technologies we choose. And because all of our classes are going online or many of them. There's increased demand for things like proctoring software and I don't want to throw that practice under the bus but there's not not often enough consideration made for all of the implications for adopting a technology. And there's like a lot of tension for me right now where people are requesting things, and they have perfectly legitimate pedagogical uses for them or even are required to use something like that for accreditation reasons. I'm not sure of my role in being able to step in and like say okay that's awesome but let's pump the brakes a little bit. Because what are the implications if we roll this out across all of campus. Are we going to do bad things faster by integrating something like that. Or just, you know, are we going to allow our faculty to think that I could just take my face to face course copy paste it into an online environment and still do close book exams, which maybe, but that's a that's a point of tension for me right now is figuring out how to insert myself into those into those conversations without like, I don't know putting too much value on my own ethics. So it's complicated. But important, or I would add to that because we have that conversation is a tense one on our campus to work constantly resistant to it. We actually as part of this whole piloting in a storm topic is you know you want to adopt stuff quickly that can meet where they are so that was actually one that we jumped into quickly is because respondents offered something for free and we were like cool we're going to get hit up for this let's go out. And it did a lot more harm than good, and we didn't do our normal kind of cautious and, you know, at least mindful deployment of a tool, and it bit us for I mean unfortunately it was something that it ended up being it wasn't an equitable tool there was so I think the ethics part of it is important. And just because we are in an emergency type of situation. I think it's important to still take that mindful approach into adopting a new thing instead of just being like well it's free and people are going to need it let's turn it on. We, we did it we thought it was the best move and it unfortunately didn't end up that way. Not to say that that's not something that faculty are needing it's not a valid point because sure it is but a tool like hypothesis has been able. We've been able to kind of steer folks towards a little bit more authentic assessment was something like that so I think that's, you know it's something to be mindful of for sure. So talking about this idea of, and I love what you said Kyle about doing bad things faster. And concurrently, I would assume that many schools right now are getting lots of offers for piloting or lots of offers for free trials from various companies. So you guys have kind of touched on how you choose which trial or pilot that you might do. So you decide that that pilot or trial was successful. What are some factors that you might use to you know you get to the end of the trial or pilot and say yes this was successful and we'd like to continue with this. What, what are some of those factors you might use. I mean having a dedicated pilot group and then consistently gathering feedback along the way was is something that's integral for us to kind of figure out whether or not this was a good move. Obviously there's budgetary concerns as well that have to be factored in but try not to make those the primary. It's really getting a group of faculty involved with the tool get them actively using a tool and then questioning, you know, just ask them along the way and survey them to see how is this going. Are you liking it what's the student response been like are you seeing a value in this does it meet the expectations that you were expecting like those are some of the questions that we try to really gather. And then I like from there with the smaller initial group if things look good you can expand it a little bit further before necessarily so it's like a pilot phase one and a phase two before full adoption kind of deal. Just to kind of gather it and I mean if the if the overwhelming feedback initially is just like yeah this is great everyone needs to use it then you know we can push but I think gathering that feedback, both from the faculty that are involved and also doing student feedback. On the tool is really important because if it's something that's like it seemed really cool and it had some decent functionality but the students couldn't use it or I had to write my own instructions and by the third assignment I hated using it because no one could figure it out. You know, it's, you know, the together that kind of input is important, I think along the way. Yeah, being at a smaller institution. We, it's kind of a blessing and a curse right so we are small enough. We're kind of too small to have an established process for evaluating the effectiveness of a pilot because there's just so many things going on and there's so few of us. So I would love to be gathering data about, you know, usage and the success individual faculty are having in their involvement with that pilot, how it's affecting our learning outcomes all of those things I would love to have more formalized but on the flip side of that we are small enough that we have pretty close relationships with many of our faculty and those that are interested in violets already have reached out to us because that's how the pilot usually gets kicked off. So we're able to, you know, sit down with them and have conversations and how what does this work in your classes and often you know we're even involved in their classes kind of co teaching in some ways. So, we don't have formal criteria for whether something was excess, whether a pilot was a success or not, but I think maybe if there was a criteria it would be how many people we make mad. We turned this thing off so that's not like the best metric but it is something that we do take into consideration. But really we do only try to get into those pilots where we are ourselves invested in the tool and seeing it and invested in seeing it succeed on campus because we know that it will have really positive benefits to our students learning. So that said, thinking about, you know how we gauge the success of pilots from the school perspective and from the partner perspective or the ed tech partner. What are one or two things you wish that your ed tech partners knew about piloting from your school perspective. You guys are like, wait, hang on, let me be careful what I say. Yeah. It's a really good question. And like I said earlier I'm very much the squeaky wheel on ethics and I'm slowly starting to instill that kind of thinking in my, the various partners around campus that we need to pause and think about the implications of adopting a tool. And because they know that if they don't, I'm going to make some noise about it. Maybe out of a right now I might be that they're thinking about these things out of a concern for just like keeping everything calm but eventually I hope it would become something more like we're going to stop and think about these things because they're really important and we need to build that into our process. I don't have anything else other than that right now but because it's a small pretty tight knit group. I want us to be mindful and thoughtful about our adoptions and not just like Michaels that just turn things on because they're available and free. Yeah, I mean, I said, I don't know if it's necessarily an issue. But I really appreciate when we're able to get vendor partnerships where it is just that it's actually a partnership. There's nothing worse than having someone it feels like you're literally getting a sales team you sign on and then they disappear. We have a couple, you know, there's there's a couple big name tools that we have that come to mind when we think about that. Hypothesis is the polar opposite of that right. We actually just did a webinar this week with our faculty and you know we got a demo from hypothesis they came in they showed how the integrations work and we were able to work together to actually really tag team that experience. And that's the kind of stuff that we really look for when we're partnering with a with a vendor. I think, unfortunately, that does get lost some of the time, but it's really valuable to have. So looking back on this whole past term where we all went virtual immediately with very little preparation I think, what would you Kyle and Michael have done differently in terms of piloting any new tools. Honestly, I don't know that we would do much differently. I mean things happened as they did. It was very much on an accelerated timeline. I think we had always intended to integrate hypothesis. One thing I haven't mentioned is that we are still currently technically in the midst of an LMS transition. So we're like, we were on Sakai, and basically the understanding was that in the fall this coming fall, no one else will be teaching new classes in Sakai. So like we're right at this cut off point. So we're transitioning all of our faculty not only to online teaching but to a new brand new LMS. So all of these integrations and all of these partnerships were going to happen this spring, this summer anyway. And I think that the, the hypothesis partnership came right at the right time for us. So that was great. We're also for the first time exploring a video management platform, which was something we had planned to do further on down the line on an expanded timeline but now, since everything is happening so quickly, that has been shortened up. So if we could do anything differently, go back in time and not have a virus but the, an extended timeline, but that's impossible. So, yeah, I don't know. If, if there was something else maybe we'd have a more formal process for requesting tools and more clear criteria for how to measure success, clear communication about what kinds of resources we have and which tools get priority. But again, being at a small institution, it is much more of a community. And it's sometimes who you know or, you know, how you personally are able to sell something to those that are making decisions. I just want to applaud you Kyle for going. I can't imagine we did our LMS transition a couple years back and I can't imagine doing that in the midst of all this so. To be, to be clear, we also we had just hired on our LMS manager about six months before the virus that are going to be a year before so she's been very busy training everybody and yeah, so yeah, fortunately touching the LMS. It's a huge move. I mean, I don't know that we would necessarily do anything different. I mean, as I talked about earlier, like we did pull a trigger on something quick and I think that was a learning experience for us was, you know, even though we are in this situation, I think, just taking a step back and taking the normal questions that we would and treating pilots as we've developed a pretty good process of it. It's evolving. It's very young. I mean, we didn't have any sort of process at all and now we have. I wouldn't even call it a formalized process so much as a framework in place but sticking to that when we have something I think would just be the main thing instead of and don't let the nerves of a panic switch gut reaction happen. It's really easy to do and you know the circumstances that we're in. I think we've learned from it. I mean as long as you continue learning about how you're doing things and evolving things forward. You know, it is what it is crazy times for everybody. Wow, what a great conversation I didn't expect all the twists and turns and yes Kyle you should you and everybody there at Wake should probably get some sort of, you know, metal. I can't even imagine that LMS transitions are enough under normal circumstances. So a couple of things have come up in the chat with with the participants. One of them is, you know, Kathy was wondering if anybody has any instruments that they use like to survey faculty or students who are piloting things that they would be willing to share. And I know Aaron hypothesis, probably is has something that we can share that we we work with all the schools that we pilot with to do some common surveying. I don't know if Kyle and Michael if you guys have instruments that you would be willing to share to maybe no surprise that we know but we have kind of ours was a actually have to go back and find it was pretty much we created a form that we were using and sending out and that form is kind of evolved as we've done different ones so again we're in a framework stage more than a formalized process of sorts but I think being adaptable to some of them too is important. I think we at hypothesis have our own tools for student surveys and faculty surveys that we are actually revamping for the upcoming term, but are always happy to share those with any of our partners, prior to sending those out to faculty and students. But we also take feedback on those from our partners to I think we're probably willing to share even more openly than that like we have a. So yeah, even whether you're a partner or not so we'll look into getting that into your hands and email anybody who RSTP about that. Hey, another topic that's come up there's been a couple really juicy ones but another one is kind of really pertinent for everyone and it might be interesting to hear Kyle and Michael how you guys are dealing with this but you know there's a lot of concern about budgets now that are that are tied to you know related to what's going to happen with enrollments related to the pandemic and so on and you know obviously that ties into also to tool choice and tool use because you've got you know these tools saying hey we're free right now but you know my cost later or whatever. So how is how are budget things happening. You know how is that conversation or thinking about budget happening at wake. Gosh. Fortunately, I don't hold any purse strings. So I don't get to make any of those decisions. Again, very much like many of my other answers it's generally word of mouth. Again we're a small net community posting community and when things are too expensive or when basically been tapped out. Generally those that need to know are made aware of it and we're able to communicate that back to those folks that are requesting those tools. But yeah, I'd imagine it's very much the same like anywhere it's, although we are private institution and in a much better position than I imagine many other institutions are worse. We're feeling it to the international student announcement this week was a devastating thing for us. So we're scrambling to figure that out and you know the conversation changes day by day in terms of budget. But again, I don't touch the budget with anything so except for like my personal book budget for my departments, but that's not relevant to this conversation. Yeah I mean it's, it's a difficult spot there's no way around it I mean budgets are getting slashed everywhere. I think it's important enough to secure some msft funds right at the onset of this so as a grant fund on our campus that we're able to secure specifically for piloting new academic technologies so I mean trying to find if the campuses have any things like that where you can propose put forth proposals to get you know money to fund stuff like that. We were lucky, we put forth a proposal and it got accepted so we were able to get a secure little chunk of money specifically for this effort. So I think that's a good sense and you know we've just got a couple minutes left and I want to close out by plugging the next episode again but um, one other question that's come up in the chat has been around security practices to and choosing new tools and I think it may be dovetails back to that ethics conversation to in the sense like, you know, one doesn't want to just adopt things blindly without some careful considerations for all the implications they might have a privacy and accessibility, ethics in general, and I know that in my community, I've come across a couple of institutions that have some pretty interesting kind of published documents about how they, how they address security and ethics in general, just another tool choices I'm wondering you, Michael do you guys have any kind of like written stuff around that in your framework or is that set on a case by case basis around security. Before we do anything that we want to formally pilot and integrate to our systems we have a standard process that it has to go through for procurement so we require a VPAT and HECVAT so that's the accessibility form when we require a security review so before we can even throw anything in they have to go through and clear those so that's that's kind of so that part is super formalized with our IT department that that's yeah that's a little beyond the framework part that's set in there if anybody wants to work with us we need those yeah got it what and that week. Yeah it's very much the same here yeah we have a security team that every single thing that goes through procurement has to go to the security team before it gets implemented. And yeah that's one of very few things that are formalized in that way. Got it. Well hey thank you all so much for this conversation I will share out with all anybody who registered the documents that I'm thinking about that address ethical policies that might be of interest to the panelists as well as the attendees right. And thanks everyone for coming and I'll just plug one more time that our next episode next Friday at liquid margins. If you choose to come or tune in after the fact will be on annotation in the composition English composition environment. So really interesting. We're lining up a couple really cool guests for that too so tune in next week if you will and big thanks to Aaron, Kyle and Michael for being here today.