 Hello and welcome to liquid margins episode 41 driving engagement and building community insights from partners on the hypothesis impact. We're going to let folks sort of file in here and then get started with our conversation in just a second so I'll probably give it 3040 maybe 60 seconds before we actually get started here but you're in the right place if you're here for liquid margins hosted by hypothesis. And we're here to talk with some of our partners about how they roll out hypothesis on their campuses. So hang tight for just a couple seconds and then we'll officially get started here. All right it looks like the students have have filed in here we got a great audience today. So I'll go ahead and get started. Welcome to liquid margins episode 41 driving engagement and building community insights from partners on the hypothesis impact. So I'm really excited about this episode actually I'm going to hold off on getting into talking about my excitement to housekeeping first always forget about the housekeeping I don't want to dive right into the ideas. So in terms of the housekeeping stuff let's see here. We have an upcoming episode on stem which I'm very excited about mark your calendars for June 28. We've got a great group of scientists and mathematicians and computer scientists that will be joining us on the 28 from a variety of schools. So mark that on your calendar. And then sometime in July we're going to be doing social annotation as instructional scaffolding. I'm excited about this episode to because we're going to be rolling out some features that are going to better enable instructors to move annotations between groups to move annotations between courses. Not every feature annotates for students or with students, but I think we're going to start to see more of that as we roll out features that make it easier for instructors to do that. We're going to be talking to some instructors who do scaffold assignments who pre populate text with annotations prompts or model annotations. So I'm excited about that one. Stay tuned to all the channels about about that one coming soon. So today, look at margins is really a conversation around pedagogy and strategies for implementing annotation and implementing hypothesis if you're here for kind of an introduction to the tool. You should stick around because I think you'll be inspired, but this is not going to be a real how to, if you want to how to you should reach out to education at hypothesis and we also have a lot of YouTube videos and to check out if you kind of want some of the basics but we're happy to provide a live demo but today we're going to dive deep pretty quick in terms of how social annotation is used on our campuses and in the classroom. And then finally, if you have a question you can use the Q&A feature we don't have a chat turned on today. So drop questions in the chat and we have a team my team of CSMs are here customer success managers are here to answer those questions. So feel free to jump in there. And then finally if you'd like closed captioning there's a way to turn that on with in your own zoom window. All right, so I think that is the oops back one. Great so driving engagement of building community insights from our partners on the hypothesis impact I'm going to be very brief with my introduction but I just want to say, I'm super excited about this episode we're doing something a little different today. We're typically speaking with classroom instructors about their use of social annotation and teaching. But today we're taking a step back and talking to a group of folks who support instructors using hypothesis and other technologies and practices in our teaching that's not to say that some of you guys may also still be in the classroom I know there's many hats that get worn in your roles but at hypothesis you're largely known as our kind of primary points of contact for the broader implementation of social annotation on your campuses. And that's why you're here today. I'm going to say up front here. I hate the word vendor. I'm an educator by calling and training and much of the staff at hypothesis including my co host Christie to careless and many of our CSM so we're manning the, our human in the q amp a were educators by by by calling and by training. That's our background some as classroom teachers some as instructional designers. And we really still view ourselves as educators supporting the learning of students and so as such we really view our relationships with customers are collaborators as partnerships. We listen to our partners about their unique goals and challenges and we try to help them leverage social annotation to address those challenges and goals. We're always learning from our partners about new and interesting ways to use hypothesis new and interesting ways to introduce hypothesis to a campus. And our partner feedback really guides everything we do from future development to what workshops and other programming we offer liquid margins episode ideas all all of this stuff is really driven by the needs of our teachers and our you know instructional partners like those that are joining us today so that is all to say let's introduce the group. We have four schools represented and two schools have sent to reps because they were eager to bring the whole team, which is awesome. So these are our panelists we've got Megan grady associate director of academic partnerships at Butler University joined by her colleague Kristen Palmer team lead in instructional and instructional technology also at Butler so welcome friends from Butler University. And then from University California Santa Cruz we have education technology specialist Dana can art is probably of this group I'd say been around the longest I mean that in the nicest of ways. Santa Cruz is a longtime partner and we've been working with Dana for a long time and as I've gone to conferences in California, a lot of people will say that she's the one that intrusive to hypothesis sometimes even when they weren't there not no longer at Santa Cruz so thanks and welcome Dana. And then from Portland Community College we have two folks joining us James Pepe welcome and David Vasquez. This is our instructional technology specialists at Portland Community College. So welcome from the great state of Oregon. And then, finally, last but not least Vince sent germane senior instructional design specialist at Grand Valley State University. So this is the group welcome everybody. Thanks for being here. And let's get started. So, we're going to try to make this as much of a conversation as possible. Again, if you're in the audience and you want to to ask a question feel free to drop it in the q amp a and we'll sort of include you in that conversation. But the first, you know, area that I wanted to talk to you guys about is just what led your campus to adopt hypothesis social annotation. And maybe since you've been around the longest again I mean and the nicest of ways Dana, you might start our conversation on this topic. What led you guys to adopt social annotation. That's in a cruise. Thanks to me. Well, it was the enthusiastic and excellent instructors in our writing program who were leading the push for adoption. They immediately saw the benefit of social annotation and we're very persistent about getting it. We piloted in fall 2020. So as you said it's been a few years for us. This was during the period of emergency remote instruction and I think immediately students and instructors are like, like we're very excited to have a solution for this that wasn't simply over zoom. Of course, we've moved beyond that now and hypothesis continues to blossom across all subjects on our campus includes chemistry, history, sociology, biology, and more. It really Santa Cruz is a great use case for seeing a particular discipline adopt, but then it grow from there, which has been really cool to see over the years and now I think Santa Cruz is one of our, our biggest campuses in terms of usage. Let's go to Vince next Vince what led Grand Valley State to look at social annotation as a teaching tool. So first I'd like to thank you very much for inviting me today as a panelist during today's session. And we've been a partner for approximately two years now. So going back to August of 2021. We had a faculty who had been using hypothesis using the web based version of it who was trying to integrate it directly into what was her blackboard learn original course and she was experiencing some problems with the setup and with that installation so she had reached out to our department. And then we contacted what was hypothesis technical support. Now they were just absolutely fabulous they provided us with all of the information that we needed. We were able to get her setup and she was immediately able to start using that during what was that upcoming fall semester but during those conversations we also learned that there were other faculty at our university, who had been using hypothesis as well. So we engaged in a discussion then with our customer success manager to try and determine what different options were available to us for implementing what was hypothesis on a more seamless and easy to set up basis. And that's when we then started to talk about the LTI integration into our blackboard LMS. And we, at that point, discuss the opportunities to set up a pilot and begin that process for what would be our winter semester for 2022. I'm sorry Jeremy are you were you speaking. Thanks, thanks. Do you have any memories of what led that initial instructor what discipline they were in and why they were excited about social annotation initially and sounds like they were using hypothesis in the wild outside the LMS and then you help them and others get it integrated but do you remember what they were excited about. She was in our writing department, and they have been huge advocates for the use of hypothesis during the past two years, we have a number of faculty who were part of that early pilot that we had initially set up then for what was that winter 2022 semester. One of the things we did was we took a look at or after getting what was basically a list of contacts that you had at hypothesis we reached out to those faculty, and just inquired as to whether or not they would be available and interested in participating as part of that a pilot and they were more than happy to hear that the university was taking steps to now fully integrate the tool directly into our courses. Yeah, that's so cool that folks were using the sort of free what we call you know hypothesis in the wild, and then you were able to help them kind of integrate with the LMS which is far easier when you're starting to scale out the usage of social annotation. Awesome. Well, friends from Butler, Megan, what led Butler to explore social annotation for teaching and learning. So, prior to my current role, I was an academic technology specialist, and we had, you know, a handful of faculty who had come to our office and say, I'm getting really concerned about my students not doing the reading. What can I do to encourage deeper engagement and excitement about reading what might that look like. So we started having some conversations around annotation tools and collaborative annotation, which then led us to do a pilot with a small group of faculty who had expressed interest over the years of for these kinds of collaborative annotation tools and had used the tool, as you said Jeremy in the wild, and had found that it was very helpful in that regard. And also, one aspect of education that faculty really want to push forward with students is this idea about scholarship as conversation, and getting students to think about scholarship in that way in the papers they write, but also in the annotation, you know, world that gives you the opportunity for entry points into scholarship in a different way and have conversations with your peers. And so that's driving that home and since we did the pilot, and we did some, you know, assessment to see how students liked it we got some really good feedback and then move forward with with the contract with hypothesis. So we moved out of the pilot stage I want to say, near the end of 2020. I mean, yes, was it. Okay, trying to think remember Kristen if that was the the correct one so Before we go to PCC I just want to pull the room here real quick I really, you know, interested in what you said making about this concern that faculty came to you with about students not doing the reading does that resonate with others here that like you have faculty that are anxious about this. Have you heard that before. You guys are so beating you're you're not, you're not in your head let's go to PCC and hear what they have to say about what led Portland Community College to adopt hypothesis and again don't be shy to it's okay if you cough or you know, laugh, jokes, you know, don't feel like you have to always be on mute. Again, we can make this a conversation but the Portland Community College I think James were you going to kick it off for Portland Community College with these earlier questions. Thank you Jeremy and thank you everyone for taking the time to be here with us today I do hope for what we're sharing is being useful and useful that experience for time is important precious and I do hope that we'll get something out of this. So, I was actually in a shock that come from a lighting writing and literature background before I was brought into my role as a technical technology specialist so I think I can speak to hypothesis from the instructors point of view I feel experienced teaching online remote good old fashioned brick and mortar face to face in multiple settings and as soon as I saw hypothesis because it was chosen by my excellent team before I was brought on this. This is fantastic. This is great. Because well, it reminded me very much of what they noted on technical writing called the Anderson says in his book, technical communication or readers centered approach. Besides audience awareness is the fundamental difference between ineffective writer based communication and effective reader based communication. So what does that mean. This communication is a dialogue. It's a dialogue between you and your potential reader, our readers are not trained so fat so what we're trying to do is anticipate their emotional and intellectual responses line by line by moment to what we are writing. By anticipating the readers needs, we can communicate complex information to fellow experts and not experts like and what hypothesis we're doing. It was doing that from a reader's perspective, you are engaging in a dialogue with the text into boss and language from a excellent Portland area, I pop is just the bullet in any case, the text in the discussion in a way that the traditional discussion from the LMS just cannot do and do well. We've seen so many of our instructors are comfortable with Google Docs. Okay. All right. And that kind of annotation. This parallels that it seems so nice. It's either as much benefit of assignment, those takes assignment, or as a great assignment, directly into the great work. It works wonderfully. Very smooth. Our instructors like it because it's something you can pick up midstream and of course. Okay, so you know what I feel like replacing. We have a discussion board with hypothesis, and they can do it quickly, very quickly, syncs up nicely, and they just excited about enthusiasm about it makes me excited too. So hopefully that answers your question. Thank you. That's great James. Yeah, we like I sometimes talk about hypothesis is sort of Google Docs for every anything right anything any text on the internet. There's a couple of themes that I just want to pull out and see if anybody wants to riff off of James echoed something that Megan said about dialogue. Right. The importance of helping students understand they're entering when they come to college. When they start to work as you know scholars in an academic setting. They're part of a conversation and to encourage them to participate in that conversation is so powerful so I want to know if anybody wants to sort of riff off of that idea of either. The importance of kind of helping students think around themselves as scholars are being part of an ongoing dialogue, and then also the, yeah, anybody want to riff off of that the more sort of pedagogical reasons that folks approach hypothesis go ahead Vince. Well we've got a faculty person who has become a real advocate, and actually one of our hypothesis champions here at the university. She lists in all of our training sessions providing her experience, as well as her students, and what that's meant in terms of students ability to engage then in the course content. She starts, or at the very beginning of her course she uses hypothesis for the purposes of having to delve very deeply into what is her course syllabus, and she gives them an opportunity then to be able to review that syllabus to be able to indicate then directly within the margins, the kinds of things that they don't understand or that they might be concerned about things that they are very excited about experiencing during what will be that course, as well as talking just overall about what other things could actually be brought to the course in terms of suggestions that they might have for improving the kinds of interactions that are going to be taking place. And one of the things or feedback that she got from her students was that in all the time that she had been a graduate student at our university and had engaged in hours and hours of discussion boards and forums where all they were being text. She really felt that by using hypothesis and being able to review that syllabus that she was finally able to engage with her faculty, as well as the other students, so that they could all begin in this place where everyone had an understanding of what the course requirements were going to be and that the faculty was also deeply engaged in that work and going to ensure that students were going to be successful. The faculty was sort of opening up the course itself as dialogue right the syllabus is the beginning of the conversation. Dana I'm just basing this on unmuting. Do you want to add anything. Yes, yes, that was my good. No, thank you. Megan said scholarship as a conversation and that really is resonated with me and my experience with the instructors at my campus. One of the comments that really sticks with me every time I think about hypothesis and I'm telling other instructors about it is one of our instructors champions I like that word fans I think I'm going to start using it. They use it very deeply in their course and their comment was that seeing the quality of the annotations transform from the beginning of the quarter to the end of the quarter was really incredible. Students start asking more in depth questions and being more analytical as they engage with the text and that's a skill that's going to stay with them beyond the course. Yeah, just just to piggyback on that this conversation is kind of bringing a mind and to mind a very popular quote this is not going to be news to anyone that the Benjamin Franklin quote that talks about tell me and I forget. And I think one thing that hypothesis does is it involves students at a level that typical kind of academic technology tools can't and particularly like responding to readings and doing different things like that. I think the involved me and I learned there's also kind of this idea involved me and I belong. I feel like I have a community around me and I feel like I have a place in this community. And I find that talking about someone had just mentioned doing it with the syllabus. I think one of the things that gets students more comfortable with using hypothesis is to start out with a low stakes type assignments, because it can be very intimidating when you start saying this conversation and you're scholars and you're part of this students are like I'm not quite ready to have that be part of that conversation. Like, like don't let me do I'm not ready for that but maybe give me some low stakes that I can do up front and start getting responses from my peers and start communicating with my peers in that way. And then when we move on to annotating like an important historical document. I've had the opportunity to see that inserting my voice in these documents doesn't have dire consequences and then I'm not doing some things really scary. That's great. Yeah, I want to hear your voice is a pretty simple thing to say and to offer a platform for all right so we're very this is all very nice to hear how wonderful hypothesis is but of course, it's not easy to introduce a new technology on a campus to shift from the discussion forms been around since even I was in the classroom so it's early 2000s. Right it's a tried and true tool and there's some people who probably had a hard time getting even to use the LMS I'd imagine, right. And so I appreciate how much how excited you guys are about hypothesis and how excited your faculty are and how far it's come in the time that you've worked with them but a little bit about some of the strategies or obstacles that you might have encountered as you tried to roll this out on campus and maybe we'll start this time with Portland Community College. Thank you Jeremy. So, one of the possible obstacles that we've had just had to do with sort of understanding and meeting our faculty where they are at our particular college 76% of these partners there, which I was once a part of our time yet, there's nothing part time about what they do because they're usually working from multiple institutions and sometimes doing three different elements in the same day, you know, that duration. So, how to actually encourage them to try a new tool. Okay, because they're all subject matter experts they're all very good at what they do, but they're balancing so many different things. Okay. And so, I understand anyone's competition. So what we did is we found certain instructors who are champions of hypothesis and using it effectively in the classroom. And what we did is we created what we call we call faculty show and tell us in other words. Well, once again, our excellent open area hypothesis trainer work with us to co facilitate these trainings so we have faculty members giving demonstrations on what they're doing with hypothesis. How students are more engaged to do surveys sharing what kind of objective feedback we're getting on a subject matter and cause and effect. And now now instructors immediately after their training with them contacting me contact David contacting my other I guess colleagues like how I get this started. I don't I'm interested. That's not so hard. What can I do. So, on plus, we were able to offer some stipends as well that actually account for professional development. That was an improvement. And did the, how did that stipend piecework James that's provided by the system. So our department had put together a fund of money, you know, to encourage professional development. We're in house training, facilitated by us, and that is actually encouraged. Really encouraged and rightfully so. To amplify your celebration of Jessica Fuller, the CSM for the Northwest, and also do appreciate she's from the Portland area but she's given me a hard time for not being able to pronounce the town that she actually lives in. I think it's scapus. I just want that on the record here because I've been teased before for not saying it correctly. Vince talk a little bit about roll out at Grand Valley State. So we actually wanted to start small, just because our learning team was not very familiar with social annotation or really what the capabilities were for the tools. So, we had that existing list of faculty that we knew were using it in the wild, so to speak. And we had reached out to them, and they all committed then to helping to facilitate what was the pilot during our winter semester. And so they used it in their courses we got feedback from them as well as their students and found that it was really going to be a great fit in terms of the work that they wanted to try and accomplish and the engagement that they were asking for from their, their students. In the learning team we also discovered that it was going to be fully supportable that there wasn't any specialized training that was going to be necessarily required in order to be able to ensure that faculty could use the tool. So at that point we then opted to purchase what was our license and we prepared them to roll it out in the fall of 2022 to the entire university. And to feed up to that, what we were doing was trying to get out as much information about hypothesis and promote it to our university, basically across all of our campuses. So we have a monthly e-learning newsletter that goes out to all faculty, all administrators on our campus. And that created the fact that we were going to be introducing the tool. And along with that, we were going to be providing then faculty with a number of training seminars that were facilitated initially by Becky George and that work has continued with Suzanne Miller they just do an outstanding job of informing our faculty and training them on the use of the tool in what is now our blackboard learn ultra learning management system. There's a variety of emails out to targeted faculty and departments who we knew were going to be, you know, maybe more interested in using the tool, or not now knowing that we have it available and that it's an integrated part of what is our blackboard learning system that they could immediately begin to certainly take advantage of that. And we built a blackboard course site where all faculty who are enrolled in what is those training sessions get automatically enrolled which gives them access then to a variety of hypothesis resources. The one thing that we have found that has been very important to the increase in the amount of usage across campus is just simply word of mouth. And we've gone ahead and proactively asked faculty who use the tool who liked it to encourage their colleagues to start using it as well. And overall we've seen just a huge increase over what has been that first year and a half of usage to the point where now that we're getting ready to sign what is our second license agreement. We've actually doubled the number of seats that we expect to actually be using them for this upcoming academic year. That's great. So many wonderful tips here for folks maybe you guys are learning from each other as well but I'm hoping there's some folks in the audience who may be in similar positions and thinking about how to roll this out and getting some really great pro tips here. Butler, how's it worked on your campus to roll this out. Yeah. I saw there was a question in the chat about a competitor tool, and we did look at other tools in our process. And didn't just go with hypothesis because we had faculty using that. We have quite a quite a few number of faculty using that, but we have a process at Butler where we have our IT colleagues look over the tools for data security for students, what data is being collected and security things like that. And hypothesis did pass our IT kind of standards whereas other tools did not so that was also one of the things we appreciated about hypothesis was their data and privacy policies. Meg, I saw you come off mute and I'll let you continue with the rest of it. Yeah, no thank you for that so much. You know, I would say our rollout here at Butler has been very gradual. And it's been a combination of like open invitation come check out this tool come to this workshop, and also targeted communication. I think we've had the most success when we connect with department chairs who are passionate about these kinds of tools, and then there's a trickle down effect in their department. When I look at our dashboard for instance, I see like splashes of interest from 2021 from, you know, a certain college and then I'm like what happened there and I think we did one workshop, but the sustained use over time has really happened at the department level. We have history and anthropology it's mostly on the history side where we're seeing a lot of people use it from one semester to the next, and using it over a period of time, which to me indicates more success than someone using it one semester and then we're using it again, right, because they're improving their usage over time so we see a lot of it with history. We also have a really strong advocate and political science, who has shared it with her faculty, and then they've become interested and scheduled times with us to use it. So, I think our best rollout strategy so far and I'm going to let Kristen talk about kind of future state what we'd like to do with more rollout around first year. And I think the best strategy so far for prolonged use and for increasingly expert use is to talk with department chairs, get into the department group and and really find your people who are open to this and are using documents in ways that they really need students to have this ability to annotate and have conversations. One of the things before we go to Dana that I'll just say is I appreciate the point about disciplinarity and departments. One of the things that I'm very interested in I should have disclosed at the front that I'm also a composition instructor by training so my background is English and a lot of the entry point has been the English department for several of these schools, but we do see as Dana can talk about, you know, diversifying the disciplinary use over time. What I'm very interested in and our CS team is very interested in is, how do different disciplines use it differently, and how can we help learn about that and share those lessons right so we're we have some profiles on our site we call them success stories that are always a particular professor in a particular discipline and one of the things we really try to learn and then build training materials around is, what does this mean for composition or first year writing, what does this mean for history how is it different right maybe it's primary source what does it mean in political science maybe it's a lot more academic articles and annotation is of course similar but also different in those different contexts and we really try to work with to bring some ourselves and bring some with our faculty and then be able to bring to you guys. Department specific trainings or department specific demos that speak to the goals and challenges of that particular area. So Dana, how did we do it started with writing, but now it's like across the entire Santa Cruz, all the departments and colleges are involved it seems like. Yeah, so many. I like Megan's idea of talking directly to department heads that seems to be the most effective path that I'd like to be able to take on, not to jump ahead to the future but I'm a big workshop person. And when we had started our adoption, we were able to lean on the good people of hypothesis fabulous collaborators to facilitate early workshops for us, and they were open to everyone, and very well attended and workshops that we've had since in the past have been, you know, a trickling of people. So I really want to emphasize Megan's idea of trying to target department specifically seeing how hypothesis can work with their courses. Like Vince, we also have a newsletter to a monthly newsletter which has also been very helpful with communicating with instructors about upcoming workshops, when we do have them, and also just new features that are coming out. And also like Vince, we had also done a self paced course as a resource for instructors. What I really liked about the hypothesis workshops when you guys did them with us and how I modeled my workshops after that is that the instructors get hands on immediately with the tools not just a presentation but it was here's the text we're going to annotate together. So I tried to model the course after that of like, here's an article annotate it, and also just resources, lots of links to hypothesis resources resources that we've created. It's been repurposed recently. It's now focused less less on the technical instructions and more about the benefits of learning with social annotations and how that activity helps their students and putting that information directly in front of the instructors with their lens when they're evaluating technology. Yeah I just want to emphasize something that you mentioned I think a couple of you guys have mentioned that you have or hypothesis has helped you build a course shell in your LMS that houses resources that can be used as a home base for work a workshop or multiple courses so you can go students can go in there and actually practice using social annotation of course experience what it's going to be like in their own classrooms, but also there are resources they can go back to. When we're able to do that it's really exciting I think it becomes a lasting resource at the school to for everybody to be pointed to well here's a course you can jump in there and it's always fun sometimes we give a workshop in your two or three, and you can go and you can be annotated by the original cohort of early adopters, and they can see their colleagues right the new the new cohorts can see their colleagues so definitely encourage folks to build up, build that kind of resource out if possible and we, we have a model for we have templates for them we literally have you know canvas shells if you're a canvas school to just go ahead and take it and it will be pre populated with resources and if you can get our CSM access then we can build that out for you. All right, let's move on to what the response has been. I think this has been peppered throughout the conversation here that you've got some champions you've got some excited teachers, they've helped get others excited. But let's talk about what the response has been and I'd love to hear, you know, if anybody has had contact with students or heard feedback via teachers from students would love to hear that as well and maybe we'll start with Portland Community College has been quiet for a while. Hi everyone. So, some of the feedback that we had from faculty it's been very positive. Some of the things that like you mentioned have come up from other people so far is that using hypothesis and of course has really kind of promoted reflection and made it more visible to students in the course and and also helps build that community that also was mentioned and it kind of takes what a typical discussion topic where the conversation would be from a solitary form of communication into this more collaborative form of communication. So, students are really enjoying that part of it that they're actually like, almost even just a variety of the tool it's a little bit refreshing from what a typical discussion tool in an LMS would be a learning management system so it's kind of the student feedback about it and it's also helped them with comprehension. I feel like some of the feedback that our faculty have given us is that when they kind of survey their students that they're noticing that there's better comprehension because they're seeing how their peers are going through a reading and kind of modeling that using that as a way to kind of understand the reading in a different way that maybe they weren't getting on their own. It's been a really good way for them to kind of see how other people approach similar readings and kind of engaging the content that they're going through in the course. And then from the instructor's point of view, you know, similarly they really like that it seems to be building that retention. It's because they're they're kind of going through these talking concepts in the course like they they can talk through the concepts with each other. And it feels like that builds more again that that kind of blow through the conversation where it's not that individual person kind of going through on their own there. It just feels like students through that sense of community again, kind of builds that option for them to just persist through it and be able to read the same kind of content, and kind of gain those different perspectives, I think is kind of the feedback so it's been mostly positive again, I think everyone from the faculty this to the students have been enjoying it it's been kind of a really untold to use and so that's kind of been the type of feedback that we've been getting here at PCC from the faculty that are using it and the students as well. Effective and fun fun. It's an important piece I do think it's fun right the social pieces fun butler you want to go next. So, I have some, we did some formal assessment. I worked closely with the department chair for political science. She was part of our pilot. We did hypothesis and we sent out Qualtrics surveys to the students just to get a sense of, you know, what their what their thoughts were about hypothesis and let me just export this really fast so I can see it. And some of the things that we got from students is you know it made me a more active reader. It helped me to organize my reading. I learned a lot from my classmates annotations which I love this because I love when learning becomes like it's not just the responsibility the sage on the stage model right we want to learn from each other and other students to so I really love that comment. It helped me prepare for class I had a student say that and I also hear that from faculty, because they can look at like the annotations and be like okay there was an annotation desert here no one said anything so no one wants to talk about this. So we have a lot of annotations around this particular quote we need to talk about it. And it said some students said you know it helped me think critically about the reading they really appreciated that. In terms of finding what features they found most useful for students. I was really impressed with what students had to say a lot of them. And 19% said they really liked annotation tags which I didn't expect them to get into that level of annotation where they were using tagging categories to organize annotations but we did see some of that and, and that's to the credit really of the professor, who was coaching them through this process. As far as features they'd like added it's funny because some of these I know are in the hypothesis roadmap we had one student say different color highlights would be sick. So they were super enthusiastic about this idea of different color highlights and we know that's on the roadmap for hypothesis so we're really excited about bringing that to students. But yeah so we've gotten pretty positive feedback and I will say, a lot of the questions that we use to build out this survey to use with students we pluck directly from the hypothesis website. So hypothesis has suggestions on how to do assessment and how to, you know, check the efficacy with your students, and then you can work with faculty to implement these sorts of surveys. And I think it's really helpful and I give full credit to the professor. Shout out to Dr. Robin Turner at Butler University. She is so organized and thoughtful in her implementation of academic technology and was a great partner and kind of reaching out and being like hey could we you know what kind of assessment can we use with students what does this look like. So, so that's really been a good experience. Yeah, and talking about the resources on the website as well. I think one of the things that impressed our faculty right off the bat who are considering using hypothesis was the rubrics and rubric examples that were supplied by the hypothesis team. And then we do a new training that's, you know, that's that this was on the, how do I assess this what does assessment look like for this activity, and the fact that there's not just examples of assignments but also the rubrics and the evaluation I think has been really powerful. Thanks for adding that Kristen yeah we're really I'm really proud of the resources we built again we're a team of educators on the success team largely and so everybody's contributing to assignments and when we find somebody like your colleague at Butler that's got a great assignment. So thank you to our resource and resource collection and Becky George has added a link to our success stories but we should also add a link in the chat to our to our resource collection where those rubrics and other assignments and activities. And I just also want to reiterate one thing that Megan said which is if you are interested in surveying we do have some questions that we can share as a model for how you might survey. And what that's so cool about the tags and imagines that even someone like Dana has been working with folks for years with a social annotation. You know maybe we could do another survey to kind of see some of the deeper ways how are how are folks using the tool right are they using tags. I think tags is something that I really would like to help folks like you all help teachers leverage. You know for the reason that that student mentioned and we could maybe dig into it like why tags are helpful why they maybe help occasion what David was saying the kind of peer to peer learning that can happen. And I guess I kind of forgot where we were in the sequence here but maybe you could go next if you have anything about what the reaction has been. Sure. Thank you for sharing the reactions from students because we don't have feedback directly from students that's not information that we collect but something I would like to see so it's really awesome to hear that the students have the reaction to this tool. I personally use it to I love it. So I'm not even a student or an instructor and I use hypothesis myself. The responses that we've seen from teachers. Of course we have a steady use of new users every term along with our champions. And as everyone else has seemed to have the theme of you know it spreads word of mouth. I wish I could take more credit for spreading hypothesis across my campus but it's really instructors talking to each other about what's working great for them in their classes which is great and I really love that. I'm curious. Can I ask a question. Please. I heard David say kind of emphasize like the community building aspect of the tool and I'm, I'm curious like the other institutions, you're like if you could talk about that a little. Great question, Kristen. Yeah, community building. Anybody else have anything to add about that building the kind of community of the classroom to a tool like hypothesis. Yes, it's good for reading comprehension critical thinking but what about this other idea of community. Thank you. If I may share something directly from one of our back at the show and tell us one of our champions of my office. Teresa love me from Europe. We worked with pretty closely, you know, actually on a basic so she's going to back to the show and tell me, and she had to take a ways from her use of hypothesis because for individual and social innovation, very important. And of course she said okay, it's important so okay, use the instructor clear directions about what's the nature of the science, right, you know, interconnectedness from the science texts, right. There are so many highlights of the prize required, but her final takeaway really, really struggling so I will pull from it directly to see it's the importance of valuing classroom community in the social dimension of one. She loves this happening. And W and MW, no matter what protect or allow meeting confusion, share text or remote medical and build a collaborative culture of safety and respect. So she actually promotes this within the online class. That is great. Thanks for sharing that and also Leslie from the audience has said that she uses the group tool in D2L, and that is something that can help especially for teaching a large course. And, you know, there's lots of students in the course you can break those, those, those big classes into smaller groups so sometimes, you know, calibrating the size of the reading and annotating group can help nurture community because maybe in too big of a, of a course those those voices can get lost so that's a good practical strategy thank you Leslie. I want to shift to the final question here but kind of change it at the same time so the final question is about results. I'll let anybody who kind of prepared for that question to share but don't feel obligated because I do want to go on and sort of talk about like, what's what's the future look like for hypothesis at university says anybody want to talk about results it sounds kind of I want to talk a little bit about the qualitative response, but does anybody, anybody prepare or think about this particular and there's a question about efficacy from the, from the q&a we do have some studies that we can share about hypothesis and about social annotation and a variety of different contexts and efficacy. But in terms of results what would you guys say what are you seeing at this point. I would just go back to the idea if you have department chairs who are enthusiastic about it and promote this kind of technology with the faculty they work with. Those are the users I'm seeing time and again use it. And that's been really helpful so that's been a major takeaway at this point. And one of the other things that's things that's interesting is the departments that use it most here at Butler which would be histories or top department here at Butler that uses it. A lot of those faculty also teach in our core curriculum. So our core curriculum courses are really designed to kind of anchor the curriculum and the liberal arts so we'll have everything from global and historical studies to, you know, first year seminar and all those things so they've carried it over not only using it in their disciplines but also using it with students who might otherwise not be exposed to it in their discipline right through their core curriculum courses so I really like it if you can find those faculty who are who are with first year students or or maybe second year students in the core curriculum and getting them involved at that point in it so that even if they don't continue to use it throughout their educational career they can use it in the wild and get excited about using it. I think the other takeaways that I see, I see a lot of faculty interest. But I think there's this type of question around like how do you implement it. Right. How do you implement it well. What does good annotation really look like versus, you know, just kind of phoning and the annotation. How do you do a low stakes assignment versus a high stakes assignment what does a high stakes assignment look like and hypothesis. I think really relying and shout out to Becky George she's our CSM here for Butler amazing. I think really relying on more and more and we're going to be relying on Becky to help us in terms of getting messaging out to faculty getting resources out to faculty. So that faculty feel empowered and confident in their use of the tool. And then that way they can get to the point where they, you know, start to develop their own assignments. Also, I'd like to be better about getting faculty involved in like the educators forum. There's an educators forum and slack. And just part of the conversation so that there's a wider network. You know I had a faculty member the other day mentioned that it would be really nice to have a consortium of rubrics and hypothesis does have rubrics available which is nice. And then just leaning more internally throughout our campus and getting examples of rubrics and assignment prompts that we can share across faculty, and then Kristen unmuted to so I wanted to give her an opportunity. Yeah, I think it's also in addition to that. We've seen ties to some of like our larger strategic initiatives on campus. And particularly our low cost no cost textbook initiative, because hypothesis works best with like oh we are text open texts. There's some like new like J store integrations and things like that but we've had faculty, you know be interested in this and then have conversations with us or librarians about, okay, I want to use this. I want to find a new text that will work with hypothesis. And, and so it's, it's aligning pretty closely with our goals to like lower the cost of textbooks with for students. And in addition to that then like driving some student engagement as well around the text. I think also on our campus, we've, we've identified our first year students as one of some strategies around first year experience specifically. And this is one tool that we're considering and having starting conversations with that group, specifically about so thinking about right like what, what department should we target next that that was a big key one because institutions already moving that direction there's momentum that way. And so, there was a question in the chat like where's the data on the effectiveness of it. We're super thankful that hypothesis partnered with Indiana University and came out with that big study about the first year experience, because that's something that now we're able to share with our instructors and say, you know like just down the road. From Indiana. They've actually done a pretty extensive study and let's look at those results together and is this something that we should consider here. Yeah, and we can share the result. There's several publications that came out of that Indiana study and I don't just put a plug here that in terms of partnership in terms of collaboration one of the things that in my role and hypothesis as VP of Education, that I try to do is to look at the differences, whether it's around the surveys that Megan did butler, or something larger like the, you know, IRB sanctioned research that we did in Indiana so you folks on the call but also you folks in the audience like their opportunities for research or efficacy studies, big or small like let's get together. Right now I'm working with you to Austin on a, some work around a physics course that's been using hypothesis with a textbook which was a question that got asked in the audience here we do work with vital source as a delivery for delivery platform for e text. And we're seeing some amazing data coming out of that basically that back to I think Megan's point early on like there's a lot of students don't that aren't doing the reading that aren't cracking the textbook that vital source can see that when they look at the data and how many times people are opening the textbook throughout a semester throughout a semester, and when social orientation is part of it, it flips right so there's a lot more people opening the textbook not just opening it by using it throughout the term. I'm just going to go down here for these kind of final words then see your next. I wanted to say that the feedback that we've been getting from our faculty has been overwhelmingly positive. And one of the things that we have found is that hypothesis really fills a sort of a unique niche within, if you want to call it our technology well, but our suite of tools that we offer to faculty across our campuses, because it really encourages that collaboration and critical thinking among students that we really previously didn't have or was made available to faculty in the form of a tool that they had in their classroom. So that has been just absolutely transformative for those faculty, who are not necessarily asking their students to create video content, but need them to really think more critically about the kinds of readings that they are doing as part of their coursework. And I just wanted to say that we've also been very closely involved in the J store, as well as the vital source integrations within hypothesis we found those to be, you know, very positive opportunities for faculty to, you know, further extend their use of the tool. It has enabled our library and their liaisons to get more closely involved with faculty and how they use resources. So, you know, overall we're looking for more opportunities like that from hypothesis, just to continue to grow its, its operation and use. That's awesome then let's talk you mean Suzanne later about growing the vital source users because I'd like to share what we know what we've seen at UT and elsewhere. Portland Community College. Go off of what Vince said a lot of the feedbacks been really great with faculty and it is another great tool in our tool set as well we do offer a couple of others there in regards to like communication kind of creating that community in a course. And some other things with. I think with our kind of faculty show and tell that we've done. It's been a great place for our faculty to see how other instructors are using the tool in their courses and kind of using that as a way to implement it into their own course. And so the post training feedback that we get from our faculty in the situations are oh you know I want to change one assignment into hypothesis assignment in the future term because they can see the benefits of using it or they at least want to try to engage students in a new way using a new tool and hypothesis is you know constantly involving so that's something that we really enjoy seeing the changes that are being implemented the team hypothesis has been taking our feedback into consideration making improvements and we really kind of work with, as we mentioned before with Jessica and and the others to try to just you know kind of implement what we can so we're really looking forward to seeing how the tool evolves and how it can really be another great asset for our college here at Portland Community College. Thanks David Dana. Well, everyone has kind of said a lot of fantastic takeaways from their own courses so I'll just say broadly of course we, it's been positive at my campus to. I wanted to build off a little bit of what Christian, Kristen had hinted at which is, you know, using hypothesis with OER resources and a nice side effect of hypothesis has been making people more aware of what makes documents accessible since optical character communication is a requirement for hypothesis. I love the integrations with JSTOR and vital source fabulous but a lot of the readings that our instructors want to use our, you know, image scans of PDFs and teaching them why they need to make their documents accessible if they want to use this fabulous tool that they've heard about has been a really helpful conversation to get started with faculty. Yeah, lots of digital literacies attached to social reading not just the reading the text. Well, this has been a wonderful conversation. I very much appreciate all of you, your partnership but also the conversation today. I've got a few closing remarks before we say a bit of do. I want to mention if you're in the audience and you're not yet partnering with hypothesis to reach out to education at hypothesis there is going to be a fall back to school with hypothesis so again if you're not yet a partner please reach out to education at hypothesis and we can tell you more about the discounts we're offering to get started with the fall. If you're our partner and I'll just stop and say, you know, our CS team was shouted out many times today the customer success team I'm very proud of the work that they do. They're a key part of this conversation and part of the work that the that our guests are doing. And there's a wide range of very valuable resources that they are responsible for creating and sorting. One of those are our partner workshops of course if you're a partner, we do customized workshops for your campus, but we also host partner workshops that are open to all customers. And you missed annotation starter assignments but there is a recording of that one that you can access on YouTube. And then there are three more workshops as part of our summer series. There's one on creative ways to use social annotation one on using multimedia and tags, again to deepen in the use of some of the features and then one on grading and feedback and that's again it's one of the neat things about hypothesis. It's a simple tool, definitely takes you know, a little bit of learning to onboard but really the work is around the pedagogy of it. And I want to go back to one thing that Megan says was that said was about, or made me think about was, what is annotation look like in the first year experience program. What does it look like is one develops into different disciplines. What does it look like across those different disciplines it means something different at all in all those places and we're still doing the work this is still in biology. Our collaboration is, you know, we're learning about from the you guys what does this mean in biology what does it mean in upper level biology what does it mean for grad students what does it mean for a big entry level biology course. And I'm very excited to continue to build out that thinking around what social annotation looks like at different levels, and in different disciplines and really welcome collaboration around that. What we're going to share before we close is hypothesis Academy. Christie to careless who's been the co host here today, haven't had a lot of questions from the q amp a so she's been a little quiet, but working in the background to answer folks as questions. She runs hypothesis Academy she invented hypothesis Academy it's an asynchronous training course two weeks long. It's a really great way to dive into the tool with some guidance from our team but also in collaboration with folks from other universities and colleges who are also going through the course so we have a kind of introductory course. Social annotation one on one picks off into this month next cohort. And then we're also starting next week. Our first, you know, course on social annotation and artificial intelligence. It's a new course, obviously a new topic. And if you have folks at your campus that are interested, please, please let them know. And with that I'll just say, thank you so much guests for the conversation they really really great to hear your stories. And so thrilled that we're partners and we can continue in this collaboration and conversation, moving forward. So thanks for your time today and be well. Thank you. Thanks so much, everybody. Have a great afternoon.