 including an asynchronous E.T.D. pre-check serving the off campus and the time for sharing this information with you are Kristen Terrell, Lily Compton and Thomas Elliott from the Iowa State University Graduate College. My name is Terry Robinson, and I will be your moderator for the next 45 minutes. A quick reminder before we before we begin during the presentation portion. Please keep your audio and video muted unless you are presenting or you are asked to participate by the moderator. You may use the chat tab to ask questions, which will be addressed during the Q&A portion. All sessions will be recorded and available from US E.T.D.A.'s private server shortly following the conference and all videos will be edited and posted for public availability on the US E.T.D.A.'s YouTube channel by the end of December 2023. Thank you all for joining us and I'll turn it over to Kristen, Lily and Thomas. Hi, Terry. Can you hear my voice? Okay, great. Also, Siri heard my voice, so that's not helpful at all. Hello. Good afternoon. Thanks for coming to this presentation. The topic is our pilot project of a new asynchronous electronic thesis and dissertation course that we put together using the Canvas learning management system. I'm Kristen Terrell. I work as a graduate student services specialist at Iowa State University in the Graduate College, and I'm co-presenting today with my colleague Lily Compton, who also works at the Graduate College as a program coordinator and Thomas Elliott, who's a graduate assistant in the Graduate College. So what motivated our development of this Canvas course was the problem that we were seeing some students who were submitting electronic thesis and dissertations that clearly were not well prepared for publication. And there are a lot of resources available at Iowa State University for graduate students preparing electronic theses and dissertations. So we kind of tried to hypothesize about why there were many cases of actually lack of preparation. And we identified that some students might not be aware of what the format requirements are. They might simply not realize that there are format requirements dictated by the institution. We also assume that some of them don't realize that there are templates and format checklists available. They simply don't know that they're there. And then we also hypothesized that many students don't know enough about using digital editing technologies to effectively meet those requirements, even if they are aware of them. So some of the most important technologies that are needed to produce an electronic thesis or dissertation are Word and Overleaf LaTeX, and also Adobe Acrobat if you're doing any kind of post-processing on your PDF. We try so hard to get the word out and to support students in many different ways. We have a lot of consulting available. We do, you know, boot camp events to massively reach out to a lot of students at once and to make sure that they know not only what the requirements are, but how to do them using the available technologies. We've got a, you know, very detailed web page with a lot of downloadable resources. And so, you know, based on what we were seeing, we had to hypothesize that some people simply were not participating in those events. And part of the reason might be because they can't. So Iowa State University is a large university and we have a lot of graduate students. Graduate students are farther along in life than undergraduates. They're more likely to be off campus. They're more likely to have families. They're more likely to have jobs and full-time jobs that may get very hard for them to show up for things like a boot camp that may take place, you know, at a time they're simply not available, or they might not even live in our state. And so they might not be physically able to go for that reason. And so these reasons for lack of preparation, we thought, motivated something that was a little bit more accessible to students who were not able to take advantage of the existing resources. What we ascertained was that there are some students not getting the right information at the right time. We have a lot of orientation for graduate students at Iowa State University. We've got graduate college orientations and each department has orientations and a lot of clubs do it. And they're basically all for incoming students. And those are not students who are working on their thesis or dissertation for the most part. And even if they are, it's years away. And so they probably forgot what they learned by the time they get there. Thesis and dissertation work requires intense focus. Anybody who's ever written a thesis or a dissertation knows that you're not exactly wandering around campus looking at flyers and trying to think of something to do in your free time. You don't have any free time. And so you might not be engaging without reach. You might not even really be checking your email because you're focusing so intensely on writing your thesis or dissertation. We also learned just from interacting with students that experienced students who have been at the university for a long time over rely on their network. They're talking to students in their lab. They're talking to their major professor, their supervisor, and the information that they're receiving about electronic thesis and dissertation requirements is outdated. Essentially, this can be summed up with the problem of a student downloading their colleagues the thesis because their major professor said they did a really good job. And that thesis was from like four or five years ago. And we've changed the requirements since then. So they're looking at a model that doesn't reflect the current state. And so this is actually a pretty, pretty recurrent situation that we run into a lot. And it comes out at the very end when we send the student a request for revision and they say, Why do I have to do this? Look at this published dissertation that looks a certain way. And then we have to say that's not the standard. You know, that's actually just an old dissertation that doesn't meet the current requirement. And then the other side of the problem is, as I mentioned, synchronous on campus services like a boot camp that we host at the library and a computer lab are not accessible to remote students. So distance learners or all but dissertation doctoral candidates who are working while completing their dissertation aren't probably going to be coming to an on campus boot camp to learn about Microsoft Word. And so that service, however useful it is for the people who take advantage of it isn't helping those students at all. So we thought through, you know, what can we do to get the word out. And what we decided was to use Canvas, the learning management system. This table that I'm showing on the screen right now highlights some of the features that Canvas courses offer and how those can reflect affordances for our specific situation. One of the features is that it allows you to selectively enroll certain students. So whereas the orientation services that kind of cast a wide net and are targeted to all the students starting that year. This Canvas shell allows us to only enroll and only target the content to students who are getting ready to graduate and who are actively working on their thesis or dissertation. So that means that we're not wasting a lot of outreach on students who simply don't need to know that right now. It also allows for cohort grouping. This is really useful because you can you can sort of apply this this attribute to any given student enrolled in the course and it allows you to reuse the course for multiple groups. And it also allows for targeted communication. So because you can group cohorts into these into these smaller sections, you can send out announcements that are really only relevant to students graduating this semester and not pester the other students. Canvas has a modular format, which is somewhat different from the website that we already had. It allows for presentation of information in a logical chronologic structure. And when I show you the course that we designed, you'll see how this is very different from this the website that we offer that, you know, houses most of our thesis and dissertation resources. Websites are really useful for people who have a sense of what they need. But of course is much more useful for somebody who may not even realize what's out there. So you can introduce things in more of a chronological way and require completion of one module before they even start the next module. So this is a good way to prevent, for instance, somebody going straight to the template before they look at like the graduation schedule. You want to get all the information to them so that they can make sense of it. Canvas courses also offer assignments and this is a little bit more of a stretch because an assignment for a Canvas course what you think of is a teacher with a classroom full of students giving them an assignment and then grading them and then that's like an assessment of how well they did at learning a concept. We used assignments in a little bit of a different way which was just to do two-way communication and file sharing in a controlled environment. So of course there's many ways to share files and many ways to do two-way communication, but if you're relying on things like email, it can be very hard to know who's looking at it or when the time frame is for it or how many people took advantage of the service. Using an assignment to control how our electronic thesis and dissertation students took advantage of interacting with us through this course allowed us to keep really good records and it also allowed for a certain degree of accountability between us and the students. So we did use the assignments in probably like an off-label kind of way, but it's working very well. Canvas also allows you to make announcements. I kind of like this as my favorite feature because I feel like graduate students get so very many emails and it's very easy to ignore them. I recently graduated from my graduate program and frankly sometimes when I saw things that were coming from the university I would flat out not pay attention because I had to write my dissertation. Announcements come from Canvas and because students have this relationship with Canvas where that's where their classes are, it's a little bit more attention grabbing to get a message from Canvas that there's an announcement versus an email from the graduate college. So I really like to be announcements as a way to get their attention. And then the last feature of Canvas that we took advantage of was the discussion board. If you've ever taken an online course, I'm guessing your assignments included having a discussion where you probably had to respond to your classmate and do this like critical thinking exercise. We use the discussion boards again as an alternative to email and we're still using email. Of course, students can always email us at the graduate college. With the discussion board, they can ask us that question that they ordinarily might email and then everybody sees it. So if many people have the question of, I don't know, like, you know, it says the deadline is November 30th to submit my dissertation, but what if I have revisions? Am I not going to graduate on time? Well, that's a very urgent question and probably one that a lot of people are concerned about. So because we have this Canvas course and because we have discussion boards, instead of getting like eight or 10 or 20 emails with the exact same question, now everybody who's graduating that semester can see it on the discussion board and maybe not feel like they need to ask it to. And it in that way also serves as a repository of relevant information. So it's a way of sourcing the burning questions that people have in a way that then, you know, allows future students to see the answer even though, you know, they may not have so the email themselves. So to implement this, we did have to put together the ETD Canvas course and it was a big task. I've used Canvas a lot as a teacher and actually as like a worker at the Center for Communication Excellence at Iowa State University. I'm very familiar with this tool, but every time I use it, I learn something new about it. It's not a simple tool to use. And so I don't want to send the message that we just kind of like made a Canvas course and it was great. It was a lot of work and it took a lot of thinking. So one of the things that was a big challenge was adapting our existing materials into a chronological modular structure. So we have a very, very complex website and I think it works really well. We have essentially a hub for thesis and dissertations and it links to things like graduation requirements. It links to planning tools. It links to video demonstrations of how to use the technology. It links to our templates. Those are all links. So this is like a tree linking HTML type of document and it doesn't necessarily work for somebody who doesn't know how to graduate. It works pretty well for somebody who's like walking through it with a graduate and it works pretty well for someone who knows what they're looking for. But we then had to think through it from the perspective of somebody who doesn't know those things and how would they use a course in contrast to using a website. So we broke it into three modules. The first module is called preparing your thesis or dissertation and that's everything you need to do basically before you do your thesis or dissertation. Drafting and formatting and then finishing and publishing your thesis and dissertation. So each one of these modules follows from the previous one and you have to complete each one before you can move on to the next. And then another thing that we did was compiling links to available resources. A lot of those were links to our own internal website but we also took the opportunity to include links to related websites that we haven't yet integrated into our website. And that might be things like the digital accessibility services on campus or the library where they have all kinds of information about plagiarism and copyright. So that's really useful because it's something that you know it would be redundant to have it on our website but students do need to know about it. So this Canvas course also gave us another opportunity to like put that in front of students without having to like try to reconfigure our entire website to include it. And then for every resource we had to then compose a short and user friendly explanation for the resource and explain why is this here and what is this have to do with you as a graduate student or your thesis or your dissertation. And that was kind of a lot of work to compose those but well worth it. And that's another thing that we don't really have on the website. There's parts of our website that do have explanatory content but a lot of it is just like a label and a button. And so if you don't know what it is then you have no reason to click it here in the Canvas course we did have a much better opportunity to give them a little blurb and say hey like here's why you should click here this is what it has to do with you. So it's really worthwhile to put together that content. And then also we wanted to practice what we preached and exemplify some of the requirements that we are expecting them to know how to do. And so as I was putting together the Canvas course I was trying to be very cognizant of putting in the alt text and using the heading styles for digital accessibility. And using responsible attribution practices for all of the copyrighted material that I was including. And that was just a really good exercise for me and it really helped me understand what that end user is going to be experiencing when they're putting together their electronic thesis and dissertation. And they're trying to figure out like how do I provide attribution for a picture this is you know creative comments it should be free. Well you know now I've done it so now I'm more capable of helping them do it and that that was a really good experience. And then also within the implementation I mentioned on a previous slide that we were able to use the assignment feature of the Canvas technology to enable two way communication with the graduate college. And I kind of want to take a step back here and give a little bit more context to what that assignment is analogous to in our in our other services. So at the Center for Communication Excellence one of the services that is provided that's very popular is format pre checks. And what that amounts to is a one on one consultations very similar to a writing consultation at a writing center where a graduate student will meet with a peer another graduate student. Either face to face or on WebEx and they'll show them their dissertation and that graduate assistant is trained and has basically memorized the full list of format requirements and they will pre check that students thesis or dissertation and anywhere where there's a deviation from the required format. They'll show them how to meet the requirement in Microsoft Word. So this is this is a very synchronous and very responsive and very interpersonal service that you know to some degree is not accessible to everybody, especially those who are off campus and especially those who are working and don't necessarily have time to set aside half an hour to meet with a peer and walk through their thesis or dissertation so the challenge of creating an analogous experience in a Canvas course was not necessarily that straightforward and so when you imagine this assignment. Think about what that would be replacing for a student who can't come in for a one on one consultation. The one on one consultations by the way are a total game changer and I realized that Iowa State University is very lucky to have something like the Center for Communication Excellence that has this robust system for training graduate assistant peer consultants. Not everybody is going to have that but because we have that it saves the end format checker. Our literally hours per hour per student because they've already been through a pre check so it's a wonderful service and I that you know essentially what motivates this Canvas courses I want more people to take part in it. Because I'm the final reviewer and I love it when I see one that's like really close to to compliant and it's it saddens me when I see one that's really you know really deviant so we wanted to make this this format check available and more popular with a broader group of students. So like I said, we created an assignment in our Canvas course and just going to click here and show you how that worked. Oh it's not letting me I need to like get out of. I need to get out of display mode. All right, I'm going to stop sharing because I don't need you all to see my username. I don't know how I did not anticipate that this was going to require a login. What I'm sharing right now is the Canvas course so you can see if you've ever used Canvas this probably looks really familiar to you and this is my teacher view so it's got a lot more buttons than the student happens to see. But it starts with this explanation of what they're looking at and this is really key because I invite them the day or like the weekend after they apply for graduation. Most of them have no idea this is coming and if they're doctoral students they're also probably not enrolled in any classes so they're probably thinking why am I getting a canvas invitation. I didn't sign up for this class it's the middle of the semester so I'm trying to give them a little bit of a like ease them in and say hey you know this is what this Canvas course is you applied for graduation during the current semester. You know we're the Center for Communication Excellence. We want you to have access to this information and this is also a good way to get announcements. So it instructs them on how to participate in the course it says you have to complete each module. You can complete them one by one here's some links to the module and then here's what you should do if you have questions and I just want to note that I'm not giving them out my email address. Not because I don't want them to email me but because I want them to put their questions into a place where their peers can also see the question and the answer so that that that information gets distributed widely instead of only going to the one person who asks the question. So this is the overall Canvas course and then I'm going to navigate down to the asynchronous format check assignment. So I give them these detailed instructions and we have it sort of lined up to happen between when they deliver their finished written like thesis or dissertation to their committee but before they do their oral defense. So that what the the checker is actually looking at isn't like a rough draft it's the final draft this kind of helps reduce the amount of time we waste on format checks. When we talk about the in person ones there's no reason why they have to be finished before they come in for a format check they can just ask their questions or get the information from the get go. But for this asynchronous because it doesn't have that immediate interpersonal response aspect. It seemed like a good idea to just have them do it just with their finished thesis or dissertation so that they weren't wasting a lot of time doing repeat checks. So they would submit their thesis or dissertation just upload it to Canvas and click submit. And then it says within approximately a week you receive feedback listing any revisions necessary for your thesis or dissertation to be accepted in pro quest. And then it says make the revisions listed before submitting to pro quest and if you need help you know you can schedule an in person one. And then the last thing is you may resubmit the file for a recheck if you would like so the idea is here that they're they're submitting for a precheck and that hopefully they're going to get that list of revisions. Very comparable to what the final format checker would provide them if they had just gone straight to progress. And then of course we relink to all of the requirements so we relink to the requirement checklist the formatting examples the tutorials and the templates and these are resources that this student by the time they get to this point in the course they've already seen it. But it's just a reminder of where to find it. So those were the instructions for the students. The other challenge was actually making it possible for the consultant format checkers to do this to work with canvas. So again this is an assignment that when they designed this interface they were thinking about students in a class and a teacher who might you know if you if you submit a written paper. You might get some like marginal comments and maybe they'll like write you know a comment in the in the feedback block. But what we're talking about is something that's much more intense because we're talking about giving detailed feedback on potentially up to like a 250 page dissertation that links to specific page numbers where there might be a specific you know divergence from the requirement. And so the feedback options that were available through canvas really were not adequate and we tried a couple of things like we tried using rubric but it simply doesn't work so what we what we landed on was this. I think the word convoluted is strong but applicable procedure for giving feedback and it actually relies on SharePoint so they had to leave the comfort of the canvas ecosystem and go out into the Microsoft SharePoint ecosystem to create a document where they can provide that very specific feedback. And then what they would do is create that document store it in SharePoint. Find a you know generate a link to it using the SharePoint technology and then they would just paste that link into the assignment feedback area. And so that allows them to give very detailed feedback in a very concise way. And I just want to show you the SharePoint site because I think this is really kind of like the the the most useful thing that we found and it's it's a little bit like I said convoluted but it's actually really useful so. With SharePoint you can create a template and that template will have like preset content into it so I know we've a lot of us have worked with templates before but if you're thinking about like a thesis and dissertation template. It's not pre populated with a lot of content it's just got like content areas where you can fill them in. With this SharePoint template it is lined up to be essentially it's lined up to be reflective of our checklist of requirements. So it's totally populated with content and the reviewer would just go through and anywhere where they saw a problem with the margins and fonts. They would list that here anywhere they see a problem with the page numbers they would list that here. And then they can save this file and they can link to it and send it back to the student in their feedback. And I don't know how well I'm explaining this I feel like it's like I said it's a little bit convoluted, but what it does is it allows for a reviewer to give very detailed feedback in a way that's very easy for the student to follow and understand. Without demanding that that reviewer essentially write like a five page report from scratch to send to the student and so we can give them extremely detailed and extremely organized feedback without taking like hours and hours to do it. So those are the instructions for the reviewer and there is going to be time for Q&A. So let's, you know, talk about your questions and answers. Okay, so now I'm going to talk about how it went. So we started piloting this in fall of 2022. We enrolled 248 students. 11 students participated in the asynchronous format checks, and they submitted 16 submissions so three students did rechecks. They got more than one. Excuse me, I just scrolled away from the slide I was on by accident. 96 students viewed the content at least once which means that we got really good participation. And a lot of people were, you know, seeing what we were putting out there. And then 62 of those viewed the content after the first week they have rolled so they didn't just log in once and go okay great now I know that's there and then forget about it they actually engaged multiple times. We learned a couple of lessons. One was don't enroll people who are doing a non thesis masters degree because they don't want that canvas course and then I got like 100 emails asking me to unenroll them. And then I also learned that adding students to any individual students to groups. Remember I was talking about how you can group them by cohorts is actually very tedious. And so what I learned is that you can actually download and add on to canvas that will like automatically do that but you have to, you have to, you have to dig for it but it does exist. Onward and upward we're still doing this. We are enrolling it seems like between 200 and 300 students for the fall and spring semesters fewer students. We had a graduate in the summer but we do still have a good amount of those students are engaging the pilot semester we only got 96 but spring we got 158 and summer we got 52 so we are seeing kind of increasing student engagement. And we're also seeing, you know, pretty pretty reasonable use of the asynchronous format check it's not a lot. You compare that to the number of students who are participating in that like synchronous person to person format check which tend to be in the three digits. Honestly, I'm still happy with these numbers because with that canvas course it's pretty much like we don't have to do any work. But the canvas courses built I just enroll the students and then the format checks are pretty much the only engagement that we need to do with them so even though we're seeing small numbers we've also got a really low cost to reach those students so I still feel like it's a pretty good participation. The lessons that we've learned are about targeting enrollment. We do want to be flexible about how we use this canvas course. Currently we are targeting students who are currently or who are graduating in the current semester. And we're actually narrowing that only thesis and dissertation students that leaves out some students who might benefit from having the information so we're kind of revisiting this regularly to try to understand who should be enrolled so that we don't pester them but we also do make it available to those who need it. The technology that canvas offers is not meant for this kind of application, it works for it but you do need to adapt it. So that solution that I found to the problem of the tediousness of grouping students I had to like go to a community forum and download a solution. And then another thing we learned is that it's very important to talk to the stakeholders and to talk to your colleagues about what you're doing. But it's also a good way that you know the canvas itself allows you to communicate with stakeholders because you can use the announcements to get information out there to those who need it. So with my presentation, I do apologize it took a little bit longer than I'd hoped so we have somewhat less time for q&a but please do let me know what questions you have about what we're doing. Thanks Kristen I think Lily's address most of the questions that have already come through chat but if anyone else wants to type anything in please do so now. I do want to bring up a question maybe to everyone who's been in here from the morning sessions we talked about how canvas can be used for digital accessibility checking right because there's a function in there or tool in there. So those who have used blackboard or canvas for similar reasons. What do you think about mandating a pre check for digital accessibility in here. So, mandating is always the bad word right because we cannot mandate but if digital accessibility becomes a mandate, can we then force students to actually submit a one time pre check and make sure they do the digital accessibility checker in here, rather than the other general formatting stuff. And then also, I think Kim had a question about plagiarism. We have not done it, we have not actively done anything with plagiarism, unless something funny comes through and then we kind of research it and that's an whole entire issue that we ran into with authorize reuse text reuse of self publications right so let's go back to the question about using canvas or blackboard or some other way to require a digital accessibility checker. I mean, for my institution where it is going to be a mandate by 2026 what do you think. It would be nice to see people's thoughts in the, in the chat. So, Angela, yes, we do have the resources on our website as well. And like Kristen was saying, it's all there but not everybody goes there so this canvas is just an alternative method of targeting graduating students that we don't leave anybody behind. So, a lot of students still go to our Center for Communication Excellence website to get all the resources to register for the synchronous or on campus events. But this canvas course is just another way of reaching out to students. TMS asks how overwhelming does it get with the format checks. Is there a time we get more slam absolutely. Tom was is our grad student. We have two grad students who do pre checks, they, they work 10 hours each per week. And then Kristen is the one that does the final approval in ProQuest ETDA. And then why we have a differentiation between pre checks by grad assistant versus staff for final approval is that we don't believe a grad assistant can do approval processes. Kristen is the one that approves, but when Kristen is busy approving or trying to get through 250 files per semester in a timeframe of two to three weeks. So when there are issues that some students have and they have too many issues, then we can say hey go book an appointment with Tom, or with other, our other consultant, and also in that last three weeks we set up walking clinics as well and we can sign another consultant to about 10 hours or more for anybody who can't get an appointment, they can just come into the office and get help. Okay, so let's see, I think, I think we, it's always a challenge about pushback so that's something that we need to think about as well. I was just saying too, it's very useful for remote and distant campuses so yes we are discovering that there is a need for asynchronous one in fact we have a thesis and dissertation writing student who requested for asynchronous after summer. Okay, I think we have four minutes left. I hope I got all the questions. I don't see any new questions coming in. Okay, thank you Lily. So we're going to have I think a 15 minute break and then we'll have two concurrent sessions. One will be in a breakout room that you can choose to join or you can stay here in the main room for the other. Thank you.