 Hi everybody. Thanks for coming. Thanks so much everybody for coming. This is awesome that you're really excited to get ready for the semester start. I know we can hardly wait until the 21st. And make sure that you, yes, continue enjoying your lunch. There's cupcakes for dessert. They are chocolate chocolate. No blue grape this time. Homemade by you. Homemade by me. They're an old family favorite recipe. Mrs. Moore's chocolate cupcake. Make sure you sign in on our sign-in sheet. You can do it on your way out. I just want to know who came and if we had any follow-up information, impressions or something that came up today, we can quickly choose everybody who came up this year today. So that's why we are interested in who came. Today's session is called Kicks Off Our Academic Year for the 2017-2018 year. And we're talking about academic integrity. We're hoping this is going to be a two-part session. First part, where we're going to be talking about with Alan Kimmel, who is a system professor and associate head of undergraduate studies in the department of material science. I'm in engineering. He's going to start us off and we're also going to hear from Stevie Rocco, who is assistant director at Dunn and April Malay, who's learning designer in Dunn. We're talking about ways that we can try and prevent and motivate students not to cheat in our courses. And that's what our goal is today. We're really hoping that we can look at what does it look like to cheat. We understand all the ways that what cheating is. And then also looking at what can we do just easily to prevent cheating. And then next month we're hoping to talk about what happens if you do have people that cheat in your class. So any questions that you have about what do I do with that? Save those questions for next month and we'll address those. Anything that I may miss? Okay. We're going to begin first. You're first. So Stevie is going to start us off and look. April did these slides. Yeah, go. We are recording this session. Make sure that I mentioned that. So before anybody says anything, they wouldn't care. Right. There are several people that ask for the presentation. So this is being recorded. Thank you. April did these slides. Full disclosure. So I just need to give credit where credit's due because if I didn't, it would be cheating. Academic integrity. This month we're talking about prevention, about how to limit opportunities for cheating. And then next month is again about policing, catching and punishing cheaters. I went the wrong way. Let's go this way. So there are five types of cheating that we deal with. This is information April found, plagiarism, that is taking someone else's work and claiming it as your own, either accidentally or on purpose. Fabrication. Fabrication is creating data and facts, things that you might need to support your argument, just making it up whole cloth. Falsification. That is data driven. Data driven. Okay. And then misrepresentation. Misrepresentation is finding something that works for you and making it work for you and it has nothing to do really with what you are supposed to be doing. And then misbehavior. That could be almost anything nasty, anything in a class that's disruptive. Okay. So I'm going to start by talking about last year's university level academic integrity committee. Many of you may not even know that this committee existed, but I was on it. There was a university level. It was called the proctoring committee, but it, you know, it's so tied into academic integrity. We ended up doing a university wide survey in last fall. You may have seen it. You may have even filled it out. If so, I thank you. It was sent to all colleges and campuses. We had 1700 respondents take the survey. There was big interest. 24 different locations. 55% of those respondents were from University Park. The second most responded from campus was Harrisburg, which kind of makes sense. It's a larger campus. Behind that, it was Addington, which I found fascinating. Like we had 60 people from Addington fill out the survey. So clearly there's a high concern there as well. We had, we collected data on over 1900 classes in that survey and the results were kind of interesting. First, class size. You can see that we mostly had class sizes in the 11 to 30 range and the 31 to 50 range. So if you say 11 to 50, that's over half of the responses there. And then 51 to 100 was this purple. And then the 12% is 101 to 300. Very small slice of over 300. That makes sense when you think about only University Park is going to be reporting really large classes anyway. So makes complete sense. So 59.4% were between 11 and 50, 35.6 over 50. Mostly it was undergraduate, like 86% of the responses were about undergraduate classes. Some small slice 4% were both undergrad and grad and then a small slice of graduate students. Um, when we looked at the Pollock Testing Center, this was interesting. The number of University Park faculty who responded that they use the Pollock Testing Center was really small. Only 51 of the 904 people responded that they use it. But 850 people responded to the follow-up question that asked whether they haven't used the testing center because they couldn't book it. Right? So of the people who said they haven't used the testing center, 14% or 110 people said they don't use it because they can't get in. It doesn't, they can't get booked because it's always full. So even though that seems small, that indicates a fairly large number of students aren't figured out, aren't being served because of the testing center is booked. So what we did was we assumed that the enrollments were in the middle range of each of these things and for over 300, we said it was 300. And so we said, okay, if 14%, we took the 14%, we assume only one exam is given per year, right? And we took the middle number of enrollments, that means that like 26,000 exams are not being served by Pollock. And that was only based on the respondents who answered that question. So I personally think that this shows that we need another solution because the Pollock testing center is not serving enough exams for people who need them. Then we asked if you would be likely to try an online proctoring system. And of those folks, 1,480 people responded and 66% said that they were at least somewhat likely to try such a system and only 34% said they wouldn't try it. So most people seemed that they would at least try it once. Trying a proctoring system or an online? Trying an online proctoring system. Yeah. Then we asked about their concern about academic integrity. And then we figured that out by like a bunch of different things. So nearly half of all respondents indicated a moderate or stronger level of worry about academic integrity. And over 85% of faculty members teaching classes of over 300 students were at least moderately worried. So I mean that makes sense too, right? The larger the class, the more you might worry about academic integrity. Then we asked about their concern by discipline. And this was interesting. People with a moderate concern are higher and our college kind of spans disciplines. Engineering and natural sciences, 53% indicated a moderate concern or higher. Formal sciences, 51% had a moderate concern or higher. And social sciences, 50% had a moderate concern or higher, which was interesting. This is also in a report that I can share if anyone really wants to get into the nitty gritty. What's really interesting is then we asked them about the confidence they had in their own assessment strategies to prevent academic integrity issues. Engineering, you remember they were concerned, right? 81.6% of them are moderately or fairly confident in their assessment strategies preventing cheating. So even though they have a moderate concern about academic integrity, it's clearly not happening in their classes. And the same was true for every other discipline. Engineering, 81.6%, natural sciences were 80% confident. Formal sciences, 81.6%. And social sciences, 82.3%. So I don't know if that means that people are really sure that their students aren't cheating. I just know that their level of concern about cheating and their confidence in their assessments, there's a mismatch in that. The really sad thing there was this answer, I've never considered whether or not my strategies help to prevent cheating. So there's a large number of people, 7.9% of people in the arts, maybe that makes sense because of what they do. 5.2% in education. You'd think that they would be, right? So I don't know, I just find that fascinating as a thing. So to move on, the committee recommendations were these five things. We said that the university should adopt a university-wide online proctoring system at the enterprise level. You may have heard that examity was approved for use by anyone at the institution. It was moved through World Campus, they were currently paying for it. So if any department wants to use it, that's fine, but the department gets charged for the use of examity. It's not really an enterprise-level solution. And the cost for examity is somewhere between $12 and $17 per exam per student. So it is not a cheap solution. I have a question. Can you give paper exams with an online proctoring system or only computer exams? Because I give, I mean, paper and pencil exams. That's what you do. And that's how my field does it. So if you can't write down things on paper and pencil, pointing and clicking is fine. They get that enough. But I mean, it has to be an in-class proctor. It might have to be an in-class or in the Pollock Testing Center, but there would be certainly much more space if other people didn't need it. We did this, yeah. I mean, I just did my in-classes and just online exams. Right. I could see how it went in-track. Yeah, proctor-track could do an online exam just because of the way their algorithm works, but even a paper exam. You think? Yeah, absolutely. Okay, all right. We also recommended that there should be a broader dissemination of other tools that instructors can use, and I'm going to talk about that in a second. We recommended that the university do a regular review of academic integrity tools and disseminate that to university faculty members. I found out a lot of things that people didn't know about. One person who'd said, and I was diving deep into the data, one person said they give paper exams or they give papers a lot. And then when I asked them if they thought there was a question on there that asked them about, do you use Turnitin to check your papers? And the open-ended response was, I never thought about using Turnitin to check my papers for plagiarism in my class. So I thought, well, okay, teachable moments, somebody now knows they can use Turnitin. So that's good. We want to provide better guidance around study sites. People have heard of Study Live and Course Hero where students will upload completed tests, completed homeworks, completed labs, and other students can pull those down and use them. We feel like we need to give better guidance around those. And we suggested developing a syllabus statement and templates for removal of materials from said sites. If you put an exam, even a completed exam, if it goes on Course Hero, that is your slash Penn State's intellectual property, depending on the circumstances around that class. They do not have copyright permission to upload that material to Course Hero. So you can issue a takedown notice. Is this going to be part of the presentation later, like what is Course Hero? Because honestly, I don't even know what you're talking about. We'll talk a little bit about that in a second. So tools and services to help. So first of all, we have a university license for Turnitin. It is integrated into Canvas. You can create an assignment in Canvas using Turnitin. If it's a paper or an essay or some kind of a writing assignment, students upload that assignment into the Turnitin interface from right within Canvas. Turnitin checks that thing for plagiarism. It flags it, and then it's up to the instructor to decide, does this rise to the level of plagiarism? Was this an accident, et cetera? One thing that a lot of faculty members don't know is that you can actually set up in your class Turnitin so that students can upload their paper before they Turnitin to check themselves. So students can use it as a tool to say, before I turn this in for credit, did I accidentally plagiarize? And in those cases, it doesn't go into the central Turnitin database. So students, it's a good check to set it up that way as well. Also, it might encourage them to not do things at the last minute if they want to upload it to Turnitin and get a report and then have a chance to revise it before they actually Turnitin. I actually took a grad class where we were required to do that and just submit evidence that we did, which again meant we couldn't wait till the last minute because we had to, like a couple weeks before the thing was due, show that we couldn't draft it. And then Turnitin, which I thought was a nice strategy. Another one I did not know until I was on this committee and that this probably won't affect most people here. Moss is an open source plagiarism detector for computer code. So it's used a lot in engineering and IST. It's out of Stanford, I think, but you can actually upload it and it'll know if somebody, you know, if they were supposed to write a program on something, they'll know if they pulled code from somewhere else. Again, use the testing center for quizzes and exams, monitor course hero and other sites. Course hero, it builds itself as a study site. It says, get notes from other students. Search for a course at Penn State. If you look for your class, 10 to 1, there will be hundreds of copies of exams, labs, papers students have written because the way they get free entry to course hero is it's an upload or pay. So if you upload materials, you get free access to the site for so long and then you have to keep uploading things. Many students we've discovered, and I don't want this secret to get out, but many students use their Penn State access account user ID as their ID in course hero. Okay, but that makes them really easy to find. They're not supposed to use their password. They're not supposed to use them. Like I'm not using. Oh, it's a course copyright. If the student actually took notes in your class, they own those notes. If they want to upload those to course hero, that's perfectly legal. Okay. If they can't take, if I put a power point, that's the university copyright, they can't do it. So why doesn't the university pursue them? The university pursues one. We had this discussion a couple years ago, actually. If the university, this goes back to the intellectual property thing. If the university owns the intellectual property of the class, it's an online class that you developed in conjunction with the Dutton Institute, then the university will issue the takedown notice. If it's something you did on your own, like it's a regular class that you teach and those are your materials, then the university expects you to issue the takedown notice. There is a template letter for that, which we have at this college and can be distributed again, but it's basically your standard wording for takedown. And they're pretty fast about taking things down, but it's an arms race. As fast as you take stuff down, students put it back up. So the best option is change things up often enough, so it's not going to help. Does the takedown notice go to course hero or the student? Course hero. Yeah. A question from online. Someone is wondering if faculty can access Course Hero without paying to hand over. Yes. If you contact them and tell them that you're an educator and you need to check for copyrighted materials on your class, they will give you a free account. Yeah, so you do not have to pay. Yeah, because I feel like kind of strongly that I don't want to pay them for what I feel is encouraging bad behavior. I think our college has a statement, a standard statement, that we've got our syllabus on our back desk site. Yes. It is a required statement. The copyright statement that's in the required syllabus statements, that's meant to cover course heroes. So maybe going over that in class the first day would be a useful thing. Yeah. But if you do discover that a student has uploaded a bunch of stuff for your class. Yes. Things like that. There's no action you can take against that student. I consider it facilitating cheating. It's on your syllabus that it's not allowed. Yeah, you absolutely may. Yeah. It's not going to take it as long as you're up. Yeah. Yeah, we actually successfully prosecuted. Yeah. Study Lib is a new one. That seems to be a new site. It's very bare bones. I'm trying to go really quickly. Study Lib.net. Study Lib.net is another new one. There is a preventing academic integrity issues website, the Dutton Institute manages. It's on our FACTIV site, so that factive.e-education. I send out links to that all the time to people. If you go on there, there's a whole page on preventing academic integrity issues and April's going to be talking more about that. And then the other thing you can do is request a consultation with one of our learning designers. We do that fairly often. It's not a big deal. You don't have to be working with us for any other reason. You can say, Hey, I have this assignment. I've had issues. I'd really like to talk to somebody about how I might reconfigure this assignment in order to do things. And if you want to do that, just email me and I'll set you up with somebody with some time. The report was submitted and accepted. There is now a new committee because the university called the Academic Integrity Committee and a lot of higher ups are on that. They have the report and the survey results. And I've actually also given them some more of the raw data on their request. So I'm hopeful that over the course of that committee, we will end up with a solution that is free and available to everyone that wants to use it for anything that they want. Okay, I'm going to turn it over to Alan. So I have one more question about Coursero. So a student writes their solutions to my homework and uploads their solutions to Coursero. Right. That's facilitating academic integrity. So I can say that this is an academic integrity issue, even though it's not a copyright issue. Martha? Yes. They're facilitating chances to other students. That's one of the things. The other issue where it gets sticky is it's hard to, if you discover it after the students already left your class, like they're already not in your class anymore, that's harder, right? Martha, because it's, isn't it meant, like can you do it retroactively after the semester's over? Yeah, we have. Once again, we've successfully prosecuted. Okay, I think we're getting into September's talk. I'm satisfied. This Coursero thing is quite illuminating. Very interesting. It's nothing on my course. So hi, I'm Alan and my job here today is to talk in the trenches, if you will. In the Department of Material Science and Engineering, we've actually been gathering data on this concept of cheating in academic integrity for about the last two and a half years. I'd like to thank Roman Engle-Harbert, Allison Beesey, and Lauren Czarzar for helping with this process. So basically the reality that we've come to in our curriculum and assessment committee for the department is that any assessment that's not performed in front of the instructor and or in a controlled environment is not an accurate assessment of what our students know. And that's because of the sharing of information which we then call cheating or misrepresentation. And we've discussed this as faculty. We've done the data studies and I've created this plot to sort of show you some of the things that we're seeing. So in several of our courses, and one of the reasons we started doing this is because in the last five years, our core courses have gone from 50 or 60 students to 120 or 130 students. And so what we did is we started looking at specific courses and we would plot the student homework average as a function of the student exam average. And one would imagine that in an ideal world, they would fall somewhere along that red line, how they're doing in the homework is how they're doing on the exams. In fact, you can divide this plot into four different areas. So we have our overachievers and our underachievers, right? So studying doing the homework and doing well, not doing their homework, but just good at taking exams. Then of course, you have the contrary of students who are just completely lost. But this upper left hand corner is really interesting. And what we've been finding in these larger courses, I mean, more than 100 students, is that upwards to 30% of the class will fall in this sector. So they have literally a homework average that's almost twice their exam average. And one may argue that, okay, maybe we have an issue with test anxiety or something of that nature, but those numbers seem unrealistic. But part of the solution is trying to address the test anxiety issue too. So here are some things, oops, I went the wrong way too. It's interesting. Yeah, okay. So here are some of the things that we are recommending with our faculty that teach these core courses. So first, we don't want, we still want to give homework because homework is a very important active learning strategy. And so we're still giving the same amount of homework. But what we're doing is decreasing its impact on the final grade. And so we're recommending that no more than 15% of the final grade is actually a homework score. And then to address the concerns of, oh, maybe some of this is test anxiety, we're encouraging our faculty. Most faculty, it's an interesting rule of thumb. I don't know where it came from, but they have two good terms in a final. It's a pretty common place. So what we're encouraging them to do is try to up that to four or five exams or quizzes during the semester, typically allowing the student to drop the lowest score from one of them. And all of this together starts to lower that anxiety level. Additionally, giving those exams at night, where you have an hour and a half window, but you design an exam that should be completed in, say, 70, 75 minutes. So you're also trying to get involved there. Another thing that we've started doing is the students, instead of turning in the homework for the grade, we actually happen to take a homework quiz. And a lot of cases that homework quiz is online. Now for my courses, I actually on the quiz will ask the exact same question they had in an open-ended homework, but I will then present them with multiple choice answers and they have two minutes or less to answer the question. So I tell them up front, you need to complete the homework before you do the quiz, because if you're trying to work the problems when you open the quiz, there's no way in the world you'll have enough time to finish. So do the homework before and then you'll go through, yes, question one, I got A, question two, I got D. Some of our faculty are using these homework quizzes, but not using the same, but using the same questions, but different numbers or rearranging. So in the homework, I asked them to calculate Bragg's angle. And in the homework quiz, I'm going to ask them to calculate these faces. I'm just manipulating the same question. We've included participation grade, this is pretty much universal across our core classes now, because we want to encourage attendance and we feel like encouraging attendance gives a better performance and lessens the anxiety and the desire to cheat. The other thing that we've been doing with this, of course, is taking data on how the students perform in these usually eye clicker questions that we're using to gauge participation. Once again, we can develop an entire plot of attendance as a function of class average and you would see the relationship that you would expect to see. But in addition, we've also been designing eye clicker questions such that we begin every lecture with a retention question. So in the first two minutes of class, we're going to ask an eye clicker question and your attendance to class is based on you answering that question. So if you come in five minutes late and you miss that question, but you answered the other three questions during the class, we don't care as far as we're concerned, you won't care. So it's really sort of fascinating because now what we see is students running in the class. Regardless of whether it's right or wrong, but they want to be there and be in part of this. But once again, we're still breaking this data down. There's a lot of data to look at. But looking at how are they performing on that retention question and then periodically through the lecture, we will ask concept questions, right? Are you getting what I'm talking about? So we're trying to hit them every 10 to 15 minutes in the lecture. So I call it re-grabbing their attention. And that seems to be giving us some interesting stuff, but we're still working on that. Other things that we're in putting in place, which is sort of obvious, like multiple versions of an exam, if you're giving an exam in person, there still needs to be multiple versions of that. And it's fairly straightforward to do. I mean, you can rearrange questions, rearrange answers. It's not that much more work once you have the original exam written. And then in online classes, it's imperative that you give unique exams to each individual student. And you do that by using question groups. And then Canvas randomly chooses questions within that group to give to an individual student. And I can tell you that what I've seen in this implementation is with Maxi 201, we've seen a decrease in the exam average of about eight to nine points by implementing this method. What's interesting is that the range of scores has not changed, but the standard deviation has decreased by 35%. And then the average time on the exam has increased by 20 minutes. So anyway, so this is what's going on in materials. And some of these things we've just implemented in the last year. We've been collecting data for a while. We're just now starting to use it to make changes. But anyway, so that's my story. And I'll take any questions now or later on things that we've been trying. Another strategy that I'm wondering if you try flipping the classroom so that outside of class, they're maybe, you know, doing more readings or watching, you know, or listening to video. So they're getting the lecture outside of class and then using class time for the homework. Or the assessments. I mean, that's an interesting idea. Another thing that we've been playing around with but have not implemented was this idea of quick quizzes. So for example, instead of a homework quiz, I mean, we're fortunate enough to have an incredible computer facility downstairs on the first floor. That computer facility could handle a class of 120 within an hour. If you divided them up into three sections, they came in and a talk to the environment took a 20 minute quiz on the homework. And so in an hour, you would have assessed the entire class. So yeah, so we've been, yeah, we've been thinking about different ways of flipping the classrooms. An interesting idea. And if you do the quizzes and assessments online, that it can lower the grading burden. So I mean, the problem with my class is that the grading burden gets very high. Yes. For more frequent exams and more frequent homeworks, but I like this idea of you sign a homework, people can go and work on it, and then you give a homework quiz that's, you know, frequent enough, but high stakes, either you know it or you don't. And that can be done online. I think that's a good idea. Where I still keep my paper and pencil exams, because I mean, I study the exams because I want to know how the students do it. Absolutely. Right. I mean, it's a really good feedback, but I would encourage you. I mean, this is something that I talk with my faculty all the time about of this idea of going to multiple choice. Everybody is low to do that. But the fact of the matter is, is done right and done carefully. You can get the same data, you can get the same feedback on knowledge and weaknesses and strengths on a properly designed test. And of course, the learning support and instructors and that can give you a lot of insight into that. The Shryer Institute for Teaching Excellence, Vandals offers some wonderful workshops on making multiple choice exams too. So how automatic is the iClicker integration with the grade book in campus? Is it still all done by hand? Because that's also big. So I probably have not perfected it, but what I do, it's fairly straightforward. So the iClicker software collects what happens real time in the classroom. In Canvas, I already have my grade book designed to accept those scores. I download that grade book from Canvas. I put in the iClicker stuff, copy and paste, and then I upload it back to Canvas. Canvas says, oh, okay, here are the scores. So you put, you don't have to go one at a time. You just have to manipulate some files. It's all an Excel spreadsheet. That's easier then. Yeah, it's fairly straightforward. The key is having the grade book built before the fact. Got it. And then trying not to change it. But yeah, the grade book means that you know what all the assignments are and everything. So you filled in maybe a couple of dummies if you need it. Okay, so Stevie talked kind of big picture with the universities doing some tools, those kinds of things. Alan talked about something very, very specific and what he did in his class. So what I'm going to talk about are practical strategies that basically anybody can use in their class. And they're broken up into loosely into some of these kind of overlap, but into four categories. Communication, what research says it works, how to use design to actually help, and then some tools that will help you. I'm going to focus on Canvas since that is where a lot of us kind of spend a lot of our time. So communication is totally key for this. It's absolutely imperative. And so the more you communicate with your students about academic integrity and what is going to happen, and as clearly as possible, the better chances that you're going to reduce academic integrity in your class. And so you can add a statement to your syllabus. Require students to read and sign an academic integrity quiz at the beginning of the class. Some will actually go beyond that and have them sign an academic integrity statement before they take every single exam, turn any assignment in. It's just constant, constant reminder. Set expectations about all of this stuff super early in the class. As soon as you start talking to them in the first being now you send out to them. Just keep reminding them throughout the entire semester. So don't talk about it the first week of class and then never mention it again. You just need to keep reminding them. And so it actually shows that if you say, you know, assignment by clicking this box by answering yes to this question, I certify that I'll work on this assignment doing it right within the assignment tends to reduce academic integrity. Just putting that one question in. Yeah, reduce academic integrity violation. Another thing I would just add having just had my second kiddo start at Penn State is they don't don't assume they know what it is and how they're violating it. They probably work taught very well about this in high school. So a lot of people, you know, in as part of everything's April saying is also educating them. Yes. And frankly, my daughter came and she was in freshman last year and she called me. She goes, oh, my friends, will you sign up for this great site? It's called Torsier. It's here to help you. And I was like, no. They don't know that that's not what it is because that's not what it is. So what research says reduces cheating. So one of the things that as I was reading some stuff to kind of prepare for this presentation is that developing good rapport with students is key to reducing academic integrity issues. Because if they know you, they feel they have a relationship with you. They are not going to want to disappoint you. And so the same goes if they have really good relationships with the peers in their class, they're not going to like cheat off of those peers. And so that rapport within the classroom is super important. Something else that has bubbled up and some of the reading I've been doing is linking academic integrity to professional integrity is also very important. So you wouldn't cheat on something or turn somebody else's work in on a job that you're getting paid for. So why would you do that in your academic class? And so trying to link those as much as you can, something that kind of goes along with that is setting up like an honor code for your class and kind of have everybody work towards that and understand that talk about that early on. That also fits right into that kind of integrity issue. Using students intrinsic motivation. I think most students want to do a good job. That's just something that they do. So figure out ways to actually do some virtue integration into when you're meeting with students and talking to students and writing it in your syllabus and all those sorts of things. Take that like intrinsic motivation seriously and figure out ways to kind of just remind students about that. And your research says that that will reduce cheating. Another thing that Alan actually talked about was reducing test anxiety. And there's a bunch of different ways that you can do this. He talked about a lot of them but just giving students like maybe a practice exam every now and then or giving them more chances to get points throughout the semester than having three chances. Two midterms in a final freaks students out because those individual assessments are worth so much of their grade. They are going to do whatever they have to to do well in that exam and that includes cheating. Yes. Does anyone in here teach first year seminar and if so or these concepts mentioned in that class? So yes and this is new for this fall. The academic integrity committee Eugene and Brandy and working with CVF put together a presentation. They can come to your classroom and give it. This is for anyone teaching the class by the way. Well I'm actually proposing something even more dramatic. I propose that every first year seminar instructor should be given a 10 to 15 minute presentation that they should be required to show. Period. I think we're headed in that direction John. So our goal this beginning of the fall is to first come to the faculty with all our new procedures and ideas based on the current app up and cases that we've had in the past two years. It's more complicated too because we also have to catch those students coming from the branch practices. So we're going to come to 421-300. We're going to try to catch them everywhere. You're never going to have that problem but I would say you should do something for the students who begin here at University Park. That's where it should be said. And I think this first year when we develop the top program of proposed to come ourselves people from the committee we'll see how it goes. We'll work with the faculty and we'll design something exactly globalized for what you're saying for the following year. The library has developed an online badge sort of credential for students. And there's an online training module that walks them through the rules governing academic containers at the University. I'm requiring it in my college class. Because they wouldn't have gotten this elsewhere yet. We're going to be integrating it into our orientation class for our major. But that might be another way to capture students require them. It's like some of the online training that we do. You have to go through these modules and then at the end it says it quizzes you when you pass it and you don't need to put that. I don't think that's as effective as having someone come to the seminar. I think it needs to be delivered by a faculty. Yeah. Any other questions? Okay. So right along with the research is how can you use pedagogy and learning design or instructional design to actually build in the reduction of academic integrity issues. So one of the ways is to use more authentic assessments. If you're making assessments harder to cheat on and having them actually do something more than the things that are really easy to grade. Chances are it's going to be harder to cheat on those things. Setting achievable assignments. So if a student feels that an assignment is just completely out of the realm of their ability to do it or you're assigning a 50 page paper for a one week every week or something like that like crazy kind of things. They're going to cheat on that stuff because they're going to feel that they don't have any chance to be able to do well on that assignment. You're always going to have the over achievers that will and they just you know will do it but a lot of them if they feel that that assignment is just not achievable they're not going to they're they're going to find ways around it. Spreading assignments over the entire semester. So you don't want to have all of the assignments do at the end of the semester because that is just like students aren't very good at time management. So they're going to just not they're going to wait until the last minute. So having them turn in drafts for something that's a very large project having them you know giving them like very low stakes quizzing along the way having more exams those kinds of things that Alan has been talking about those things will actually help reduce. You know how there's generally a midterm season for the semester where everybody's maybe movie was right so it's not at the same time. Change assessments out each semester and for for faculty this can be a bit of a pain. Yes it is because you're going to have to think of a new way to like assess the same thing over and over again. But what that does is if you're getting a different assessment they can't cheat on it. So using peer review sometimes will have an impact on reducing academic integrity. If students are looking at each other's work they're going to want to make that work as unique and as good as possible because their peers are going to be like judging them as well as well as the faculty members. So maybe you use the peer review and the draft together and that's kind of what you do a peer review and then that's the first draft and then they turn in the next draft to the faculty member. It might help with some of that stuff. Providing detailed feedback to students this is shown that it will actually help reduce the amount of cheating that goes on because then they can take that feedback and integrate it into what they're doing and that helps with the feeling it connects into the faculty member because if you always give canned feedback to students they're going to figure that out because students talk and so that is something that is pretty valuable and something that's kind of like just makes sense to a learning designer but make sure that you're linking your objectives to your assignments. So what that does is the students feel that those assignments are achieving something. You know they told me that this is what I'm going to learn in the class and by gosh there's an assignment that is actually helping me learn what I was supposed to get out of this class. They see value in that and they don't feel that their time is being wasted and they're going to less likely to cheat. If you're worried about course hero too, if you do exams or you do labs even if you change out who get third questions each time you teach but third round is completely new. So it's not you don't have to do the whole thing every single time but if you had a bunch of questions that you were asking you just swap out the third to third but the time you've done it three times it's a brand new assessment and that also makes them harder for them to output. So what can what can you do in Canvas because a lot of the assessment and grading and everything actually happens in Canvas. These are just some things that learning designers and Dutton talked to the faculty that we work with about so using randomization you know if you have a quiz randomize the order that the questions come in randomize the order that the answers come in and so and write better questions. So within that randomization having both A and B or all of the above or none of the above get rid of those kind of answers because what that does is it makes you have to have the question in exact same order every single time. So if you get rid of that stuff it'll help make that a little bit easier. Yeah good point. Require posting first before seeing others for discussion forums. So there's always that one student that wants to be the first to do everything they're the overachiever they go in there they answer the question and then if you if they get to see that answer then all the other students in the class read what they said because they know they're that student and then they just paraphrase what that student said and then you end up with not very unique sounding responses to discussion forums. So avoid simplistic or googleable questions because you have a phone you have a computer whatever the case may be somebody can say oh well that's a you know what is blah blah blah blah try to make questions a little bit more less simple you know make them think about it make them really kind of evaluate well okay so in this I learned about that and so maybe it's this and so they have to really think about it it's not just like a memorization thing. Show one question at a time and not the answers on exams. Um that one's pretty self-explanatory I use tire time limits so instead of giving somebody and it yes can we go back to show it I'm not sure okay self-explanatory okay so when you when you have an exam show one question at a time instead of showing all 20 questions oh this is if you're doing something online yes yeah that wouldn't work in yes can you say no answers you mean that when they submit the the test yes doesn't tell them doesn't tell them that that's what I did I think it's important to make notes that I'm pretty sure the default from canvas is to give them the feedback immediately because this is a setting you have to change or it will automatically get them the right answers. Good point Brandy thank you. And the wording is confusing if you set that date to provide the feedback to show them the correct answers they still can see the correct answers if they got it right right so if I take an exam and I know whether I got it right or wrong and I got it right well then I know that I got it right that much right so eliminate all that until after your quiz closes we'll help with that too yeah um music tighter time limits so if you have like a 20 question quiz don't give them 90 minutes to take the test unless they really need the 90 minutes to take the test you know that that kind of gets thrown out the windows if you have any students that have special needs and those sorts of things that handle those students individually and canvas makes that pretty easy now to do instead of just saying oh well I've got one student that needs time and a half so I'll just give everybody 90 minutes for the quiz it doesn't work it doesn't really work that way so make sure that the time limits go with the difficulty of the quiz that you're giving um use a logarithmic equations so algorithmic equations I always say that word wrong um in your assessments and so what that will allow you to do is for students to actually have very different exams when they're taking exams so they're getting different they're getting different data they're getting different information still the same question but what they're being asked to do with it is different yes yeah I'm probably exposing my own ignorance what are what do you mean by algorithmic equations? algorithmic equations are one where you can actually have it like if you're doing a mathematical type of a problem um somebody who's still intercept why do you present x plus b because that's got all I can remember from algebra um you can say give these values for m within this range and then it will randomly and say how many decimal points it will randomly give a student a question with a number that they have to then solve and they don't know what the answers are yeah so for example like you can you can have them do the math but you wouldn't have them each have a little bit the same formula but a little bit different math they have different numbers so when they put the answer in and then you can say how many digits the answer has to come back in you can also do you can't you can just do multi-step formula questions where like the answer to one question is the input to another but the Dutton Institute has built a system to do that because we know our faculty members need it so if anybody wants to do multi-step formula questions talk to me and we'll set you up with the system um change out questions each semester and cd was kind of talking about this so if you change like a third of your questions each semester each semester by year three you've gotten an entirely different exam and you're building a question bank uh as you're going along and that's my next thing is to create question banks and so what these do is um they allow you to have multiple questions and so on your on your quiz or your exam you say okay i'm going to set up question banks for for these different questions and then you say okay i need two about this topic three about this topic four about this topic and then it will pull those and different students will get different questions so what you have to be very careful with with related to this is you need to make sure that you are giving students questions at the same level so you don't want to give some you know because you could have one student get all the easy questions and one student get all the hard questions and you've got two very different exams and so there's a way to make sure that you're doing that properly and it is called using an assessment blueprinting thing but i have another slide next that will talk about that that will show you kind of how that works so this is a table that i found so if you have a number of questions you set your question bank up and you say okay i need five questions and then you've got only 10 questions in your bank 2.5 questions are going to be common for two students so the more questions you add to your bank the the less chance that students are actually going to end up getting the same questions so you can kind of look down through that so that so keep that in mind because if you only ever have like two versions of the question chances are if you're going to have students getting the same question quite a bit so and that could not encourage academic integrity issues and cheating but they would be getting the same questions and what you want to do with your question banks is give students different questions randomly so they don't know but making sure that those questions are at the same level so everybody is getting you know a knowledge question and you don't have one student getting a knowledge question and then you're asking somebody else to apply something in the same question so that makes sense okay i i have a question so scheduling our computing facilities sometimes it's difficult for when you'd like to give an online quiz where are we with requiring students to bring a screen with them to class because that's how we're going to do the quiz and can we can we require that or are we not there yet i don't think yeah we're still a quarter year of device university to some extent and but but can we require i mean well i i'm actually assuming that most students probably have one but i would do it because i would put that in me in my initial survey the first day of class but they all have one and yes required i don't know what the current uh statistics are but when we when the university has done surveys in the past it certainly is most students but it's not all anyone of the disadvantage yeah exactly you can't disincentivize the can't you borrow one from the library no that's what i was going to suggest is that uh multimedia technology support services has computers you can you can bought you can check out so maybe you do what steve is saying you survey your class how many of you have a laptop you could bring to class and you know three people don't this campus campus you know they can do it on their phones they can do it on almost anything yeah yeah okay because i don't and then you're getting closer to 100 yeah i mean then the idea is that i don't have to schedule our computing facilities but i can do automatic assessments in class that i don't get a big stack of papers that i have to grade at class yeah look good okay um so we do have on on this slide i mentioned the the blueprinting and so what that is is that you look at your objectives you look at can i actually go there i wonder let's see if i can yeah i guess i can okay so this is all on our fact def site which steve mentioned a little bit early earlier and so what you do um is we'll just go down to the bottom where it's filled out so what your goal is here across the top you have bloom's taxonomy so are your questions of knowledge questions comprehension application analysis that synthesis or evaluation and then down the left side for the objectives for the course um and then for each objective you kind of pull out the topic so what were the topics that you kind of got to with your objectives and so what you do is say okay i need one knowledge question about aquifer basics two comprehension so that's a total of three questions and so you just kind of do this for your class and identify those things they also help you figure out whether you're living in a museum at the correct level for what you really want to teach on yeah and so then at the end of this you know how many questions you need and so what you can do then is to use this and a question bank together so if you need if you want three if you want for every one question you want three possible choices then you need to write three aquifer basics you need to write six comprehension level off the basis and that ensures that every student because of the way you set up the question rules getting evenly difficult test you know when you get an artificially easy one the one you get an artificially hard one because there's the same number of questions at every level and you double check to make sure that you're actually testing the level that you think you are and so what you would end up with is probably uh oops so what you're going to end up with is probably I guess you would end up with probably two question banks and you would have two for this one two for that one for every topic yeah so if you are you probably would never do this but you could end up having like six question banks for each one of these topics but you probably would never do because you wouldn't want all of those things the same but same topic I don't think and so it gives you really a blueprint on what you need to do once you get into Canvas so you're going to know and so whether you're making three questions for each one of those things some people have like I think Dr. Ali has what 10 10 questions for each one of his yeah yeah and so and instead of doing those all the same time add one add two each each time you teach the class and then you'll have a nice question bank at the end and so students might end up getting those questions but they're not going to know which one they get but if you give the same exam every single time they're going to you know be able to cheat a lot more easily so that's what question banks are really helpful for that okay so how do I go back now there it is okay so we do have some Arthur Merrill science resources there's an EMS academic integrity information page there we also have a lot of information on the Dutton website we have academic integrity resources and then information about academic integrity and there's a bunch of other ones there but those ones really kind of encapsulate everything that we've kind of talked about today and these were just some sources in case you're interested in like where some of the the theory came from and some of the suggestions and strategies there's a bunch of things there when we send that out so we don't have much time for questions and answers but we have been answering questions as we've gone along so hopefully that's that will work anybody wants a consultation with a designer send me an email April where will and Jane where will this power point live so that people can click on those links we have a food for thought section on the website on the fact that we'll post that this recording of this presentation to and and the power point there and then we can also send an email especially you remember to sign in which looks like just where you ask we can send you all did you not get that we can send this over here sorry yes a request for the next follow-up session and I know I've asked Jane about this but maybe you have to update um turn it in is useful for for looking at whether a student's play draw something that's already out there it does not allow me to compare assignments within that a group of assignments that return and that is to me the most important thing because what I so to each other correct the cheating that I uncover is two students who turn in the same so if there's some way through that that would be no if you go if you go to their help set it specifically says this is not designed to compare assignments I understand but I've had cases where student submits their assignment with everyone else who needs to do and I upload them all the time and then that student realizes they sent me a draft not the final version so they do send it to me and as soon as I upload the second one it flies for as a as a full match for the one they had already submitted I don't know I had two assignments that were identical and when I uploaded it they did not grant quarter of this because it's the second record yeah yeah maybe all together yeah and then it's better than the second record compared to the first level yeah so maybe this one should hire you the the integration and you're not sure how about the work upload them in the initial but it'll upload them individually so it might it might catch that look you can contest it you can try that I want to be respectful people's times I appreciate everybody coming to watch we'll stick around for more questions if you have more but it is one clock so if you do want to stay we can continue the conversation yes and there are more right thank you all