 All right, welcome guys. We are on to chapter five. So we have we have hit the we've hit hump day in terms of readings Although honestly, this is not like a regular workday. I have a work week. I've been okay I like I like my work weeks too, but I've been enjoying the chapter. So I'll probably be kind of sad when we hit chapter nine but Today we are looking at feedback Which obviously is one of the things that is most valuable and awful also that we often fall down on it I think or at least maybe I'm speaking to myself there But one of the areas that I know I can improve the most is is the topic of really today's chapter Which is timely and useful feedback with timely being very important. So yeah Patrick You're gonna take the lead on giving us a chapter summary on this one. I am most definitely Okay feedback yeah, definitely it's it's it's it's key for key for sure I kept going back to the To our coaching the way we kind of started, you know, it's it's she she begins the chapter with the With the with the two key areas formative assessment, which we talk about as being assessment for learning All right timely and high quality or what she calls effective So it's all of that. Well, it's basically how we learn how we learn anything Grading has very little to do with actual learning It's the it's the formative assessment the feedback the opportunity for practice and honing the skill or whatever it might be And then of course summative assessment, which is assessment of learning So once the formative part is done the practice is done then you get the the evaluation The judgment as they're to grab two quick pitchers once again You can see one formative assessment very detailed feedback one person Giving giving advice on stance and and then the judge the flashing of the numbers, right? One happens before the other and I think it's important to help students understand the difference between it Yeah, there will be grades, but you're not ready yet Grading comes towards the end you build towards that But of course there's always the motivation and how do you get students to take that formative assessment and do the trying and the practicing There might be some interesting ideas for to incentivize that as well So she talks about her five Models and she kind of breaks them around, you know timely versus high quality and some Overlap on each other, but as well, I was reading I kind of went back and saw well each of these actually builds on those those Presences that that we talked about in that model in the previous chapter. So deadlines strategically And I think we all do this right you look at say, okay, what's coming? What will students need to do? looking at it from a selfish perspective what I want to be Marking and grading at what time but also looking at it to see well What what fits with students to be able to give them some lead time? She talks about, you know, Sunday nights as opposed to Monday nights and those kinds of things Real-time conversations. I say even since his book giving teams has that little telephone Icon there, you know audio you can just you can just speak to someone in a quick teams call have that immediate kind of conversation that allows for for for, you know instant feedback and We see as Moodle allows audio and and video feedback and that that kind of runs throughout Office hours once again timely, but also high quality being able to rethink that she had some really nice ideas in there about Coffee time and and changing the the language around office hours, right? It just sounds so Stayed but ways of kind of rethinking that Using the tech to streamline the grading. I really like the the rubric and maybe that's something We can we can look on Jason. I don't know how to do the whole rubric thing as a part of the grading and in Moodle She just talks about her LMS and you can do this I don't know how much it is to set that up in the grade book But that allows students to to see their progress which you know informs them She used the example of I think it was her husband was 80% through the course and hadn't received or 75% through the course and hadn't received any real feedback And and that happens more often than we think that students are in the dark about how they're doing in any particular course So and of course using the tech for feedback and we have so many opportunities to do that and What I was sucking up too much bandwidth, I guess but quick little video we talked about this earlier video feedback is And students give feedback on that type of feedback and they really like it they value it But as I said, this also builds teacher presence It builds social presence because you're connecting with the students and also engagement, you know You can talk give feedback the students are aware What do I need to do next? How can I improve and that engages them and helps them go back and and really learn those Kind of things so just to finish up quickly. I like this quote What students crave and what too often they don't receive is feedback specifically designed to help them get better Especially through frequent low stakes assessments So that's kind of really key and I took that away as a as a closing closing comment Thanks, Patrick So Jeff I just wanted to acknowledge your your comment there Maybe just in the to keep things a little bit more focused right now on the discussion Maybe I'll get you to throw something in the in the discussion thread afterwards Sure Or if there's a video that already exists with the CTL or something you can throw that in there as well I don't recall if there is as a reminder for anyone who's following along in real time The the CTL has now over the last you know a year and a half cheese man. We've been doing this a while But I a massive database of instructional videos Jeff and I made a couple of the beginning ones But they've just been keeping and going and going and going and there's I know there's got to be 40 or 50 videos They're easily on pretty much anything you could want to do exists there now. So well, let's start off the discussion maybe by talking not about the the The timely stuff but talking about Real-time feedback in the form of office hours. I know this is something which is which has come up in conversation Previously, do you guys how do you go about trying to offer synchronous opportunities for for discussion with with your students in this Asynchronous world that we're in now someone want to take a lead and tell us what you do what works. What doesn't? So At the beginning of my Term In almost all of my classes usually all of my classes and I started doing this actually long before Anything was online. I started doing it When I was all face to face I have students fill out and this is not original to me, but I have students fill out a Simple form Prior to COVID it was literally like a handout that they wrote on But now it's a Microsoft form anyway, they fill that out and the catch or the difference is they have to deliver it to my office Again, obviously they can't do that Now and so, you know, but when they deliver it We would have like a five minute conversation and I would do this all in the first couple of weeks of class Of course, I have smaller classes so that helps, but I just decided why not try to do it virtually And so I did that in the fall and I did it again this term Where basically they have to fill out the Microsoft form and set up a meeting with me and I used I think Calendly which I'm not wedded to but that's what I use this time and they sign up it pops up on my calendar We call each other on teams for five minutes or ten minutes. Usually they go a little longer If it's going well and they ask any questions It gives them a chance to say like hey actually there's this thing that like I don't really want to write down but I have to tell you or Just like I'm super overwhelmed and confused and don't know where the syllabus is right like you give them a chance to Do any of those things and it really makes a difference with community Haven't gone as much of it through this semester, but I actually just today Had I have a class that I'm teaching Synchronously once a week and the other day they do in asynchronous stuff So for today's class period I said I would be there and I had a couple of students come by and I didn't call it office hours I think I called it a coffee hour or something And I only had two students out of like 18 show up that we had good conversations Then I answered a lot of their individual questions. So I'm a big fan I think that's one of the biggest things I took away from this chapter is like to find more creative ways to do some of that like Face time for lack of a better way of putting it Patrick Jeff you guys want to add something? Yeah, but my my big fail on the office hours I guess was just kind of posting it and being online and just Sitting there waiting, you know for someone to show up and Invariably it was just you know crickets, right? So I Appreciated that that part of the chapter where she says yeah even changing the language around it and see intimidation factor for a lot of students too I said what the hell I'm not gonna get out and talk to my professor. I'll probably be the only one there How terrifying is that right? So? So I'll be trapped with this guy So yeah finding finding ways to to soften that a little bit and to to to maybe just be Coming right and invariably when I do get a chance to speak to people would never never come to one of these open sessions Yeah, they so how things going and they have questions for me So yeah, well once you drop by and you're right, but but they're not gonna they're not gonna take the initiative So I thought that was really interesting to just rethink that I never even thought of the language around it But yeah, sure it's something we could we could do better at I think on the on the same note to the The the the less is better kind of idea that that's something I might take into heart that yeah It's gonna hard to get students to show up Show up every single week to something but just thinking back to when we used to have review sessions before minterms and this sort of thing Right, those are the only times and it's still I mean the attendance is still often pretty mediocre Right and you know who's gonna show up to a lot of these but you'd still get third or half of the class Whereas now I don't know about you guys, but I'm I'm averaging maybe 25 percent 20 percent It's showing up to my yeah I have weekly review sessions You know one of my classes because they actually scheduled a time right? Because it's yeah, but I'm running I'm running it functionally asynchronously So I knew that time was at least in theory free for the students I was like oh, I got to take one of these one of these scheduled time blocks I'm just gonna make it open I get maybe 20 percent and it's they literally it's the same three students every single week But yeah, I mean in review sessions used to come more So maybe maybe those pre-scheduled four times in advance and of course we have pre-scheduling And she noted this in the chapter importance so that students can work it around their increasingly complicated lives So I do and this does the thing I do is in some sense Totally giving up on the whole idea of office hours and trying something else entirely different So This is something I do in all my first year courses I have these things that I call mastery tasks So the way they work they're they're required In the current classes, I'm running they have five of them that they have to do at some point in the course and they each have a deadline and The way these work First I'll I'll tell you how I do them not online and then I'll tell you how they work online So when when we're not online They show up Whenever and I say I have office hours scheduled and you can show up to do these during my office hours But anytime you can catch me is good So they show up I Hand them the mastery task. It's a little piece of paper about this big And it has something for them to do which uses one of the key ideas and they're always they're intended as quick little essentially skill checks of core skills in the course, right? So these are like in first-year physics courses. These are things like drawing free body diagrams drawing Energy bar charts things like this. So they turn up. I hand it to them. They sit down and do it They're designed to take perhaps two minutes, although often they take longer Then they bring it in and I hand it to me. I mark it right in front of them, which takes about 10 seconds If it's right, I just mark it off as done if it's wrong I show them what was wrong and hand them a new one Repeat until they can do it Yeah, and and so, you know a This is a requirement and so I get them all coming to my office or in fact not to my office because what I do is I hold all my Office hours in the math and science center, right the learning commons Which has a dual purpose first of all it means that when a bunch of them want to see me all at once for stuff like this It's more convenient than my office But to it also Introduces them to the math and science center and gets them in the habit of going there, which I think is crucial Anyway, so that's that's how those work Online the way I've made this work is that again they have to come and do it Well, that means Sending me a chat on teams and saying hey, can I do the mastery task? I said, yeah sure. I'm free and I call them Right and now I open an online whiteboard right a shared whiteboard I hand them the link and I drop the mastery task on the whiteboard They do it there. Tell me when they're ready for me to look at it I look at it on the whiteboard give them their feedback if they need to do it again Delete it give them a new one on the whiteboard rinse and repeat So now it's a little time consuming Although what I find is you know unlike office hours, right office hours you schedule, you know We're required to have five hours a week whatever and nobody turns up Well these I set the particular deadlines and for about the two days before Each of those deadlines tomorrow is a deadline So I'm actually really busy today for about the two days before one of these deadlines I'm really busy with people doing mastery tasks, but then that's it Right, it's and so that's sort of five times through the term I have a block of two days where I'm pretty busy running people through mastery tasks It's a little more time consuming online because I find I can't manage more than two students at once doing whereas When when it's face-to-face and I can be down in the math and science center I could have like 15 people turn up. I just hand them all pieces of paper They go and sit down and then I'll develop like a little short lineup of two or three people at a time waiting for feedback, right? It's not too bad Yeah Speaking of time so one of the One of the you know the big points was timely feedback and you emphasize this Patrick And I know this is something I fall down on all the time because well, I have trouble concentrating and so Marking always takes me my ratio of marking to like looking at Twitter or You know my brain does not do well these kind of menial tasks of running through these so it I'm not efficient I'm not efficient But even it's even worse now because I have so much more stuff I could be doing so last semester for example I spent all the time desperately building out, you know online labs and lessons and things and I put off the Marking of labs until the end of the semester and of course that means they're not getting real-time feedback, which I I mean, it's better that they had some kind of lessons to do or labs that existed but the other prior needs to be done too, right and the I've moved towards and this is more relevant for people who do labs and things where it's you know Yes, no factual kind of answer But as much as possible I've moved towards automated feedback on things and one of the things That's really useful is when you to build things out either in the lesson format Which I do sometimes and you're getting the real formative feedback inside the middle lesson and Jeff's done a great A great thing that's hosted the CTL is a couple four or five lessons the lessons on lessons that Jeff put together but also using the How do you say it closey or how do you say that a little it's just called clothes and it's actually an actor knows format I Was I was intimidated by this for years because it involves Involves a tiny bit of code and I was like I'm not gonna do this code Then I found a great website that just has like a cheat sheet of doing it and it turns out to be really easy So that you can build out a you know an answer where you've got your question You've got your sentence and then you have you know a drop-down list for words within your sentence You have a whole block of paragraphs and you can have you know Eight or ten where they have to just fill in the words as they go along And then you can have a you know short answer at the bottom of it And you can have a whole list of things which are very easy to make and you can do a lot more with it And so I automated things like you know I'd have exercises where they have to identify a rock and they'd have to say what's the name of this rock? How did it form? Where did it form? You know what minerals are in it etc. And so you've got four or five You know different answers for each one of these specimens, but they have ten specimens and I've got 60 students And so now if I'm having to look through and do each one of these I'm looking at you know A thousand times I've got to look through and it's ridiculous, right? So the time I get that it's two weeks later now I've got it so it's instantaneous and automated it takes a little bit more lead time to do it But then it's instantaneous and I can put that time into doing real things and you can set it up on Moodle and And it reach out to Jeff reach out to me We'll show you how to do this if you don't do this but you can set up in Moodle So not only do they get the answer But you can also give them hints and give them multiple tries again. There's a little bit more lead time for building that out But but you can do that. There's a lot of neat stuff. I'll tell you one more neat thing so I did this a couple of years ago and Barb in biology actually built this out to a much bigger thing I Suggested this and then she really ran with it And if you ask her she'll probably be willing to provide the document she created But I know everyone hates or a large percentage of people hate turn it in comm and All of these kinds of nanny software kinds of things but we took that around and Flipped it and used it for formative feedback, right? So the instead of using it to catch cheaters You can get to show students what they're doing so they don't do it Anyways, so I did that were required students to actually hand a draft in Then they could look through and they could see where the thing was flagging them And then they would either sit down with me and I'd go through and we Explain how to do it or alternatively they just keep doing it until they fixed it And in fact, there's actually they can't just keep doing it a thousand times They have to there's a lag each time you hand it in but that allowed them to actually learn their errors so that one they're not getting a set up to the dean and And two they actually learn how to do it because most of my students weren't cheating on purpose They just had zero idea how to cite something right so using that tool and Turning it into you know automated formative feedback and you can either have the meeting with them or Alternatively you can just let them do it on their own and they can teach themselves the errors And if they're really not getting it then reach out to you so that saved me a lot of time on both Sending things to the dean but also on On going over drafts with them right because they could go to their peers They could just go to owl, you know go to one of the websites and see okay What am I doing wrong and the algorithm is showing this flagging the areas for them So that saved a lot of time and gave them immediate real-time feedback during draft formulation Anyways, I I can give you the basic version But Barb made it a much bigger thing that was part of her class and apparently had a really good feedback really enjoyed the experience Great, yeah Yeah, all-time lag is not created equal right, you know for big some of the things Students you can take a week two weeks to get it back to them. It's kind of things like Jeff mentioned I mean that's instantaneous they get it you look at it boom there it goes and same with the same with you Jason right that's the kind of you know if you were saying take a week I'll get back to you on your mastery thing, you know, then the learning is just kind of just halts, right? So yeah, you can you can build it in that way. So it's similar And this is something that we've done for years in the lab Which never really worked and then this year being forced to do this online. I found a different way, right? So Pre-labs, okay So I mean, you know the idea of a pre-lab is it's something the students are supposed to do before the lab And they have it done already when they turn up and it's to make sure they're actually ready to do the lab Right, so I mean for years I've been you know building these labs and they have a pre-lab section and the idea was they turn up at the beginning of the lab They hand it to the lab instructor and in the first, you know Five or ten minutes as everybody's setting up the lab instructor is very quickly marking the pre-labs And immediately handing them back to the students so they have some feedback at the start of the lab Well, that's the theory But the practice of course is that the lab instructor is too busy helping people set up and and so on and so forth and Rarely would actually get the pre-labs marked in those five or ten minutes and turn back, right? So I said well, okay fine this year. We're having to do this all online Anyway, I'll turn all the pre-labs into Moodle quizzes, right and they're do, you know Midnight on Monday and the lab is due Friday and so on right and so I'm just gonna keep that right. Yeah, the pre-labs will just continue to be Moodle quizzes Works way better and Jeff tie tying it back to a previous chapter. Of course. I'm sure you do this through the alamance It's really easy to set it up for conditional release, right? So that they're not able to actually do the lab until they've done the three That's right, and I've done You know, they're not able to hand in their essay until they've you know, until they've run a draft through a thing or something like that It's it to force them through those initial stages of getting feedback on their own and as much as that stuff The more time we have to do the really the really substantive things that make a difference things exactly But I just wanted to say one of the things I've also found about timeliness Because a lot of not everything but a chunk of the things that I end up assessing even if they're formative are like, you know It's a performance or it's a Free-page reflective essay. It's like you can't you know, a machine can't do that for me. I can't really automate it but One of the things I found is just being really upfront and human about it and saying like so like for this Just real example My students who have never most of them done a performance in their lives for their first performance this term It's taking me longer than I wanted to get back to them and I said I'm working out Like I just sent them an announcement basically that said like I have them working on them I'm really proud of the work you all did I want to make sure you get real feedback that actually helps you improve so it's going to take me a little longer and I haven't received one complaint or question or and I mean they're all done now But like in that time period that I had It took the stress off of me a little bit to not be so like panicked about it But it absolutely I think they appreciated it. They appreciated the honesty like oh cool Okay, so I'll get some good feedback. All right, and you know It saved any of those like when are we getting those back questions, which it's fair, you know Anyway, that's just something that I tried this time that really seemed successful And now I'm thinking like how can I build that in ahead of time, right? So I say like These might take a little longer than a normal assignment for me to hand back to you And here's why and here's what I'm doing with that time and then adjusting expectations as we go Sure. Yeah, good call. All right. Well, let's let's end it on that note And we'll throw a few extra little tidbits down in the in the discussion any little any tricks and videos and things like that and Next week we will finish up the second part of the book. All right