 Good morning, everyone. I'd like to welcome you to the note session. It's the fourth day of Celebrate Learning Week. It's been a very busy week, but a very enjoyable week. I'd like to start this morning by acknowledging that we come together today on the traditional ancestral and unceded lands of the Musqueam people, who for thousands of years have lived and learned on these lands, and we're thankful for our continued partnership and collaboration. So welcome to this morning's keynote, both people who are here in this room online. This is being livestreamed to the Okanagan campus as well, and people watching will have the opportunity to participate and ask questions when we get to that stage of the proceedings. Just before I introduce this morning's keynote speaker, I'd like to remind people gathered here that immediately following this session, you are invited to attend the TLF showcase, which is happening outside in the foyer. It will start up pretty much when this session finishes around 11 o'clock, 11.30. In fact, you can't really miss it because it will fill the foyer. So as long as you go out of this room and turn left, you'll find yourself in the middle of that session. I think this year is a record year. We have 50 posters showcasing some of the current and recently completed TLF projects. Many of the visitors who we have at UBC, and particularly in weeks like Celebrate Learning Week, when we talk to them about TLF, their jaws hit the floor when they hear the scale and scope of support and resources that the university puts into funding teaching and learning innovation. So it really is our innovation engine in many ways for teaching and learning. So come and find out more about the projects and chat to some of the faculty, staff and students who've been engaged in them. So I have the huge pleasure of introducing this morning's keynote speaker. Peter Felton is Professor of History, Assistant Provost for Teaching and Learning, and Executive Director of the Center for Engaged Learning at Elon University. Peter's published extensively around the theme of students in higher education and their role in higher education, including volumes on engaging students as partners in learning and teaching in 2014. And those of you who are interested in that topic, Peter is giving a workshop tomorrow morning in Ike Barber on the theme of students as producers and contributors in teaching and learning. Another publication, Transforming Students, Fulfilling the Promise of Higher Education, again in 2014 and most recently the undergraduate experiencing, the undergraduate experience focusing institutions on what matters most, and that's the theme of his keynote today. For those of you who don't know Peter or his work, he is widely known and highly esteemed in the teaching and learning community of higher education. He served as President of the International Society for the Scholarship of Teaching and Learning in 2016 to 2017, and also the Pod Network, the US Professional Society for Educational Developers. He's a co-editor of the International Journal of Academic Development and many other attributes and accolades as well. We're very fortunate to have Peter. He is a man in very high demand. Every time we email Peter, we get his out of office saying that he's travelling somewhere. I'm delighted that he's agreed to join us for Celebrate Learning Week this year for this keynote today for tomorrow's workshop as well. Please join me in welcoming Peter and we'll hear his keynote. Good morning. Good morning. Good morning. Thank you for the very kind introduction. I always appreciate it when someone reads the intro my mother wrote. It's very kind. And there's actually seats in the front that aren't in the way front, like where I'll step on you, so feel free to come down here. But before we begin, just one obvious thing and then one request. The obvious thing is I'm from the United States and a bunch of the research I'm going to be talking about today is in US higher education which I know is in some ways similar to in some ways different than UBC. And so I'll be offering that. I encourage and invite you to think how does this fit or doesn't in our context or my context, what's different. I hope this can be really conversational as we go, which gets to the second point, which is I said good morning and where I live in the United States, there's this call and response culture. And so if someone says good morning, I have a Swedish friend who says why do Americans always have to, why do you have to do this? But I do. So I say good morning and then you say thank you. Doesn't it feel better? Maybe not. It's sort of uncomfortable for Canadians. But as we go, I really hope you feel like you have a voice in this session. There's going to be times where I'm going to ask you to talk to each other and then talk with the whole group and we're going to invite in people who are far away through Twitter. But at any time, I hope we can be just really interactive and you can add your points or ask for clarification or disagree. So welcome. So like I said, I come from the U.S. and we're thinking about the undergraduate experience here. And so just to get us started, I want to tell you the news from the U.S., which is this. That is not specifically about politics, although you could read it that way. But it's really about if you read things like the Chronicle of Higher Education or Inside Higher Education, which are prominent sort of news sources on higher education in the United States. This is sort of the central theme that we're doomed in lots of different ways. Three reasons. There's lots of reasons, but three I'll give you right now. One actually I know was here on Monday, right? The robots are coming for us. Or more specifically, the robots are coming for our students, right? And artificial intelligence and rapid gains in artificial intelligence are changing what it means to work and what is uniquely human that we can contribute in work, right? And what that's doing is putting pressure on, so why are we doing undergraduate education at all? What's the point? I mean, back in the last century when I was an undergraduate, you went to a university because that was the place that had the knowledge. Now, if you want the knowledge, you know, this isn't the only place you can go. You can go to UBC or you can go to your phone. You can go to lots of places. And the knowledge isn't the same in all those places, but the access to knowledge is radically different. And then the capacity of machines to do things, for example, as the book talks about, that lawyers do, that engineers do. Historians, they haven't figured out how to mimic yet, but it's not worth enough money to mimic what historians do. But there's an awful lot that is changing about what it means to know and what it means to use knowledge, right? And that is changing, or should be changing, what it means for us to do our work in universities to educate students, right? So there's challenges of fundamentally what's the purpose of undergraduate education. Then there's technological challenges. At the top, there's a box, I don't know if you know this article about data technology and unbundling. The idea of unbundling, it's a technology term, as far as I understand it. And what it's saying is that for many, many years, you went to university because that was the place where you got the whole package, right? You came here and you studied physics. And you were here for four years or five years and you took the established curriculum because that was the only place you could do this. Well, you could go to Calgary, or you could go to Seattle, or you could go to Toronto, but you had to go somewhere for four years to get the curriculum. And the unbundling argument is saying technology has broken that. Why do you need to be here now for four years? Why can't you be anywhere taking UBC courses, right? Why four years? Why any amount of time? Why can't you take a physics degree over a 20-year period of time from UBC, right? So the idea here is you could break up the traditionally packaged together pieces of an undergraduate experience into separate discrete units and then have them where you need them, when you need them. For a campus like my own, which is a residential liberal arts-focused undergraduate education, this is super-threatening. 80% of our students live on campus in university-provided housing. If they decide they could be students with us but they could live in Vancouver yesterday, it was much nicer in Vancouver than it was in Emon. I'd live in Vancouver and take Emon classes. But then what's the model of the university? Because we also have residential learning communities. Well, those go away, right? Unbundling is technologists and technology enabling different ways of interacting with very traditional learning and teaching experiences like courses. And if you hear me talk about this and think, yeah, it's maybe not that important, Randy Bass from Georgetown has a line that I think is true and sort of horrifying and funny at the same time. He says, there are literally hundreds of millions of dollars being spent every year by venture capitalists and corporations trying to monetize parts of what we do in undergraduate education. And then there are literally hundreds of dollars being spent on campuses like UBC trying to improve what we do. They've got a lot of money and what they're aiming for is not upper-level history courses. What they're aiming for is the first year of an undergraduate experience. If you don't know it, there's the Freshman Global Academy at Arizona State University. You might look at this as a model. I don't know if it will affect UBC or not. Arizona State is very ambitious as a university. They're offering right now 13 courses that are designed for first-year undergraduates. The interesting thing about this model, they're sort of high-end MOOCs, but they're clever. They're free to take. You pay $60 if you want to get assessed. If you decide you like your assessment, you pay $600 and they'll give you course credit. So you earn your A's after completing your courses. But they're then transcripted as Arizona State University courses. So they show up at my university or your university perhaps. And they have a whole first year under their belt. Interesting. So there's technological pressures here. There's cultural pressures and sort of purpose pressures. There's technology pressures. If you don't know the book, Economically Adrift, it came out in the U.S. eight years ago, seven years ago or so. And it was a study of 24 different universities and colleges in the U.S. and student learning in the first two years. And a wide range of colleges and universities from very, very selective to open access. They looked at all sorts of dimensions of student learning. And what they found is in the first two years, the actual quote from the book is how much learning is there? Quote, not much for most students at all different types of institutions. And the analysis gets much deeper than I'm providing here. But what they say is often the students drift into our campuses, not quite sure what they want to do or why they're doing it. And then they come into a place like my campus, maybe yours, where there's a pretty broad and open curriculum. Lots of different options of things they could do. And lots of enticing things that aren't academic around. And they drift through their time on campus. And we, as the faculty, don't push them too hard. And they, as the students, are pretty happy not to be pushed too hard. And so they drift. And it's not surprising they don't learn much, right? So I put this slide up here really to say, thinking about it, if in our existing model, students may or may not be learning as much as we want or they want. There's people trying to break apart our model and steal the most profitable parts and the most students from us. And the very purposes of what we're doing with undergraduate education are of necessity being called into question because technology is changing. And then maybe the end is near for how we think about undergraduate education. And maybe that's not terrible, right? Because if we have a system that doesn't work very well for our students, doesn't educate them that much, and is replaceable by technology or by robots, why bother, right? I think in this, at least in the U.S. context, there's a couple reasons why to bother. One has to do with this chart. I'm a historian. I didn't make this chart, and it's U.S. specific. I don't know if it applies in Canada, but it's really troubling if you look at it. On the y-axis is your chance of earning a four-year undergraduate degree by age 24 in the U.S. On the x-axis is your SAT score, which is a common college admission. It's a university admissions test score. On the near the zero-zero spot is scoring well. Scoring right in here, let me see if I can do this. Right in here is the mean score for the U.S. Does that make sense? Okay, the top line, that's if your family is in the top quartile of income in the United States. In other words, if your family is in the top quartile of income and you score well on the SAT, the odds that you'll have a college degree by age 24 are quite high. If you score below average on the SAT and your family still is wealthy, the odds are still better than 50-50 that you'll have a college degree. The bottom line, as you might expect, is the bottom quartile of income in the U.S. If you come from the bottom quartile and you score at the top of the SAT, the odds you'll have a college degree are 44% by age 24. If you score just a little below average, the odds are almost zero. And the gap is pretty steady. In other words, in the U.S., at least, we have really serious issues with equity around undergraduate education in our system. Maybe that means we should change it. But it also, yeah, difficult. This is students who took the SAT. They might not have entered into university or not, right? There might be a whole lot of students in this area who are three credits away from graduating. This is just a large-scale study of who took the SAT, family income, what happens, generally six years after taking the SAT. I'm curious, is there data like this in Canada? Anybody know? I'm sorry? There's no SAT. There's no SAT. So yeah. There is this, though, right? So maybe not the same level as in the U.S. Anyways, it's an interesting and challenging question in the U.S. And the research in the U.S. is really quite clear, too, that systematically underserved students, the students on the bottom of that last chart, do better, as this says, in environments with mentorship pathways and challenge. In other words, they do better in bundled education. They do better in universities like UBC than they do in online diffuse education, right? And so if we're trying to, in our context, my context, address equity issues in higher education, if you're trying to address equity issues in Canadian higher education, one of the questions I would have is what kind of system do you need for students? What are the educational, what's the educational environment that's most going to support students who most need your help? That's the question. And that was, in some ways, the question at the heart of this book. I'm not trying to sell you this book, so I'm going to mention this book. I really am putting this slide up here to note that all of the good ideas are coming from my colleagues, Betsy Barefoot, John Gardner, Charles Schroeder and Leo Lambert. But what we did in this book is we tried to look across U.S. higher education, look into the research literature, but also into practice to make the argument that there are, despite all the narratives of trouble in American higher education, there's actually some really good stuff happening. And if we focused on what the really good practices are, we could actually get there. We could serve more students more effectively without more resources if we focused on what matters most to our students, right? And in the book we suggest six things matter most. And part of the argument here is that these six things, you can think of these from whatever your role is. So if you're a faculty member or a graduate student, you could say, how do I contribute to these things? What's my role? If you're a dean or run teaching center or president or whatever, a board member, you could think of those from your own perspective. If you're a student, you can think of them from your perspective, okay? These six things will dig deeply into three of them. Learning, obviously, right? It's the university. Relationships, expectations. We're not going to talk much about expectations, but the research on expectations is really quite clear that the expectations we have of our students shape their behaviors. The expectations they have of the university shape their behaviors. That we can contribute to what their expectations are. They can contribute to how, right? So we're establishing clear expectations, holding people up to those expectations, helping students develop the ability to articulate expectations for themselves and figure out what they should be doing is really an important part of what happens, what should happen in undergraduate education. We'll talk about alignment in this context. Improvement and leadership. Improvement is about using evidence to inform what we do. Both as a way to get better in our own work and as an institution, but also as a way to model for our students that what professionals do and what humans should do is use evidence to improve or to inform their behavior. And then leadership defined broadly as this is not something that the president does. This is something that all of us do. And this is something students need to have spaces to do, right? To lead in their own education. So the argument of the book is just those six things matter most. And that if we focus on those, whatever our role is, will be fine. What I want to do today, just in the next five hours, a little bit of time is dig into just three of these. Learning relationships and alignment. The most, because I think it's the most important, it's central to what we're trying to do in undergraduate education. Make sense? Okay. I'm going to start with a quote from Herbert Simon. You may have seen this quote before. If you don't know Herbert Simon, he won the Nobel Prize in Economics. He won one of the Adventures of Game Theory. He won the Research Award from the American Psychological Association for a lifetime contribution to cognitive psychology. Anyone at Nobel Prize in Economics? He's sort of smart. Anyways, you can see what he says. I think this is really important to remember as we're celebrating learning all week, is that really we're celebrating what students are doing. And our job is to create conditions and environments that challenge them, that support them, that encourage them to do things that maybe they didn't know they could do. I want to dig in to this part of the quote. What the student does and what the student thinks. Those are Simon's words. And sometimes I want to change the word think to feel, but it's Simon's words, I'm going to let him go. What Simon says, right? So what do we need students to do? I'm going to suggest, based on all the research about learning and teaching, at least in the United States, is we need students to do six things. Regardless of what your discipline is, regardless of what your role is at university, we need students putting in time. Learning is not time light. Learning is time intensive, right? And they need to put in effort. Without those two things, learning is likely to be really superficial. If you're going to get to deep learning, you need meaningful time, meaningful effort. They need to do work that they find meaningful. The key there is they find meaningful, right? If you walk into my class and we're talking about 19th century American race relations, I think that's really interesting and meaningful. If my students don't, it doesn't matter that I'm passionate about it. So they need to spend time and effort doing things that they find meaningful, right? They need to get feedback on what they're learning, so that they can adjust their work, so that they can be appropriately confident or know that they have to work harder, so they can become more expert in their knowledge and behavior. They need to practice, and in this case, they need to practice in multiple different contexts. One of the biggest challenges of higher education is students come into a classroom like this one for a particular course, let's say history, right? And then I assess them, and even if I use a really sophisticated assessment, they know they're in a history course. So they're thinking, what do I know about history that applies to this question? When they leave this classroom and go out into the world, unless they're a professional historian, they're not going to be queued as to when to use their historical knowledge to shape their behavior, right? They're watching the news, and they're going to say, oh, that's probably a history point this person is making. And so what we have to do is give students practice in weakly queued environments. The more environments the better, but the less and less queues the better, so they have to learn to be able to use their knowledge, practice using their knowledge and their skills in lots of different settings, right? And last but not least, they need to reflect, specifically they need to reflect on how they're learning and what they understand and what they don't understand, right? So that they can guide their future learning. Because one of the challenges with undergraduate education is we have them only for four or five or six or however many years, right? But then they go on and they're supposed to use these things, and often they use these things professionally for a long career. So they come here and they study civil engineering, and they're going to build pipelines, right? And maybe, but they're not here after they graduate. But pipelines presumably will change in the next 40 years of their career. So how will they have the skills to adjust their own learning and to recognize what they know and what they don't know and when they should ask, how they could study differently, how they can learn more? They need to develop that metacognitive ability while they're with us. And they do that by reflecting, okay? So the point is these are the six things they really need to be doing over and over with us, regardless of where they are in the curriculum. Whether they need to think or what do they need to feel? The research in the U.S. says two things. They need to feel that they belong here, right? And that can be here at this university. It can be here in higher education. It can be here in computing sciences or physics or medicine or whatever it is or law, whatever it is they're studying. There's an increasing body of research in the U.S. about belonging that suggests this. You take two students who have similar performance on a first-year physics, first exam in physics, similar performance. One student grew up, both parents, both her parents went to UBC. She always knew she was going to go to UBC. She comes here, she takes her first physics course, she takes her first test, she doesn't do that well. She says, typically, professor's unfair, the exam wasn't very good, I should study harder, things like this, right? Student, first in family to go to university, not quite sure they fit in on campus, not quite sure about physics. They look around, there's a lot of men here, there's not so many people who look like me, etc. They get that same performance on the exam. And that student, if she doesn't feel like she belongs in physics at the university, often will say, see this is proof. I don't fit, I can't do this, right? Same performance, same feedback, different interpretation by students. The good news about belonging is you can cultivate it. You can help students develop a sense that they matter, they fit, they can do this. We'll talk about that, right? So they need to feel like they can belong, and then the second thing is they need to feel that they can learn what it is they're studying. That's a not very sophisticated way of referring to Carol Dweck's work on mindsets often, and if you know that. The idea that it really, really is important that students develop a sense that learning is something that develops. Knowledge is not something you either have or don't have. Intelligence is not fixed, but it's malleable, right? If I don't do well, the right response is to study harder, right? But that I can do this. And again, just like belonging, that sort of mindset, a growth mindset is something we can develop and help students develop. So what we say in the book, what I'm saying here, and we'll dig into, and you'll have some chance to talk about, think about in your own context, is how are you and how can we together help students do these sorts of things and think those sorts of things. Does that make sense? If we do that, we're doing pretty well on undergraduate education. Let me give you three examples of this, and then ask you to think about examples in your own context, what you do, what you could do. Peer instruction, a program called Grow at Iowa and Exam Wrappers. Now, peer instruction was partly invented at UBC. Does anybody do peer instruction in their courses here? Raise your hands proudly. And there's tons of people online raising their hands right now. Anybody willing? Well, here, let me say what peer instruction is, and then maybe we'll get an example of how you do it in your own courses. So the idea of peer instruction is it's designed to be used in large lecture courses. Where you periodically interrupt a large lecture, ask students a challenging, conceptual, typically a multiple-choice question. You have them work that question individually, often respond electronically, then talk to a peer about it. That's where the peer instruction comes in. Then respond again, thinking about what's right. Then you display the answers, their answers, and you talk about whether, what the right answer is and why. Peer instruction has been demonstrated to lead to more confidence and more understanding for students and better exam performance and better learning long-term. Anybody have an example? You're holding a microphone. So tell us who you are and how you use peer instruction. We have a first-year cell biology and we use active learning quite extensively in our classroom. And a few years ago, we were a part of the Biology Flexible Learning Initiative right here at UBC. But we do this quite frequently in our classes. So we pose a question on, I'm really blanking out for a really good example, but we do this quite often that we show an example. We ask them to work through a problem maybe and then show potential answers. We take misconceptions that we are already aware of and then we ask them to talk to their neighbor and about it and then let's say, repole, et cetera. So what happens when you ask the question in class? Do they stay passive and quiet? Yeah, so with the clickers, they're always highly engaged. Using the clickers and when you ask them to talk to each other, it's really loud and noisy and really interesting. If you then ask for their feedback, they go dead quiet sometimes. Yeah. So they'll talk to each other, but they maybe won't talk to the void of the large. Yeah, they're a bit shy about promoting their own responses, even their group responses later back to you. Is that similar to what other people do who use peer instruction? Any questions about using peer instruction right now? It works beautifully. It's hard to get the right questions, but as you're saying, the best questions in peer instruction, the wrong answers are common misconceptions. So you're not trying to trick students. You're trying to get students to surface their misunderstandings so that you can address their misunderstandings before you get to the exam when they put their misunderstandings on the exam. The research on peer instruction in chemistry that I like in particular demonstrates that if you ask students what the right answer is and how confident are you that you're correct, what you can do fairly quickly is get students to be appropriately confident and correct at the same time. So if they're right, they know they're right. If they're wrong, they know they're wrong. And why that matters, at least in the U.S. context, is because in chemistry, traditionally in the U.S., ethnic minorities and female students don't persist. They start in chemistry and they don't last as often as white male students do. By including these confidence questions, what happens is students become more appropriately confident and they do persist and they are successful because you can see this actually looking in data. There was a big study at the University of Notre Dame in the U.S. where female students introduction to Gen Chem, so Gen Chem, first week, peer instruction question, many, many female students were correct and not very confident. Second week, correct, not very confident. Third week, correct, medium confident. By about the fourth or fifth week, their correctness and their confidence matched and they stayed in the class. So there's ways things like peer instruction lead to not just conceptual understanding and more active classroom, but can actually lead to good outcomes for different students. It can be an equity intervention. So that's one example. A second example that gets towards this is an out-of-class example, but it's really powerful and it's really interesting in the research, I think is really good. Maybe if we don't work that much with students outside of class, we all interact in some ways with students outside of class. And this example comes out of the University of Iowa. It's called grow guided reflection on work. The idea is that the University of Iowa in their student affairs division, they have 2,500 students every semester who are student workers on campus. And they wanted to study to see do those jobs that students have on campus have any positive outcomes educationally for students besides putting a little money in their pocket. And what they found is it was widely differential. So they developed an intervention that they now do with all student workers at the University of Iowa. And the research on this is still fairly new, but the data is mostly self-report, but it's fascinating. Students report they're better writers after doing this even though there's no writing involved. They're more likely to graduate from the University of Iowa than if not. So there's some hints that this is very positive. And the idea is twice a term, every student worker supervisor sits down with a student worker and has a five to seven minute conversation around these four questions. One thing that I like about these questions is they're actually the kinds of questions any of us can use, right? So if I'm teaching history and I have students just come into my office hours, I can ask them about the weather or I can ask them about football or something, but I can say, so what are you learning? What do you want to do after you graduate? What are you learning in my class that relates to that? So they're good prompts because what the research overall says, not specifically the grow research, but the research overall, is that when we get students to think about what's really meaningful to them and what is purposeful for them, they'll be more motivated in their studies and they'll be more successful in their studies. There's a really cool new project at the University of Rhode Island in the U.S. that hasn't been published yet, but it's coming. In Intro to Biology, where they design the course around questions of meaning and purpose, so why are you studying biology was the central question. The failure rate in that course over the last four years has gone from an average 30% of the students failing to an average 4% of the students failing. The fundamental intervention is asking students over and over why is this important to you? So the point is how do we get students in whatever ways we interact with them, thinking about why is your education important to you and what are you doing now that connects to what you care about? Grow is one simple way to do that. The last example here is our exam wrappers. Does anybody use exam wrappers here? Same people who use peer instruction. So exam wrappers were invented as far as I can tell, Carnegie Mellon University in the U.S. in computing sciences first. And the idea is pretty simple, but it's fairly powerful that when we give exams back to students, that is traditionally not in learning moment for students. Well, they might learn that they should transfer out of biology or computer science or whatever. They learn they're not going to pass this semester. But it's not a moment where they often reflect on what do I know, what don't I know, how could I do better next time? If your students are like mine, they look at the exam or they look at the grade, we hope they recycle the exam. That's about the level of interaction. And so with exam wrappers, the idea is you put something on the front of the exam when you give it back. This is an example from Carnegie Mellon, a condensed example from Carnegie Mellon. So you ask students to reflect and note this. How much time do they spend studying and doing what as they study? Now, these prompts might look different in different disciplines, right? So that's one question you ask. And the second question, now look over your exam. What kinds of mistakes did you make? I've got colleagues who flipped this and said, what did you do well? What do you really know here? So it's not necessarily focused on the negative. But are you making silly mistakes, just sloppy mistakes, in which case you need to calm down and focus? Are you making conceptual mistakes? You don't understand whole areas of the field. Are you making mathematical mistakes? Who knows? Of course, like mine, are there writing issues? Is that what the problem is, right? And then the last question on an exam wrapper is always one like this. You have all your students do this and they have to, in my course now, when they get an exam back, before they leave, they have to turn in this sheet. So I give them 15 minutes to look at their exam and fill out the exam wrapper. They get to keep their exam so they can look at it more. But everybody fills this out. They get a tiny bit of credit for doing it. So they tend to do it semi-seriously. I do two things with it. One is I look at it as a way to try to understand my student experiences, right? There's two parts of that. One, I'll aggregate it up so that next semester I can say, you know, class, first day of class, in an intro history course, I can say, you know, students who got A's last year, they tended to do these kinds of things in their studying. One of the interesting things in my own courses, and this is true in general in exam wrappers, is the students who report studying the most hours are rarely the students who get the highest grades, right? And so it can be helpful to come back to class to say, you know, after the first exam, I noticed something. Students who studied 900 hours actually didn't do that well. Students who did well studied about this amount, but they report studying every week. You know, three times a week, they didn't just cram, for example, huh? Could you do that, right? So partly I'm looking for patterns. Partly I'm looking for things that I can counsel students on, right? So if I see a student who in response to this question tells me they only made careless mistakes and I've read their exam and they don't know what they're writing about, I might say, come see me, Peter. We need to talk because you don't just need to take a little more care in your writing. You actually need to have a thesis statement and evidence. Or a student who doesn't do very well but appropriately sees what her problems are, I might send that student an email saying you're right. Let me know how I can help, but you're on the right track, right? And then last is about 10 days before the next exam, I always give these back. Students have their names on them. I give them back and say, remember what you said you were going to do? How's that going, right? And it's just a gentle reminder. Enough time that maybe they can, it'll prompt them. Does that make sense? The idea is to turn, and you can, there's essay wrappers and all this sorts of things. There's ways of turning. I've seen people actually develop assignment wrappers. The point is to try to turn the process of receiving a grade into a moment to think about your own learning. So I've been talking for way too long now. So now I'm going to ask you to get in some sort of coherent small groups, probably threes or so, but you can do this. And talk, and we've got a little bit of time, so talk for five or six minutes with each other about these two questions. What's something you already do that gets students doing these things and thinking these things? Because sometimes when we're thinking about trying to improve our students' experience, we don't have to do new things. We need to do more of what we're already doing. And then sometimes you do need to do new things. So is there, is this conversation getting you to think about something else you could do? Does that make sense as a set of prompts? I'll roam around if you want to talk, but five or six minutes like that then we'll do a little reporting out and see how it goes. Okay, great. Come back together, please. And if you're far away, there are ways to tweet in comments and questions and things like this. But I'm curious, you had what appeared to be quite lively conversations going on. We had an interesting conversation about parenting and all this, and how to convince children that it matters if they spell correctly and things like this. So there's that. But I'm curious to hear from some of you, are there, one of two things. Are there examples of things your colleagues do or you do that you think we should do more of because they get towards these purposes? Or are there questions or concerns or challenges you have coming out of these conversations? And you have to use a mic or they can't hear as far away. Catherine's got a mic, oh, and yeah. Does anybody have, well first, the question was what do you do in a class of 275 rather than 30? I have some answers, but I'm curious if other people, so peer instruction is an example of something that works well with large classes. You do that all the time. I'm increasingly a fan of peer mentoring and peer tutoring and embedded tutors in courses. This is becoming an increasing thing in the U.S. where you have students, let's say, what do you teach? First year physics. You have students who've successfully taken first year physics and you hire them as different places call them different things. Peer mentors, let's say. And there's a group of them, let's say, eight or 10 of them who come to class at least once a week. And then you have students who work in small groups and you're peer mentors. You've already talked with the peer mentors in advance about what the questions are and what you're trying to achieve by having your students work these questions. And those students are roaming around and they're not just giving out the answers. They're trying to ask the kinds of questions to help guide the students towards learning in ways that multiply your presence. It's a lot cheaper than graduate students plus you can have these peer mentors introduce themselves on the first day of class and you can say all of them were in this class last spring or last year. Not all of them maybe got A's. Not all of them were sure they were going to study physics. And they're here now so they're in some ways role models and at many universities in the U.S. that are doing this now those students also have some degree of office hours. Like right before, right after class at some place convenient. Not a ton of office hours but because students will often go to them especially if they see them in class rather than come to your office. Who knew? Yeah, so some sort of peer mentoring system because they're false multipliers to use a terrible language. It's one. The other thing that I think can be really helpful is to use examples of student work. So to have students, let's say small groups solve a problem, right? And then have them take a picture. I'm not very technologically savvy. I'm a historian. But take a picture of your solved problem and tweet it to whatever, right? Or put it in wherever. And then you pull up an example of a solution. And you haven't seen this solution yet. But there it is. And then you and 275 students look at that solution and say, huh, is that the right answer? Is that the right way to get to that answer? Or not. And at some level it doesn't matter if the answer is correct or not because you're giving feedback to everybody about how to do it correctly. As long as they're not putting up, you know, obscenities or things like this and then you can identify them. You're okay because what you're doing is you're giving feedback to everybody on some student group's work at the same time. So you're using your time to give lots of people feedback. It's another way of trying to manage some of the scale. Because you can't. If 275 students, you can't say, everybody come see me and I'll give you individualized feedback. That's just not possible, right? But there's a lot of benefit in students seeing peer work critiqued in a way that isn't brutal but is opening. Right? There's exams where our students solve the exams in a group after they did their individual exam and our group was talking about whether we could make a module for our online learning system, essentially the exam wrapper, for our large classes. And so because they're not getting the reflection portion right now so if we could make an online wrapper and then the students could get a push from the learning management system later, whatever to say. That's a great idea and then phased in and out but the idea is create an online exam wrapper that the learning management system could just push things to them so it could be set up so that if you have an exam once a month you get the data from that, you can look at it, but LMS automatically two weeks before the next exam sends each student back her wrapper with a pre-written note from you saying remember what you said. The other thing and you mentioned two stage exams and so you're previewing the next part of the session so thank you. But the other thing that I've seen done is I mentioned tomorrow in passing and I don't know if this works in quantitative fields but in fields where students write a lot. Something that can be really really effective and work fairly well no matter how big the class is you take, I'll use my own course as an example, I have a rubric I use to evaluate student essays in first year history courses, right? Lots of people have that. What I do is I take unmarked essays from a past exam where the question is different and I might put two or three of those up in the LMS with the rubric and encourage students to go grade those essays, right? And then in class the next day I'm gonna talk about the grades I gave but they get to see real student work and they get to apply the rubric to it they're almost always meaner than I am which is a helpful thing for them to learn but by doing that activity they can then apply the rubric to their own work or to their peers work, right? And so the writing gets better from that. Okay, it's great. So this side of the room had things to say does this side of the room have anything to add or is there anything in the outer void? I love this conversation and one thing I really appreciated was when you broke down in micro data points you asked them what percentage or how did you spend your time? My field of work is diversity, social justice, equity and looking at stereotypes and implicit bias and especially the biases you have about yourself and how you do not see and nor can take in any information that you don't believe in. So I was at first worried when I heard you were going to talk about what they think because in my field what they think is what distorts their observations. I really like that idea of going for the micro data points and don't think, observe and now let's track your observation and then after the real date is there go back and think about it and what do you feel about it because a lot of people have false confidence as you're saying and a lot of people have false lack of confidence and that's gendered and racialized and all kinds of things. So this is great. I'm just wondering how you can get enough micro data points how you can stall off the thinking and enhance technical observation. That's a great question. It's really hard. Two broad approaches. One, there's an old reflection technique that's really nice partly because it's really easy to teach people which is what, so what, now what. Have you heard this? So it's a three-step reflection process what, so what, now what. What is purely description? So what did you see or what did you do? So what is, what does that mean? Now what is what are you going to do with this, right? And why I like that is you could ask students as you were just saying to describe don't analyze, don't say I always or I never or anything, what did you do? So how did you study for this exam? Where did you study? When did you study? How much did you study, right? Just describe that. Don't say you're bad, you procrastinate or you're good or something like this or you're bad at math. Just say what did you actually do? And prompting students to stay in that space can be really, really helpful. And then move on because often what students do because they're human is they say again I'm bad at math and I tend to procrastinate so I crammed the night before with my roommate while the TV was on and you're like, huh, how'd you do in that exam? Is that because you're bad at math? I mean, what caused that performance, right? And just getting them to sit in that what space often gets them to say, oh, if I studied for more than two hours the night before I might do better. I mentioned this peer instruction thing from Notre Dame and they did something that I thought was really clever and so I'm going to put it up here really fast and it's a six point scale, right? So you go six, five, four, three, two, one, zero, right? And so every peer instruction is one quiz answer and over the course of three weeks or so you get a quiz grade for doing peer instruction. You get zero if you don't answer, right? That's pretty easy. If you are correct, if you're correct and highly confident you get six points, right? If you're correct and medium confidence you get five points, correct, low confidence, you get four points. Incorrect, low confidence, three points. Incorrect media, unsure two points, incorrect, highly confident, one point. Does that make sense? But what happens then, and I sort of hinted at this, is you get let's say in the U.S. context African American woman first in her family to go to college in this intro chemistry course super smart, super diligent student. She is correct and she is not confident in the first week and she does two or three of these that week and she's like correct, not confident, correct, not confident. Damn, correct, medium confident. Why am I leaving these points on the, right? And you get other students who come in like I know everything and they get punished, right? But not terribly punished. And again, there's a geoscientist at McAllister College which is a liberal arts college, not large classes in Minneapolis who does this with students and he does it multiple times every class and he says, and he asks the same confidence question on the exams, he says by the second exam really he doesn't need to grade the exam, he just needs to look at how confident students are about their answers to each question because for students except for international students he says, for domestic U.S. students by the second exam it's essentially parallel. They know when they're right and they know when they're not sure and they know when they're wrong but they don't, the first week they're all over the map, right? And this is an intro geoscience course. So the idea is asking students is how you generate some of those data points because you don't generate those data points, they do, right? But they need to be prompted to say what do I know, how do I know it? Great question. Somebody else over here, yeah. Thank you. I'm not an instructor, I'm actually a recent graduate so I thought it'd actually be nice to get sort of the student voice in here. This may seem a little bit obvious but I think one thing that's really important about achieving all those goals is providing meaningful incentives and I think that oftentimes when students don't learn effectively and achieve those goals it's because they have sort of distorted or incorrect incentives. I think part of the problem is that we have an incentive to get a high grade and not an incentive to learn. And so I have a professor who, what he likes to do, I had, is he'll actually give us a set for the exam and this looks different across different disciplines I'd imagine. He'd give us a set of seven questions and an essay based exam. Two of those seven questions will be on the exam. So you have a strong incentive to pre-write and learn and have the most effective answer but the problem is that you don't know what questions are going to be on the exam so you actually have to pre-write seven different questions and one thing he likes to do is make sure that not only those questions are actually clear but they're also, you need to be creative because they're not readily apparent what the answer will be. And so that gives you an incentive to reflect and try and figure out how you can actually apply that and it also gives you a incentive and such that you learn all of the material in the course and so I thought that was really effective and something that I might want to just kind of suggest. I think that's absolutely essential I'm really grateful for you sharing that. I recently learned of an old business article it's from the Annals of Management I think is the journal and you can look up the article you don't really need to read it the title is so good it's on the folly of rewarding me right and I think often in our courses we do this I took a year of college physics sorry Simon and my roommate after the first exam said that was funny and I said what was funny and he said what has solved the problems on the exam the formula has appeared in the same order in the exam as they did in the book I'm good at algebra and memorizing stuff and so I got an A in two semesters of physics anything about physics because what was being rewarded really if you knew the code was the capacity to memorize and repeat so there's negative examples of that but there's positive examples like this 7 that is really nice and so I appreciate that so what do you really want students to know what do you do? last comment from me on this I've been practicing myself for many many years I would say to my students I wish I could give everybody feedback on drafts but I don't have time to give everybody feedback on drafts I don't teach 275 but I have too big of classes for that so I would always say to my students if you'd like me to give you feedback on the draft of your essay before it's due get it to me at least three days before it's due and who would give me their essays the good students and then their essays would get better and I would get frustrated and one of my colleagues said to me a few years ago maybe it's not the good students who are doing it maybe they're good students because they do that and so I shifted and now I require my students to turn in an essay a draft and get feedback and I spend less time grading later and more time getting them feedback the point being if we want our students to behave like good students why don't we create incentives for them to behave like good students right if you want them really to learn how do you create incentives for them to really learn in sophisticated ways rather than to learn in rote ways okay I know we could talk more and more about this I just want to make one point before moving on which is I think if you look at undergraduate education science fiction writer but I love the quote in this context there's lots and lots and lots of good stuff happening in undergraduate education at UBC or any place the problem is it's not evenly distributed so how do we do the best stuff with all our students that's the real challenge rather than often that's a bigger challenge often than figuring out what we should do how do we do it with everybody okay so talked about learning we're going to talk a little bit about relationships and alignment this is much faster because it's not as it is more complicated but we don't need to get in quite as deeply there's quite a lot of research in the US higher education that says some version of this right so if we really want students to learn anything we need to recognize that they're human and as humans they're motivated by relational things in various ways and so how can we help support and create conditions for relationships that are really educationally purposeful notice that if you're teaching 275 that doesn't mean you have to be friends with everybody it's about creating conditions where they can learn from each other in productive ways yeah yeah exactly right exactly this so how do we not punish them for wanting to work with each other but create conditions where they can be educationally purposeful in those settings so let me give you two examples of this the first one is pretty low level the second one we already heard a tiny bit about first is note taking pairs and when I heard this at first I thought it was sort of silly but then I watched my students and especially in intro courses many students don't take notes and I thought at first they're just bad students and then I realized they often don't know how to take notes they don't know why to take notes they don't know what it means to take notes when my daughter was in high school she had a teacher who would write stuff on the board and say now you're going to practice taking notes write down everything I wrote on the board like dear god this is what note taking is so the idea of note taking pairs is really simple and again it's really scalable what you do is you tell students in this intro course periodically you're going to share your notes with a couple peers it could be peers it doesn't have to be but the idea is I'm going to stop class and say look get in small groups and look at each other's notes on what I've just been teaching I've been lecturing about notice the content notice the form how are you doing that and talk about that like you know if they've got something down that seems important that you missed well copy that down right but if they're doing it in a fundamentally different way talk about why in which way is the best way to do and then ask for volunteers or go around and you know take pictures again or something of some good examples and then put them up and say look there's different ways of taking notes in this class what would you do this way versus that way there doesn't have to be a correct answer to this right but first of all you're creating a norm that of course students take notes in this class secondly you're recognizing that there can be variety in it third you're building those relationships right so if you miss class you know someone and you know what kind of notes they take you know what to do right so this is a super quick simple back to your point of creating an incentives for students to behave in the way you we want them to behave you don't have to grade this it's just showing them you should do this and what it looks like when you do right more complicated is two-stage exams we heard a tiny bit about this already a lot of their research on two-stage exams actually comes out of UBC or originally came out of UBC and it's super powerful the idea is again instead of turning it having an exam be a time where you just demonstrate that you know things it's a time where you both demonstrate that you know things and you actually learn some more right and so you take a 90-minute class session and you create two exam periods within it the majority of the time is individual exam that's the majority of the grading and then another portion is typically students working in pre-established groups often on harder questions sort of extensions of the existing questions or perhaps you start with the exist some of the existing questions that are absolutely foundational like everybody needs to know this right and then they come out of that how many people use two-stage exams now some of you more of you yeah did they work fairly well anything I didn't say that we should speak to the room about the two-stage exam process yeah students love them yeah the room dynamics are amazing there was a big study that came out last summer in the U.S. that spoke to one of the challenges of this which is isn't this just helping the bad students improve their grades and this looked at the kind of learning students did in two-stage exams and what I appreciated about the study is they said if a student had done really well on the individual exam it's unlikely their grades going to go up right so it was looking not just at performance on those questions but on conceptual knowledge and what I found this study said is that all students learn from two-stage exams but different students learn different things if you come into the exam not knowing some of the basics you're likely to learn basics if you come into the exam knowing the basics it appears that the group problem-solving activity on harder questions actually helps you learn conceptually harder stuff does that make sense so it's just not it's not only a way for the high achieving students to help the lower achieving students it appears to help everybody right right so all the students get something out of it reinforcing knowledge and things like this disciplines can you just yell out your disciplines computer social justice, physics engineering biology health, something in health sorry languages, really awesome lots of variety you have to think about what you're asking students to do right and so the group activity portion of this should be constructive and again should extend or ask for more complicated things and it should be something that everybody can submit together I've actually I've talked to some people I was at Florida International University a little bit ago and talking to some history faculty who actually have flipped not flipped two-stage exams in the flip way but they have the group part first and so they have students talk through some of the core knowledge stuff together and then most of the time is individual essay writing right but part of what they're doing they say is they want to warm students up and they're students who get sort of hung up on what they know and don't know and don't feel very confident about their writing and when they come in and they face the blank page and 200 people in the room scribbling they think I'm terrible at this and they come in and they spend a little time talking about what they know and it's sort of like a quiz it's not so bad and then they're warmed up and they're thinking about colonial US or whatever and then they get into the essays and they feel more confident can you speak to how people should be organized into groups as opposed to allowing students to just default into who they know this is relationship building so two stage exams and group building does anybody have particular techniques they use really well random how big are your groups the groups 5 to 7 random 30 groups of 5 or 6 random is it just where they sit or do you do it randomly where they have to move around I use to connect to randomly form groups or I here I forget which one okay so there's technical tools that just put them into groups so it's not just your friends walking in the door yeah that can be important I have them line up by learning confidence along the wall and then random assignments so that I make sure the learning confidence is spread out sorry again we have people who have in engineering those who have work experience and those who don't so I get them to stand up in front of the room those starting with those who have any work experience in engineering or even in somewhere of an industrial setting and then I get all the rest of the students to stand up kind of next and then I just go 1, 2, 3, 4, 5, 6, 7 1, 2, 3, 4, 5, 6, 7 all the way through and or actually 1, 2, 3, 4, 5, 6, 7 3, 4, 5, 6, 7, 1, 2 5, 6, 7, 1, 2, 3, 4, 5 she's an engineer she could do that so that they don't realize the pattern and then go with their friends in the pattern and then it helps that there's one person, one or two people in each group who actually have more experience who are typically the stronger students who can help then pure instruct the other students that's how I do it that's great and there's a really important point in this which is if you are concerned about variation within your class you know if it doesn't matter because everybody is essentially the same in whatever ways that matter in this class random is great if there are different attribute students might bring in with them like work experience or gender or ethnicity or who knows what you might want to think about constructing in ways that attend to that right so that there's not one female student in every group of men or not you know that's up to you there's a lot that's been written about how to do this well in different disciplines too so it's worth attending to personally allow the groups to form and we have three exams two midterms and a final so after years of doing this I find that even if they get together and friends in the first exam they often are in a different group by the second and third exam because they know that relying on their friends is not necessarily the best course of action so I've seen lots of groups that are completely different because we'll have 80 groups in any one section of that exam so yeah so great does anybody use peer feedback for the group exams like to monitor how people are participating within the group so some sort of structure where you say you're dominating or or participating not participating does anybody do that here yeah Simon what I would say is not formally generally when we run this with large classes there'll be a couple of instructors and maybe some TAs and you circulate round and you just observe and interact with the groups where sometimes you get the case where there's two people diligently working and two people staring into space so it's an opportunity to suggest that this is a collaborative and a group exam and rather than anything formalized where the peers have to rate each other for how they've engaged I just wanted to add one more thing to something Karen said I also use this and having tried various ways of assembling groups have settled on on random or at least self select chosen by the students and one thing I've observed is people often tend to get hung up on what happens if the groups are very mixed ability in terms of knowledge and that's you know people can try very sophisticated things by you know lining people up in terms of performance and randomly allocating what I've observed is something that matters at least as much of that as that is how the groups actually function as a group or not and so the starting point for me is to say we do group conversations for peer instruction group interaction a lot in classes a good place to start would be the people who you sit near and there's an opportunity before you even get to the first two stage exam to talk about it in class and say the reason we're getting you to talk to each other is you're going to be doing this in a high stakes assessment situation and part of what you need to pay attention to as a group is how you discuss and debate things because it's fine for a peer instruction question to go off on a tangent and not really talk about the question but if you do that in the two stage exam which is time time crunched there's enough time but it lets students practice being in that type of an environment before they get into the two stage exam situation and I think this is really important because what you're doing is you're giving students so the way you're describing your course by using peer instruction that leads into two stage exams you're giving students lots of practice in here and then you're having some explicit conversation about so what do you need in that peer portion and what works for you and what doesn't just one last point on this which is thinking about whether you have or not because if you have stable groups so students are going to be in the same group all the way through the term then things like group composition and group dynamics are really important if you're just having students randomly getting groups then it doesn't matter so much and if there's a lot of variation but if you're having stable groups if you don't want to spend a lot of time doing group therapy you need to think about what's it mean to be in a productive group the interesting thing and this works at scale too is to get students to talk with each other about so how are we going to treat each other how do we behave and then have them write those down and encourage them to talk about it that doesn't mean they will enforce it but at least they'll say when we talked about this before the first group exam we said everybody would contribute everybody didn't contribute now whether the groups change their dynamics are not but you can give students skills and capacities with some of the structure you do the point of this which is great is to think about so how do we put students in educationally productive relationships with each other and we can put them in good relationships with us but we're not scalable they are so that's one thing to think about the other thing to think about here is this question of alignment and there's fancy ways of talking about alignment at some level and some people use the phrase constructive alignment which you might know at some level it comes down to a question what difference does it make they're not just taking your physics course and then someone else's random physics course as someone else they're in a curriculum and they're in a program here and so how do the pieces fit together and I've been on lots of campuses and talked to lots of faculty and they often say this is so hard and I agree with that but I also think if we can't do it why on earth do we expect our students can make sense of their four years studying whatever it is they're studying here if we can't one of the charms if you talk to a lot of students is they assume everything at the university is coordinated and planned and they're like oh it all fits together and you're like that's awesome I wish right but this is really really important let me give you an example from a big study that was funded by the Gates Foundation in the US I'm looking at Georgia State University Georgia State University is very large very very large classes we're talking 40,000 undergraduates essentially open access institution and in this 10 year period from 2003 to 2013 funding for Georgia State went down their student population became even poorer and more diverse economically ethnically and educationally yet their graduation rate went way way up and the Gates Foundation is like this is golden this is what we need to study so they commissioned this big study and the conclusion of the study was that no one thing did this what did this is the accumulated impact of a dozen or more modest programs right the idea that everyone on campus or everyone in a program sees that their work is to support students in learning developing power for relationships and pushing them right challenging and supporting them because we can't all do that for everybody but if we all do it together for all of them we'll do better let me give you three quick examples of ways to do this and again some of these are at scale the first ones from the University of North Carolina in Chapel Hill which is pretty similar to UBC I think and what they found at Carolina was that transfer students so students who entered Carolina not as first semester first year undergraduates traditionally did worse at Carolina than other students even if they had the same background in every other way so they created this program it's called Transfer is United it's really clever and so transfer students come in and they take a one credit seminar their first semester and the core assignments of this seminar are to do these two things first identify a student in your field of study who's successful you get to decide what it means and interview them about what they do where do they study why do they study how do they get to be successful right and then the transfer students all go and they come back and they share this and it turns out there's patterns who knew of what successful students do across majors but these students are talking about well successful students at Carolina tend to do these sorts of things then the next assignment is to study yourself and basically keep a journal of yourself and what you're doing to be successful they've been running this program for about six years transfer students now graduated a higher rate than students who come in as first semesters new students right but it's really interesting just as a way to prompt students to reflect and then practice what good students do so it's using in this case using students to develop these capacities in themselves a second example is from engineering it's from Ohio University and part of what I'm going to tell you is going to sound really traditional good engineering education I think part of it's sort of extra clutter so this is a year long senior capstone experience fourth year capstone experience where the students in mechanical engineering and civil engineering mechanical engineering mostly I guess do community based projects where they're trying to work with community groups to solve real community problems using what they've learned in mechanical engineering right nice program just good the coolest part about this is they work with first year students so this the fourth years come into introductory mechanical engineering courses several times during the term each term to pitch their ideas and get feedback from the first years and they say to the first years we're assuming you're like community members you really don't know very much about engineering and that's good right so give us feedback on this but the first years get to see fourth year students using the various things they learned in engineering in somewhat sophisticated ways they also hear the fourth year students talking about that it's not all about the math which is helpful right and so the first year students motivation and engagement changes the fourth year students get really helpful feedback before they go out to the community and pitch their ideas does that make sense so it's a clever way of bringing first and fourth year students together in constructive ways for both sets of students it takes a little bit of first year student class time that's it there's a nice chapter about this the last example comes from my own institution I'll do a quick version of this we're pretty different than UBC we're 6,000 essentially only undergraduates highly experiential so students are required to do multiples of these internship study abroad mentor research outside of class service learning or leadership experiences in order to graduate it's part of our core curriculum you have to do multiples of these things right and we were going to change the number they had to do a few years ago and so we did a study of which students are barely doing enough of these to succeed to graduate in which students are doing way too many but students who are barely doing enough had certain things in common they tended to be male they tended to be first in their family to graduate or to go to college and they tended to be African-American so the director of our core curriculum a wonderful faculty member did what faculty do he sent a survey to first generation male African-American students saying why don't you participate in these things how was the response rate on that survey right and he was complaining actually in one of his student workers in the office who was first generation African-American female said to him why don't I talk to them so they created a research team the students on the research team were all first gen students the faculty on the research team were from the core curriculum committee and what they did is they developed an interview process and they interviewed first generation students on our campus who are juniors and seniors about their attitudes towards different experiential requirements and what they found was really really helpful and really really powerful we could go in and on and on that article by Porter Porter was an undergraduate when she published in Change Magazine which used to be published by the Carnegie Foundation she now works for a bank she wasn't even an education major she was the lead student on this project but what she found is first generation students coming to us didn't for example didn't see themselves as researchers so this idea that they would do undergraduate research that's for other people I'm here because I'm going to get a job and by the time they understood what undergraduate research was they were seniors and it was sort of too late to get involved so we've changed the way we talked to first to first year students about undergraduate research in response to this and part of what was interesting about this is there were times they interviewed about 80 first gen students there were a couple times where the student interviewers were six or someone who looks a lot like me including sometimes me stepped in to do the interviews and we got different responses from students then their peers did and they thought we wanted to hear not what was real so the point here is if you're trying to work on complicated things and understand your student experiences working with your students to understand their perspectives might be really helpful okay so we got a little bit of time so we're good so can you go back to your groups you can change if you want if you didn't like the people you were in the group with the first time it's fine but talk for a few minutes about these two questions can you write something down what's one thing you can do in your own to enhance student learning this could be something you're doing in your classes it could be about building relationships it could be about thinking about how your work aligns and then what's one thing you could do with your peers does that make sense five minutes great, oh there we go hi there I could hear you talking and laughing that was good I feel bad about interrupting there's posters that are coming pretty soon now so were you able to think about things you could do on your own or together yes, I hope do you have questions or concerns or thoughts or suggestions at this point please yeah these are all teenagers often, I don't know UBC if we're talking undergraduates 2019, 1918 and keeping that in mind that they're not adults and that all of the anxieties and peer pressures and social pressures because there's school when you're at university there's two things there's school and then there's party or whatever the social sphere which are equally at least equally important it was to me and so when we're looking at trying to enhance their success you also have to look at the social sphere and how they're interacting outside of class because all of that comes into play if your life sucks outside of class you're not going to do well and if all you do is go to the parties outside of class and the book doesn't really sell you but the book tries to look broadly at this student experience not all students not all undergraduates are 18 to 22 and actually but a lot of them are and different institutions are different but at many campuses it's really really obvious where to go to go to the parties and it's not at all obvious where to go if you want to do undergraduate research but what we do to put lights and music or the equivalent around the undergraduate research office so that students are like I don't know what's going on over there but I want to go find out because the parties do that they've got that figured out and I'm not saying we turn undergraduate research into a party but that we shouldn't hide these things if there are things we really want our students to do we need to help 10 year olds understand how to find them I propose that one thing we can do with our colleagues to enhance student success is not refer to them as teenagers or as kids because it doesn't empower them as independent learners that we should as a community make a commitment to calling them young adults or experts so I think that it's important to not take on a parental relationship with your students by using that kind of language it's tricky but yeah you want to oops there's a mic coming behind you oops there's really cool research about that age group 18 to 25 18 to 29 that emerging adulthood it's really cultural as well but also across cultures I will not take more space but really really interesting and really fits with the population we have at UBC no it's I think I'm really glad you brought this up there's two kinds of research on traditional age university students in the US that is really interesting and really promising and really contradictory there's research on emerging adulthood which says that 18 to 20 some year old, 20 some things often are trying on different identities and wanting to experiment and wanting to be challenged and all this there's another stream of research there's a pretty influential book in the US called first year out that did a really deep dive ethnography into 200 first year undergraduates at a whole bunch of different schools and it said many of them put their identity in a lock box because there's too many things going on and so they don't want to think about big complicated things and they don't want to experiment they just want to make sure they go to class on time and do the laundry and have someone to be friends with so it's complicated but it's helpful to recognize the complexity of their lives the auto-anoprophy process not just helping them or enabling them to be more an adult learner but at the same time addressing the sense of belonging that you've been articulating at the beginning that they realize in fact they are kind of part of the community and they belong to here and this is where they are yes no I really appreciate that and I want to be attentive to time so I'm going to do a little bit to wrap up but I want to take that really helpful comment and go back to the beginning and say you know that quote from Herbert Simon about what students do and think matters I don't know about your students in the US many students come to university not having any sense of agency about their own learning because the K-12 system in the US has taught them to do what we tell you and perform well on these multiple choice high stakes exams and you'll be fine right and so they come to us and they're like no my job is to sit here right and the idea that I have agency and that I have say in this and that I have to do things that matter is alien to them and I'm not blaming them I'm saying the system brings them to us like this sometimes and part of what we have to do is challenge them to say so what are you going to do with this on my campus we have a first year convocation we get all the students together and the president traditionally says you know you're at this amazing banquet don't make yourself a bologna sandwich right and it's it's sort of cute but it's the point is the students have the opportunity right and we have to help them but one of the ways we help them doing this is treat them like adults right and challenge them like adults so I want to close with just two well two and a half thoughts one is a quote from an interview that I did with and I actually conducted this interview with a student at a different university once this is my favorite thing almost a students ever said to me in an interview if we're trying to help our students learn physics or computer science or history or something like that we're experts it's really hard for us to remember what it was like to be a proto-expert in our field and we're not 18 and the world is different than when we were so how do we help them see themselves as the agents the actors in this work not them see us as the ones doing everything right and then the second is questions around meaning and purpose and the more I talk to undergraduates I think this is a trend at least in the U.S. there's a beautiful book about undergraduate education in the U.S. called Big Questions Worthy Dreams and the author Sharon Parks interviewed lots and lots of students mostly at Harvard but that's okay and what she said is a lot of these students come to university with big questions about the world and themselves and all this and dreams of what they're going to achieve and then they come into our university and they come into our curriculum and they go to Intro to Psychology and they go to Intro to Physics they go to Intro to Computer Science and Intro to whatever and they're with 500 other people and they're being talked at and what they say is this is not a place where I can explore my big question and then they opt out so they stay with us but they don't try they don't engage and the point is the more I think we can prompt our students to ask questions about what are you doing and why are you doing this the more motivated and self directed they'll be as learners and actually the better citizens they'll be after they graduate and at least in my country we need a lot better citizens than we've got right now Thank you very very very much Thank you Peter that was such a wonderful way to spend a couple of hours I think a really great example of how to engage in dialogue in discussion some really thoughtful points we have time for a couple of questions what I would say is I will keep it to a couple of questions Peter will be staying will be looking at the posters outside so feel free to intercept him and talk with him if you have comments or want to discuss things that might go longer than shorter questions now but a couple of questions I know we've had plenty of discussion and dialogue but any comments or questions people wanted to make before we close we're going to stay here until something no we're not we were discussing a physicist a biologist so are there any studies that differentiate with if a student does these types of things in some classes but not others then what kind of outcomes or success do they have so does it take a village and everyone buying into this process or does it make a difference even with one class great question I haven't seen a lot of because the practice is so mixed and because American higher education is so diffuse and decentralized I haven't seen much meaningful work about sequencing in things like active learning in sciences right there is quite a lot of research in writing and in writing instruction at the university level that if you can explicitly teach for transfer which means helping students recognize that what they're learning in this course is going to connect to what they're learning in the next course and what they're doing and then in that course pointing back you learn this already or we're developing this further and pointing forward that you can get students to do much more sophisticated work by being really explicit about the writing transfer the writing transfer research is sort of horrifying in that there's a number of studies that have first year students who are doing essentially the same assignment in two courses they're taking at the same time and they don't even recognize that it's the same work because they think this is history and this is English and those are different things so it doesn't dawn on them to look across so the research on transfer is you can do it no students can do it but we have to queue it and we have to plan for it one example again from that work is about the language we use lots of times different faculty will use different language to describe the same thing and there might be good disciplinary reasons why we do this but we often don't clue our students in on that right so if we could say in this class we're going to call it this you took biology before it's a very similar thing that you already know that this is similar this is different right but looking at the writing transfer research might be really helpful I can get you some citations if you like yeah so I'm talking a bit more about belonging as students belonging to an academic community and your discipline we I've been mining engineering and we typically we have some very keen first-choicers who get into our department we also get some people who mining chose them that's how we put it, mining chooses them and I think we struggle with trying to help those students find their sense of belonging and do you have any suggestions for that good luck I think I'm not sure this will go beyond what you're already doing I think the more we can have students tell stories about themselves and how they got there so having for example recent alumni talk about how they got into what they're doing and why they're doing it and have some of those stories be like they will be that mining chose me and then eventually I chose it right so that students can see themselves in this work and see a path forward in this work the other thing and again I don't know if this would work in your context I really do think asking them about what they're trying to do what are you trying to do with your career why are you studying engineering at all what are you hoping to achieve what are you hoping to contribute and they might say things about wanting to make the world better more sustainable environment things like this and then you can get well you know what mining engineers we do that or we can and so getting them to articulate things and then you connecting back to it I think one of the key things in teaching is in some ways getting them to say the sorts of things we would say as experts not because it's good because we say it but because when they articulate and then we reinforce that they believe it more so how do you get them doing some storytelling there's there's this really interesting project on purpose in and a sense of vocation not religious vocation but a sense of students having a sense of vocation what they study and what this study says and it was a couple hundred universities is that vocation is a narrative process right so what you need to do is have students articulate their own stories even if they're not telling you the truth even if they're just answering the question because you're saying you're in mind and engineering and you do that regularly some of them will talk themselves into things or or describe things that actually are meaningful to them does that make sense so give them the chance and some of them will take it