 Today, the objective is to share some information on different ways to teach to talk about the implications of active versus passive learning and the implications of what happens when there's a mismatch between instructor and learner preferred styles. I'll be using the course I designed to teach as an example of how to overcome the challenges inherent in all of this to deliver interprofessional education. And as you may recognize from the picture on the left, this is the reconciliation totem pole that UBC constructed. So obviously we are benefiting from the knowledge generated in institutions of higher learning such as UBC and the ancestral lands on which we all stand. It might help to understand why I teach this way to understand how I got to this point. Like everyone who teaches at a university or in some university graduate degrees. But perhaps unlike many people who have a straightforward academic career, I've had about seven careers. I started out as a lab tech a microbiologist in a clinical lab about the same time that the modern era of hospital infection control is expanding. So after a few years in the lab, I moved over and spent a few decades involved in leading infection control programs. I've also been a SCUBA instructor when I got tired of working for hospitals. I became a consultant. I did commercial software development in that sector. I have as a target of research opportunity periodically done other jobs. I was a state health department program manager for one of their inaugural programs. I've been a journal editor internationally, search and crew member. And of course university professor. The reason I go through all this is a lot of the environment that I learned in was not straightforward stage on the stage memorize what the instructor said. It was more like apprenticeship was more see one do one teach one. And there's a value in that for some of the things I actually can't speak teaching SCUBA underwater, for example. So a lot of things have to be set up as a safe learning environment to explore and then ways of leading people through those in terms of challenging students and bringing interdisciplinary students together. I've done research methods courses for public administration students. Instruction to business administration students. Data analysis vis a vis statistical analysis for MBA and nursing students. And I've taught epidemiology for public health and nursing as well as the course we're going to talk about today. So from a variety of experiments to what works what doesn't work. I stick years ago moved into SPBH. What I've been trying to do in everything I've taught is create a higher level of understanding. You're probably familiar with Bloom's taxonomy of teaching and learning, circa 1956. You may be aware of Anderson et al updating and expanding that in 2001, both very valuable references. So summarizing the more complex tables. There are basic levels of understanding and it's not what I'm interested in teaching so knowledge. Can you remember a word? Can you define it? Can you repeat it? I've actually inherited several courses where the request was revamped because people come out able to speak about these concepts, but not able to do anything useful with them in our work environment. So analysis, synthesis, evaluation. I'm trying to get my students to attain higher levels of understanding through the courses that I've developed. A watershed moment and epiphany for me came in 1999 when I attended the seventh conference on distance learning and public health put on by a couple of American government agencies. It was an interesting conference because within the first hour the attendees hijacked the conference and we changed the name of it. We pointed out this was not about the difference between classroom and distance learning. This was about the difference between active and passive learning strategies and that active learning is more preferable and could be applied anywhere in a classroom or elsewhere. So as you're probably aware, passive it's culture form that the instructor knows everything. The students with their little tape recorders listen to the instructor with the big tape recorder and then there's a game. Students over the years learn to get A's by figuring out the game in the course. What's going to be on the quiz? How does the instructor like things answered? There's usually a closed book exam to show you've memorized what the instructor said would be on the exam. There is a role for closed exams. There is a role for multiple choice exams, but it doesn't lead to as rich a learning environment as the active strategy. So here the instructor has a different role as a coach or a consultant consultant in a structured learning environment. And those learning environments are very complex and difficult to build. It's not a simple easy task. Students take a role in defining and satisfying their own needs. They bring curiosity. They're also ideally going to participate in performance evaluations along the way. And the idea here is open resources. So we really want a different consultative sort of relationship to develop. These are not new ideas. Self-directed learning, as you may be familiar, especially if you have gray hair like mine, Malcolm Knowles was the great white father of the concept. And in 1975, he proposed this definition of self-directed learning. So at this point, feel free to open your microphone. Do you have prior experience as a student or as a teacher encouraged in a self-directed type course rather than just a lecture type course? So does anybody have prior experience? Maybe when I was a student, undergraduate student many, many years ago, I see the word directed here on the slide. So taking a course on direct study or undergraduate thesis was a whole course that I can do it on my own with the directions from the supervisor. I think if we're talking about a whole course here, David. One approach would be an education degree, a pedagogy-oriented program where you're taking a course devoted to the topic. The other would be any other subject where the whole concept of education is self-directed. You put forward lessons where the students have to identify what they need to learn, what resources they need to find, formulating individual learning goals, identifying what's pertinent, choosing and implementing strategies, getting a better understanding. I'm at a cognitive level of understanding. So it can be addressed in a course on the subject or it can actually be a way of teaching anything. Some things don't work as well in self-directed learning. People need to be led by the hand. Statistics wasn't about to bear that way. But other things work really well in this for some people. According to Knowles, if this is going to succeed, first off, adults must understand why they need to learn something. So as an instructor, you can't just say, read the following and memorize these things. There has to be an understanding of why this is important if you're teaching adults. You have to set a way of adults taking responsibility for their own learning. I mean, I'm not here as a kindergarten teacher, so I'm not here to remind you that there is a course outline. I'm not here to remind you every time you ask a question that's answered somewhere else. I'm here to point you to the resources that you should be using. Some people like that, other people don't. So I'm taking the idea that I'm not the nanny. The individual student needs to be given the resources, need to be given an orientation, but they need to take responsibility for their learning. I need to be available, office hours need to be generous. So if they want to discuss an issue, a question, a stumbling block, they have an opportunity to do that. But I don't want to put them in a role where they can be passive, where I'm just lecturing every week, and all they have to do is read the course notes. Students in an adult learning environment also need to exploit their experiences as a resource. So drawing from what people already know, incorporating new knowledge into what they already know, seeing how a new understanding can expand what they already know is important. So this way of doing case-based studies in medical school, for example, where you link the readiness to learn with real-life situations. And as we'll discuss later in the hour, that's what I've done with several of the courses that I've taught. And you want to orient their learning by life tasks. Again, this reinforces this is going to be useful for the following reasons, as opposed to here's just a bunch of stuff to memorize. Now I've talked about a structured learning environment. It can be done in a variety of different ways. So for example, in a distance education course I taught for nursing, a thinly disguised epidemiology course, Nursing and the Health of Communities. One of the things in there was the ability to investigate an outbreak. So there was a computer simulation. For a couple of weeks, my students lived inside of a computer simulation, and day by virtual day in that simulation, they investigated an outbreak. The design of that software was clever. I mean, at some points, for example, if you tried to open your office door and go out and do something, there was a maintenance guy changing a light bulb blocking the door. You couldn't get out because you forgot an important step. They didn't tell you what that step was, but now that you're stuck in your office, what am I going to do here? You can review what you're doing. You could chat with the instructor. You could figure out what that missing step is and then the maintenance guy went away. So a computer simulation that's well designed is one way that you have a structured learning environment. And then in the end, my students did the outbreak investigation report, submitted it to me, and I can see how well they did overall. Another approach is actual cases. So again, in the nursing course, I gave as required reading the provincial health officers annual report on the health of British Columbians and the provincial health officers supplementary material on Vancouver residents. I asked my students in that course, pick one topic in that report. You can agree or disagree with the recommendation in there, but I want you to write a one page briefing document with references telling me why you support or do not support that action and what would have to happen to make that action actually work or to block that action effectively. So it took the material from the actual case, took that information into a format that could be interpreted with an epidemiologic understanding, and then asked for an action outline, what are you going to do? An example, there was a entry in there on a benzodiazepine overuse in nursing homes. We sedate the patients to make them easier to work with. One of my students, a nurse, said, you know, I work in nursing homes. Now that I've seen this, yeah, that's true in my home as well. And I've really looked at what are in ABC, my nursing association has done. They don't have a statement on this. So she submitted her home or an ABC as a prompt, you need a position paper on this subject. So again, this can engage them in a purpose, identify their own needs, their own value and action. MBA 540 was my attempt at teaching statistics to a group that really didn't like being taught in that way. But one of the things that I included was an assignment. Here's a classic story. It's not really well explained in what my assignment is. You've got to do the research, then you got to come back and do an oral presentation. So an example of that, one of my favorite stories was a guy named Abraham Wald. If you've taken statistic courses, you're familiar with things like the Wald test, but few people, including me, after years of studying statistics as an end user, knew who Abraham Wald was. Turns out this guy escaped Europe just before the Nazis were there to kill him, got a good job in an Ivy League university. And during World War II, in the front lines, the Air Force decided their planes were just being shot up. And they needed to think about where to add the combat engineers didn't really know what to do. So they said, let's get a statistical expert out here. So they recruited Abraham Wald, this little geek with a pocket protector, flew him out to the frontline airports and said, okay, you know, here are the aircraft coming home. You can see the bullet holes in them. Where should we add armor? How should we approach this? So my favorite example there is because his answer was, these are not the airplanes I want to see. And that stopped everyone in their tracks, including my students. Well, I would then ask my students and, you know, if we had more time, I would ask you, all right, why did he say that? But what he really was implying is these aircraft made at home. I don't care about the bullet holes in these aircraft. I want to see the other aircraft, the one that didn't make it back to base. Well, I'm sorry, Dr. Wald, we can't take you there. That's behind enemy lines. So what's the next thing he did? And again, I would challenge my students to brainstorm. He then devised several things that had not been done before. He realized the data set he was looking at was not what he really wanted to analyze. It was the complement to that data set, the aircraft that didn't make it back. Since he could not see them, with a bit more math involved, he came up with the conclusion, all right, add armor, we do not see bullet holes in these aircraft. His premise was they can survive all this damage. Don't add weight, don't add armor that will degrade performance. But where you don't see bullet holes, we will presume that's where the other aircraft were fatally shot down, or those spots. And that actually turned out to be the key to modern aircraft design for fighters, things like the Warthog that attacked tanks. So again, a classic story, the AHA discovery moment, frustrating for some students till they get used to it. But in the end, it was useful to the students. Whoops. The complex group project is another approach. So in another course that I taught, I actually required my public admin students to write a research proposal. Take a topic that's important in your field and write me a research proposal. In the course that we're going to talk about today, we're going to require the students to pick a real problem somewhere in a real community in the world that deserves an intervention and write an intervention proposal. So again, a complex group project is a way of getting a structured learning environment up and running. It can be done in many different ways. I'm not going to spend time this morning on it. But again, at that conference, I ran into a guy named Christopher Dede who really influenced all of us. I mean, he was one of several where our first question during the end of every lecture, our first question is if we stay longer, will you keep talking? Because what they were doing was just brilliant. So if you're not familiar with his work, have a look. He, among other things, pointed out there are about seven different modalities for distance education. Most people just thought there was one. And when he taught the course, his students lived in each modality they was talking about. So if they were doing computer simulation, they learned about it in a simulation. If they were doing audio conferencing, they learned about it in an audio conference. So he combined theory, practice and real life experience. One caveat before I go any further, both for the instructor and for the student when this works, it is wonderful. It makes it way more an interesting experience on both sides, way more enriching experience, but it doesn't always work. Some subjects probably cannot be taught effectively this way because of a variety of reasons. And some students are just not ready. And that's where I ran into a problem when I was trying to teach one course and the students just rebelled. So I started looking into the distinction between teaching styles, learning styles and mismatches and what can go wrong with self-directed learning. I ran into a guy that probably had an even more serious problem than I did, Gerald Glass, and I followed some of his paths. He actually on the student course evaluations was reading things where his students just hated him. They not only didn't want him ever to be hired again, they wanted him fired, they wanted him dead. They hated the guy. It was visceral. So this led into thinking what's going wrong here. So looking at his work, looking inside of a doctoral thesis I found, a variety of people have talked about self-directed learning and what it takes to make this work. So Knowles, for example, talks about an explicit process set of skills, almost a learning contract, if the student is going to succeed. Others have talked about personality traits. So again, willingness and capacity to conduct one's own self-management. I'm willing to set the time aside and raise your hand when you need help taking various traits and recognizing their weaknesses that have to be worked on on a student level. These are all important. When I first started teaching distance education for that nursing course, I was under the naive impression that now that we're not in a classroom all together, self-conscious, weak students would raise their hand when they needed help because it was a private conversation with me. I was wrong. What I learned through experience is that the A students, the ones who really were doing well, were the frequent callers. Am I on the right track? I'm interested in this. Can we talk a little more about it? Can you give me some more examples from your experience about how I apply this sort of theoretical understanding? So the students were not calling. I learned you have to build in things in the course that force them to identify themselves. Otherwise, they stay weak in the background. So understanding whether a student is ready to a certain degree, you can advertise how the course works. But then you take your chances on whether they're going to enroll anyway. Gerald Rowe, as I was talking about, when he ran through all of what he could find, he actually said there are four stages in terms of students and teachers. You might start with a totally dependent student. I mean, if you don't hold me by the hand, I'm not going to do anything. I've run into some students like this. I ran into one in a lab where they would not touch the computer. Unless you sat at their side, they refused to try any of the exercises. Unless you showed them the keystrokes, they weren't going to hit the keys. That's not the point in the course. You know, when you get out to the real world, that's not the way you're going to get a job and keep a job. So to a certain degree, I'm not willing to do that. At the next stage, you get people who are interested but not quite ready to leave the lecture format and become self-directed. And then eventually you get up to students that are entirely self-sufficient. On a teaching level, there are some micromanaging authority-driven teachers. You know, I will tell you exactly when to jump and have a high to jump. I will tell you what notes to take. I will tell you what's on the exam. On the other hand, when you get up to the fourth stage, I'm really here to be a consultant. You know, I created a structured learning environment. I put some guardrails up for you. And I'm not here to lecture. I'm not here to dictate. I'm here to work with you. And then there's some examples on the right-hand side of what kind of techniques would work depending on whether everybody is aligned. If they're aligned on the diagonal here in this graphic, it's magic. I mean, if you're an authoritarian instructor and a dependent learner, everybody's happy because you've got lecture, drill, immediate correction, and everybody's happy with it. On the other hand, if it's not the way you want to teach or if that's not the way the student wants to learn, there's a problem. At the other extreme, if you've got self-directed learners who really want to run with the material, you know, they don't want to sit in a classroom and watch you read the textbook. They want to rely on you when they need you. If you're there as a consultant and a delegator, they're there as self-directed. Again, it's magic. So as you can see, there's a good alignment if you're on the diagonal. The problem is if you're off the diagonal. And this is the situation that Gerald Rowe was in. This is the situation I found I was in a couple of points. So the first is what happens if you have a student who is really ready to be totally self-directed, but you've got an instructor that sits on being authoritarian. I will lecture. I will tell you what's on the exam. This is the way we're going to work together. According to long, some people are willing, some students are willing to sit in that and they'll still succeed. But others resent it. They resent the authoritarian nature. They rebel against a barrage of low-level demands. You know, I don't need this quiz. You know, I don't need this drill. I'm already five steps ahead of you. What can I do for more advanced material? So again, these learners might rebel or just retreat into boredom. And as Fox pointed out, as a teacher, you're probably not going to interpret this as a result of the teacher being the problem, the mismatch being the problem. More likely you're going to see the student as certainly uncooperative, unprepared to get down to the hard craft of learning basic facts. You're probably going to be in a conflict. He describes this as havoc. And this is where extreme control conflicts can engage in behaviors that are just chaos. At the other, you might have totally dependent learners that are just not ready for what they got themselves into. And a teacher who really wants to be there as a coach, as a consultant, not as a dictator, a lecturer. It's not sage on the stage. And unless you've done the reading, unless you've done the homework, you're not going to work well in a Socratic style of you ask a question. I ask you a question and I lead you to an answer instead of just, well, here's your answer to your question. So, various authors in the first paragraph there point out that many students are just not able to use the freedom to learn. I talked before about certain skills and personality traits that androgogy identifies to make this work. Well, if they don't have those, if they're not ready, then it's not going to work. If you're forcing the students for something they're not ready for, frustration, anger, resentment will emerge. And if they want closer vision, if they want immediate feedback, if they want frequent directive interaction, if they constantly have to be motivated, they're in the wrong class, but they're not going to see that. And in some cases they're trapped in the wrong class because they didn't drop out in time. And they may grow to hate the teacher as Gerald Gro pointed out, his students literally said they hated him. Other students like the ones Nadler described and some I've encountered in the lab may just withdraw, you know, I will recite, but I'm not going to come up to the level that you're wanting for me. They'll shy away from the independence that Americans value. So Hersey describes this as havoc. And the problem is you can't fix it. If you give in too late, they just resent you for holding out so long. And if you give in too early, then you've really defeated your own course plan. And again, they resent you for being such a bad teacher. So that brings us to the challenges that I had to address when I designed SPH 410. The course has the longest title that I've ever seen, but it accurately describes that it's improving public health and interprofessional approach to designing and implementing effective interventions. It's a large diverse class of about 70 students. We kept the enrollment to the size of the room. And when I went virtual, we kept it at 100. Historically, the courses attracted three different kinds of students. There were those who really did anticipate a public health career. And as I pointed out to my students in the first session, as I pointed out wherever opportunity arose, I've designed a course to be as realistic a look at a public health department experience as possible. So nothing in the course is arbitrary. The way it works, the way the teams work, the way the assignments work, the time frames, the required products. Basically, this is what you'd encounter if you worked in public health. There are also people who are looking at a different career, law, medicine, politics, but they know they're going to be working alongside public health, so they want a better understanding of how public health works. And originally when I designed this, UBC had a voluntary interprofessional education passport program. And so I designed this and submitted it to the IPE program. It became accredited as one of the courses students could take. The idea is if they took enough of the IPE accredited courses, they graduated with both the university degree and with a certificate indicating that they have mastered various skills in interprofessional relationships. So some of my students have not been interested in public health at all, but they were interested in the team development skills that are inherent in this course. Another challenge is there is a huge body of knowledge to cover. I mean, as I tell the students, it's going to be like working in the fog, navigating in the fog, and drinking out of a fire hose for the first half of it. But have faith, it will become clear. And if you follow the leads, you won't wash away trying to drain out the fire hose. Obviously, like any course, especially in SPPH, where we're not, I won't get into that, any course has to be cost effective in the way it's delivered. So designing this in a way that minimizes the costs. And since we're paid per student for undergraduate students, maximizes the income to the school was one of my goals. So I've been able to handle a large complex class alone. I haven't ever used a TA. There are a couple of handouts available if you want them send me an email, David period burn bomb at UBC period CA preferably on UBC address and ask for them, I will send them back to you as an attachment. So the early experience in this course was reported at a national meeting and a subsequent international publication that's the article on the left. You don't really need that to understand what we're talking about today. The key take home points from that. The way this course is structured on day one student students self select on a broad area of interest basis into groups of no less than five no more than 10 less than five wouldn't have enough complexity. More than 10 is too complex a group to keep going, unless you really know how to manage groups. The course itself meets only monthly when we have something to discuss. As I point out, I'll do most of the talking in the first session, but after that, the students will do most of the talking. So it's peer to peer learning as well as teacher to peer to student learning. The teams meet weekly though. There's a course outline that tells them week by week what they need to be reading what they need to be working on what they will need to be producing. And it asks them to rotate the leadership of the team meetings at the second meeting of our course. The teams are required to give an elevator pinch. You know, basically you've got five minutes to sell three ideas here. So in the time it would take me to ride an elevator. I need to understand what the problem is you've selected why it deserves an intervention why what's being done isn't good enough. And why the approach you're proposing has a better chance of success than what's been done in the past doesn't have to be fully researched. But of the three ideas, you'll then get feedback, both from the class and myself. And from that you pick one project proposal to move forward with. So the teams then choose a specific topic to develop a written and oral proposal, except for a couple of financial pages essentially what you proposed and submit to the Gates Foundation as a intervention on a local national regional community international, whatever scale, the grading in the course. I grade the oral and written work, but team participation is actually graded by the students in each team. I've got a formal rubric for that which I adapted from the medical school and the business school. It's got three broad categories in it and about 15 characteristics under each. So the teams have to do a peer self evaluation and award themselves points. So somebody isn't carrying their due weight on a team. This is a way for a team to drop somebody a full letter grade in this course. The course itself has six objectives that are very specific to public health so they're not really pertinent to our discussion today, but more pertinent to today there are four additional objectives that are generic. The ability to communicate ideas and opinions with clarity and respect to demonstrate collaborative teamwork and leadership skills to demonstrate effective and respectful problem skills, solving skills, because there will be conflict. I guarantee it in this course I guarantee it in real world, and you have to be able to identify and use information resources from other disciplines. So in terms of today's talk and the whole idea of developing a, I've never heard the term authentic learning, a realistic and authentic learning experience, a practical experience, a successful experience, those last four objectives are really critical. And they're not just arbitrary. I mean they might sound like a good idea. National Board of Public Health examiners another group I work with has done any job task analysis to find out what's really important if you're going to work and succeed in a public health role in any of the domains where public health work is done. When you look at what's on this grid. It's all the soft skills of those last four. We know that you're going to need epidemiology some biostatistics etc etc, but these other things are more important because if you can't do them, then you're useless to us. So, that's why all 10 of these are important for anybody working in public health but the last four are really the critical ones for everybody, and not just in a public health setting. So, how does it work challenges. I talked about maximizing enrollment minimizing cost. Well, if I had to do 70 major essays alone if I had to deal with 70 students alone that would obviously be impossible. Students I think really value what senior level people bring to a course as opposed to what a TA is going to tell them. So, by having the students working teams, not an individuals, I grade between seven and 10 major papers both in draft form and in final form. I listened to seven major oral presentations. It's manageable. Working as a team, it promotes what's going to happen in the real world because you don't develop projects like this as individuals, you don't work in public health as an individual. So, this is a realistic thing, not just a game. And unless you're doing it as a team, you're not learning how to sell your ideas. So, with journalism in this course, and one of the things we've noticed is a difference between the journalism students and the public health students, the science students, science students think in terms of data. Public health students think in terms of data epidemiology has been described as data with tears removed. Journalism students on the other hand, recognize what public health only recently in the past few years came to understand stories, Trump data, stories of power. So, to get these teams to understand how to sell what they've got a scientific basis for proposing is equally important to having a valid proposal. So, letting the students pick what they want to deal with is what makes it engaging as an authentic learning experience. I basically in the first lecture, spell out the broad areas under which public health works, maternal and child health, infectious diseases, chronic diseases, etc. I tell the students, okay, gathering groups, if you have an area, a broad area of interest in that particular aspect. And then within that group, start introducing yourself to each other, narrow it down and pick a team of five to 10. And then with the elevator talks, pick and refine and ultimately select what you're going to work on. So this makes it engaging as well as authentic. It's not an artificial exercise I put out there. It's meaningful to the students. I really want them to deal with, you may have heard this term before, wicked problems. I mean, some things are just easy. Other things are difficult. Some things are devilishly difficult. They are wicked and public health is full of that. So to be able to get novices to tackle wicked problems on a safe path, the course itself provides comprehensive, sequential, clear, concise materials. Now, I often have to point out what's in those materials when students don't read the materials. But again, that's my style of teaching. Now, I'm not going to give you the answer to your question. I'm going to point you back to where it is in course materials. If you have a question about what that means, sure, I'll be glad to discuss that. But if you're just asking what are the grading criteria, well, that's on page X. There are also guardrails in the course to prevent novices from tackling the impossible. I mean, sometimes there's a topic that is really just trendy, you know, like Zika or now SARS and COVID. I discourage my students from picking a topic that is moving so quickly that they don't have a hope at all of keeping up with it. Some problems are just never going to be solved unless the politics changes, the war ends, whatever. So I discourage them from those topics. But again, it's not a, except this reject that it's a discussion of strengths and weaknesses in their elevator talks. Sometimes they don't separate their topics in the elevator talks well enough. I always tell them pick three distinctly different things. One year I had a group that had one topic and three approaches. My first comment to them is you didn't follow my instructions. If this had been a great exercise, I would give you an F on it. And unfortunately, since the topic you chose cannot succeed, the city of Vancouver already imposed a solution different from what you're proposing. They would never accept yours. You're going to have to start over. Other people have done a better job of staying within the guardrails. And again, I indicated building in progressive check stages where people have to do something to communicate with you, and that identifies which ones are struggling. So the elevator talk, the rough drafts, some of the other discussions that we have, which have some Trojan horse questions embedded in the class discussion. You know, that gives me a clue to people who should have been asking for help, but we're not asking for help. Promoting interprofessional collaboration skills. So I designed this as an IP learning experience, both in the content and the grading. So part of what I do is say, all right, there are different models of how a team can work. You know, you may or may not have time to develop this kind. But, you know, think about what kind of team you want, you're going to have to have some certain norms and procedures to make your team your team meetings work. I want you to rotate the leadership of your team. We've got the self evaluations within the team. It's difficult to have those awkward conversations and I realize not all the students do it. Some just, all right, we'll give everybody full marks, but others take it to heart. But giving them the exposure, if not the actual experience starts them down that path. I point out with this course, everybody, no matter what level they start with, will have something in there that advances their understanding. Promoting team collaboration and skills development, as I say, I described the options, you know, there are different kinds of team. In some cases, everybody wears the same uniform, but they're playing independently, like track and field. In other cases, it's fluid, you can't really have central control. Some kinds of teams, you have central control before the play. So think about how you want to work. I also talk about some tools. So learning, what kind of learner are you? Is it visual, auditory, is it tactile? Are you frustrated with the way your team is working? Is there an understanding of different ways people can interfere with team functioning and how you should deal with that? So there are some resources in there the students may or may not need. I basically point out, once your teams are formed, that's it. You're not changing teams. You will carry your dead and wounded across the finish line. You cannot drop somebody off a team. If it really comes to armed conflict, I'm willing to intervene in a session where we will all meet together and I will be the mediator, and we will come to agreement. I point out that everybody has to sign off on those self-evaluations. I've had situations where one student says, well, I'm not accepting this. Well, when I'm the mediator, I point out, okay, we either come to agreement or I will accept the form without your signature and I will give you a zero on that entire 25% of your grade. So walk the plank or play. Most of the time, though, I have not had to be the heavy hand here. Most of the time, this thing works like a charm. Meaningful evaluation criteria for the course grade. The team product, the proposal. They really come out professional quality. I've had some of my colleagues sit in just as a guest at the last session. And a number of them point to, you know, this is mostly undergraduates in this course and they are producing what we're trying to get the master's degree students to produce as an end result. So the team products really have high caliber. And the products which are basically graded this, what same way I mark manuscripts submitted and when I'm a journal editor, the same way I mark grant applications when I'm hired as an evaluator of grants. It's not a unique answer is correct. There is no unique answer to these wicked problems, but rather is your logic convincing are your arguments convincing and well structured is your evidence. Well appraised and well used, where are the strengths and weaknesses. What have you recognized as weaknesses versus things you didn't include in there. The worst situation would be the evaluator says well you know this other reference that you didn't say just blows all your arguments out of the water. So the end result after the feedback on the ungraded elevator talk, the graded first draft and the other presentations in class really leads them to succeed. Most of the students get an A in this course. So that brings me to my last point. It's got to be safe enough to stumble in the early stages because they're going to and you learn by your failures. It's not a hard enough problem if you can just do it in your sleep, but they have to be able to learn by doing and still excel by the end to get a good grade. They need a grade to get into medical school graduate school. So I don't want to create a course where high performers are going to do poorly. On the other hand, I really want to be challenging. So the first assignment the elevator talk as I mentioned is critique but it's not graded. So now you can see the kind of critique kind of bring to your work. And you can see where you've not done a thorough enough job in your search for evidence critical appraisal of evidence incorporation of evidence, layout of slides, whatever. Later components are safely scale and progressive constructed criticism leads towards success. The early complaints I had about my courses too many students are getting an A and why is that so that's when I started inviting faculty alright see for yourself what they're producing. I agreed on the basis of achievement, not on a curve. Here are some of the problems that my students have chosen. I'm not going to talk about all of them. But as you can see some of them are local to Vancouver, some of them are regional within the province or the Western provinces, some of them are Canada wide. Some of them are a specific population of like minded people. Some of them are international. Some of these get to be really interesting. The last second to the last one combating hazardous nicotine electronic delivery systems is by Canadian adolescents. So I had two teams in the same term pick vaping as their topic. So two teams were picking exactly the same topic, but they came up with equally defensible different solutions to the problem, which made for really interesting discussion. Some of them are just amazing. The last one, for example, improving access to contraception for women's release from prison. Sometimes my students discover real problems in real communities that none of us can believe could possibly be because it's just so stupid. So one of the things that one of my teams discovered is women in prison don't fill out a number of forms like income tax. So when they're released from prison for six months or more, they're not eligible for health care coverage, which leads to all sorts of problems, including on presses. And the whole thing really is ridiculous because the prison system has a fiduciary responsibility to reform and manage the welfare of people in their charge. Wouldn't it just be a simple set of forms six months before release to solve this problem? In that case, the students said, you know, like what we basically would propose is our action because, you know, they really learned politics is important in these proposals. But what we take as a letter to a federal ministry proposing, why don't you do this? You have a responsibility you're not fulfilling. Other proposals are way more complex in terms of the politics. But as I tell my students when I first worked in hospitals and epidemiologists, my job was 80% epidemiology, 20% politics. When I worked in government running a state health department program, my job turned into 80% politics, 20% epidemiology. So this course really encourages students to understand culture, politics, feasibility, economics, local priorities, rather than just say, you know, we from the west have got a solution to your problem. These are wicked problems. I've taken this from the work of Bob Evans, who is a very bright economist in our school here. Some fields have difficult problems. I mean, some medical diagnostic problems, some patients doctors work with are really challenging, but it's nothing like public health. When you really look at this conceptual model, first off, you have to understand the values and assumptions in a culture. You have to understand what the research and evaluation of prior projects has informed us about what works and what doesn't work. You have to understand the whole process here in his diagram is evidence based decision making. There's another guy that I really like who hates the word evidence based. I went to one of his presentations and he said, yeah, what if I told you there's a drug that's over 90% effective, it's inexpensive, and it has very, very few mild side effects. As a government policy, shouldn't you be buying this drug and supplying it? And after a bit of the body, you know, most people say, yeah, I guess so. And then he comes up with the punchline. Yeah. Well, the drug is Niagara and I don't think erectile dysfunction is the biggest public health problem you're dealing with. So rather than call this evidence based decision making, it should be evidence informed decision making. And unless you the policymakers are informing the research community of what questions you need to be informed on what decisions you're facing and what gaps they need filled, they're going to just bombard you with evidence and expect you to act on it. So being able to pick and choose among the evidence, build it into logical arguments. Understanding the values and the experiential learning. Yeah, that's part of what my students have to bring to critical appraisal. On the front phase of this, we have a number of determinants of health. These are the classic determinants of health we teach in schools of public health. If you want to move upstream and start solving root causes, you might deal with income, housing, education, not just deal with disease at the impoverished backhand. On the other hand, we have schools of population and public health. So when you talk about public population health, you're usually talking about strengthening systems within the public. So that's the right hand face of this tube. And then on the top, again, figuring out the politics of it. Are you dealing with individuals, families, communities, sectors, or society? Who is your target audience and do you reach them directly or indirectly? Some of my student projects, I've pushed them to realize, yeah, you're trying to really influence the government, but you're going to do that by creating a mob on the front lawn with flaming torches demanding action. Because the political will is not there at the moment. In some cases, you want to reach the individual that the individual you want to reach is not going to listen. Talk to the guy's wife if you want to get the guy to do something. So understanding where to aim your pitch and how to aim it is part of the sales package that the students have to understand to create a whole approach to these wicked problems. I mentioned that interprofessional is part of this. When I first put this course together, the IPE program was a voluntary program in an office. You submitted your course with all of its activities and interactive approaches, faculty interactions, student project interactions, local sort of learning situations, whatever. And then on a rubric, that office would decide whether or not you've got enough in there to accredited. That office has disappeared. The program, as I understand it, has evolved into a mandatory curriculum integrated into most of the health programs. But frankly, I'm not sure how well it's been done because I don't see an awful lot of it on the working end. But there are, there is a logo here which shows you the concept. There are resources on this slide. And I believe CTLT has distributed this slide set to anybody who's participating. And again, you know, if you're viewing the recording later, you can pause on this slide. But you could go to the two links for UBC collaborative materials as a starting point. If you're interested in how do I get started? How do I build IPE content into my course design? They have the challenge as teachers to create a safe but challenging learning environment. That's the approach I've taken. I hope this has been useful to you. I would be happy to answer any questions if there are any at this point. But that is the end of the material I've prepared for the hour session.