 Good afternoon and welcome back to this session on peer instruction. So, this morning we have looked in detail about what is active learning and what are its features, what are its elements and we also looked a little bit into how to implement active learning in your class. And in the second session in the morning you actually learnt about one specific technique called think pair share which is one of the active learning techniques. So, what we will do now is look at one more such strategy and it is something which you may be familiar with we in fact, did a little bit of it in one of the morning sessions also and this is also an active learning strategy and its main intention is to promote students conceptual understanding. So, let us actually dive into one example right away. So, what I have here is actually one peer instruction question. So, what I would like all of you to do is hopefully by now you got your tablets with you. We will launch the question on the student response and the clicker student response system. So, we would like you to use the tablets that you have the aakash tablets that you have to actually answer this question you can do so individually there is no talking at this point you will soon get to talk to each other. So, individually please answer this question you have a minute to answer this. So, individually please vote on which you think is the most likely outcome. Okay, what you see in front of you are your responses. So, it looks like about 650 of you responded and it shows the distribution of your answers and the other thing it says is that 85 percent of you were wrong. So, what I would like you to do at this point is discuss with your neighbor firstly in your class find somebody who has a different answer than yours. Discuss with your neighbor you already are being told by the clicker system which is the right answer. So, discuss with your neighbor and try to figure out why the answer that you see in front of you is the right answer. So, I am still not telling you exactly what the right answer is and why, but discuss and we will do one more round of vote let us see. So, this is the time where you are doing a peer discussion you are talking with your neighbor trying to figure out why your answer was wrong or your Debra's answer was wrong. Okay, if you want what you can do is use the chat window to tell us why you think number 4 is wrong. Number 4 was that each of 1, 2, 3 above is equally likely. Here is a question and you did one round of vote where 85 percent of the participants got it wrong and many, many said that number 4 is the right answer. However, number 4 turns out to be the wrong answer. So, try to find out why. It is okay if the quiz was not launched the first time. Okay, we all know that 3 is the right answer, why is it the right answer? That is the part that is the point of this discussion now. Okay, so let us see what you would do as a teacher here. So, this is a question on counting and probability. When you are asking a question like this it is very likely that a large number of students will pick number 4. Because it looks like well it is a probability question each whether it is a head or a tail each of it is equally likely. And many people are likely to think that number 4 is the right answer and you do one round of voting. Then what you do is you can ask your students to talk to each other and there will be some discussion that happens and they will try to figure out what are the different possible outcomes. So, here is a hint if you simply write down the different possible outcomes you have two coins each coin can take a head value or a tail value. So, there are 2 times 2, 4 possible outcomes just list the outcomes and you will know what the right answer is. So, this question was in fact an example of this technique called Pierre Insta. So, what I would like you to do at this point is think a little bit more examine a little bit more closely what exactly did you do you were the students here. So, what did you do not just in the first round of voting, but if you took this entire process where you voted then you saw the answers sometimes the teacher might not show you the answers and might ask you to discuss with your neighbor. And then you thought about the right answer once again what was it that students were doing in this entire activity. So, in a pair turn to your neighbor and write two specific actions that the students that is you did do not simply say you solved the problem or you voted, but what were the actions you were taking do this is a pair and then we will go to the next part. For those of you who are still voting before you vote please consider the four different cases you have two coins and each coin can either take a head or a tail. So, you have four possible occurrences write them down miss them down and then you will see what the right answer is. So, the teacher is in fact making the students do all these tasks in order to answer this question. Okay, some of you are saying that the students were writing the possible outcomes they discussed with each other. Did any of you change your answer after talking to your neighbor? How many of you got convinced of a different answer than what you initially voted? Once you have it with your pair what you can do is actually it looks like people are already doing the share phase. So, I will just leave this slide for another minute or so and then we can go on. Okay, so what it looks like now is that I am getting a lot of answers which say that we calculated the probability we discussed with our neighbor why four is the wrong answer and so on. Initially the only answers were four. So, what is happening with the class which is you at this point is that you think you have some answer, but the moment you start talking to your fellow participants what you see is that you are talking you are listening you are arguing about what is the right answer, you are reasoning you are trying to solve the problem maybe some of you are doing some calculations. So, this entire process where a teacher poses a multiple choice question students answer it then students discuss sometimes they vote again that determine the reasons that entire process is called peer instruction. So, I think you can stop sending your answers over chat right now because we will now look into this technique a little more closely. This technique is called peer instruction. So, in peer instruction what students do is that they are talking, they are arguing, they are listening, they are writing, drawing which means they are actively engaged remember we talked about this earlier this morning they are learning from each other and they are teaching each other and from your responses it looked like a large number of you were in fact thinking reasoning and answering it is not just three people in a class of 100 it is a large number majority of the class is engaged. Since it is a multiple choice question people can people are in fact encouraged to take a chance even if they are not sure of the answer and if some of you had wrong answers which a lot of you had the idea is to get your answer out and try to confront it and resolve what was wrong. From the teacher's perspective also why it is extremely useful is that it is immediate feedback to the instructor. So, if you look at what happened just now 85 percent of you said that 4 was the right answer if I were actually teaching you probability I would know immediately that a large fraction of the class is not yet clear about the concepts. So, it is immediate in time real time feedback and the teacher can do something at that point to address students difficulty. Students see that everybody else is also having a hard time. So, they are trying to build an argumentative yet a friendly scientific atmosphere and as a result of all of it the communication in the class the argumentation skills and the convincing skills also improves. So, this method is really beneficial both for students and for teachers and more than anything it is a very short activity takes maybe 5 minutes to do and it has been shown to have a lot of benefits in terms of conceptual understanding and getting feedback. Okay, so let us move on here I just want you to recall the different requirements and elements of active learning. Instructor creates carefully designed activities. So, this question was thought about and written before coming to class. It requires students to talk, write, reflect and express if they are thinking you already did that most of the students go beyond listening and copying of laws. Similarly, if you recall the elements of problems of active learning students were engaged in problem solving during the class then student ideas are brought out and their students are asked to figure things out for themselves. So, all these elements of active learning strategy that we discussed in the morning you can see it being operationalized through this peer instruction question. I just want to comment on this last point students receive rapid feedback on their work. So, if you give a question on an exam until the exam is graded and three weeks later the students see it they do not really get feedback whereas here they get instantaneous feedback and they can correct on their feedback. So, what is peer instruction? It is a classroom based active learning strategy based on well designed specific questions. Peer instruction is not just a multiple choice question. It is a question students voting discussion voting discussion teachers summary all of it together is peer instruction. So, the way peer instruction should be implemented we will see if we can do it for one more question little later. The teacher asks a question poses a multiple choice question and before that the teacher in fact creates this question. The teacher asked students to vote individually then they discuss with each other and usually there's a second round of voting which we skipped just now but usually here when before they do the peer instruction the discussion with each other they do not show the answer. After that the teacher does a summary and debriefing and then resumes with the lecture. So, how do you implement peer instruction? You all had clickers. So, that's one way to implement peer instruction where the students vote. What you have to remember however is that the clickers themselves are not peer instruction. What's peer instruction? The central part is the discussion. Students are talking to each other. So, let's look at this. This is clickers are one way to implement peer instruction. What happens if you don't have clickers in your class how can you do it? Does it mean you can't do peer instruction? The answer is no you can actually do peer instruction with a very simple technology using your fingers. What you do is ask your students to hold up the answer not here because you don't want others to see it but ask them to hold it in front of you like this. So, if their choice is one they do this if their choice is two they do this and when a student is doing it as a teacher you can see all the students but students can't see each other at that point. So, this is one way to implement peer instruction without clickers. There's another way to implement peer instruction without clickers and what you do there is you take a sheet of paper. Just an ordinary piece of paper and on that piece of paper you write I'll hold it in front of me you write one, two, three and four and then you fold it up. Each student carries a piece of paper like this. So, they can make their own or you can give them and when students vote they simply take their piece of paper and they hold up what they think is the right answer. If they think two or one is the right answer they just fold that paper and they hold it up again. So, using sheets of paper you can do it as A, B, C, D or one, two, three, four it does not matter. That's another simple way of implementing peer instruction in your class. So, you don't really need to have clickers but if you have clickers the advantage is that you'll be able to see, get a histogram of students answer right away. But since the discussion with each other is the core part it's completely okay to do peer instruction with these. Okay, so why peer instruction? Why should you be doing this? I think by now you already have some answers especially if you recall this slide here. The slide that we discussed earlier gives you a lot of reasons as to why peer instruction. What we discussed in the morning also gives you a lot of reasons. But there's another set of reasons and the next slide will be there on the slide for you to look at. I won't explain all of it but all I'll say is that there's a lot of research studies, educational research studies done on this technique of peer instruction. So what's actually done is that one class is taught with peer instruction and another class is taught without peer instruction. And then the researchers they look at the learning outcomes between these two classes. And it was found that in all the classes the learning outcomes and students engagement in fact improved with peer instruction. So another reason why we should all believe that this is a good strategy is that people have done careful scientific studies in a variety of environments. And here are some studies. I'll just leave this on for a few seconds and later it will be there on the slides which will be implemented in moodle. Okay, the quiz we'll launch after sometime. A lot of people are asking on chat to launch the quiz again. We will do this after sometime. We'll in fact look at ways to write peer instruction questions in a programming class. So please don't ask us to launch the quiz right now. Instead let's look at what are the elements of peer instruction and how to implement peer instruction. And then we'll launch another quiz. Okay, so now as a teacher you have two main roles. One role is to actually implement peer instruction in your classes. Your other role is to write effective peer instruction questions before you go to the class. So in a way writing the peer instruction questions is the first task you will do. But we talked about what is peer instruction and how to implement it first because so that you get an idea of what actually happens in the class. Okay, so before you sit down to write a peer instruction question you need to know what is a good peer instruction question. Now how do I write it so that it's actually effective? And we have many guidelines for this. So here is a list. We'll keep coming back to the list. But let's spend some time on some of these bullets in this list of what is and what is not an effective peer instruction question. So the example you saw was that a peer instruction question looks like a multiple choice question. It has a question which is called the STEM and it has about four or five choices. But it's really important that the peer instruction question is conceptual and it does not involve lengthy numerical calculations. So if you take a regular long numerical problem and give it as a peer instruction question it doesn't achieve its purpose. Instead you have to think about the conceptual the concepts underlying some problem and pose that as a question. Peer instruction is based on students' existing thinking. So you know often after five or 10 years of teaching practice you know what are the different what are the difficulties students have and what points they have difficulty in solving problems. So what you should do is frame a question based on such ideas. The distractors in the PI question the four choices should be believable. You should not make only the right choice to be believable and all the other choices to be complete nonsense. Then again the point of the question is lost. I think that's something all of you do anyway when you write multiple choice questions. A good peer instruction question these are some examples which we'll start looking at in the next few slides. You can ask students to predict results of an experiment if you're doing some other course or of a program in a programming course. You can ask students to apply ideas in a new context. So for example in the probability question all students and all of you knew about probability and outcomes and all but suddenly there was a new real life context. Two coins are tossed and you were asked to make a prediction of what happens. So such questions are good they're useful for PI questions. So peer instruction questions are these are obvious points but I'll let them stay there. The question should not be ambiguous. It should not give away the right answer and it should not be very trivial. Okay so now I think with this let's actually jump into the different examples of peer instruction questions for a programming course. Okay so since all of you are asking for a quiz let's do this question as a quiz. So here is a small part of a code. It's a for loop for calculating the factorial of a number. The question says how many times is the set of code executed based on the values given in the program. So let's launch the quiz. You can actually vote using your tablets and clickers since we have enabled it just now. Okay so one minute is up and we have a large number of correct responses and we have a large number of wrong responses as well. So out of 178 responses we find that nearly 55% are correct and 44% have got the wrong response. Maybe I'll just wait a few more seconds for the responses to stabilize. Okay so even after 1600 odd responses are in we find that about 50% are correct and the other 50% 44% have got various of the wrong responses. Okay so there are responses like one time n times n minus one times and n plus one times and one of the key things that we want to illustrate using this question is the fact that when we are doing a for loop the for loop runs as many times as the condition. So for i is equal to one as long as i is less than or equal to n. So there are a couple of things to notice here. One is if we were initializing an array then I would start from zero and second is it is not i less than n but i less than equal to n and the third thing to keep in mind is that the for loop semantics is that it will execute as long as the condition is true. So the moment you realize that so it's possible that students who have some misconception may look at this less than or equal to n and say that okay it will execute n minus one times or some of them may count it as they are used to seeing loops going from i is equal to zero so they may say that it will actually run for n plus one times. So which is what seems to have also happened among some of you. So this is one good reason why peer instruction questions are useful. So what you could do is now in this case we know that the answer so it has shown in your remote center it has shown the summary and it has shown the green bar as option B right which is n times so in this case the students come to know the answer straight away. However what you could do is like what has been explained you could not disclose the correct answer and have students discuss with each other convince each other that their answer is correct and then have them vote again. So that is one possibility of how you can implement peer instruction in CS 101. Okay so what we'll do in these examples we'll show you number of examples and you will be switching your role between that of a student and that of a teacher. In the sense when you're answering this question you're actually testing your domain knowledge and you're trying to see which is the right answer how should I calculate the answer to this program and so on. But ultimately what you would be doing is writing such questions for your own course. So if I say become teachers what you need to now start thinking about is what kind of questions should you be writing so that students actually are able to do the conceptual reasoning and so that their attention is focused on possible places where they can go wrong and so on. So just now what was explained to you is why this is a good peer instruction question because students will need to know when exactly does the for loop keep getting executed how to write the semantics and so on. So let's go look at one more question and we'll see. So the next question is about trying to determine what this piece of code will do. So here are the different answers and you can use your tablets again and answer what this code does. Again we have 1600 odd responses and this time surprisingly the number of correct answers is much smaller. So we have 27% of the people giving the correct answer and the others coming up with one of the other options. Okay so what this tells you basically is that even for people who know the subject clicker questions is a good mechanism for identifying where the misconceptions lie. So for example here when we do the discussion phase peer discussion phase of these questions then what we can find out is that the reason for why somebody has said that two is the correct answer or four is the correct answer. So those are things that we can determine when we do the peer discussion phase of the peer instruction question. So in this case the correct answer is three as you can see basically the first for loop is initializing the array and the second for loop is printing the array and then later on you can discuss with the students as to how this question helps them to determine what the code of a small piece of code does. So again now as teachers what you can see is giving a question like this where you give a small piece of code and ask students to determine the purpose you're not asking them to predict the output of the program but you're asking them what happens after this code executes what is it supposed to do. So this in fact helps students get to a very high level of thinking where they have to read the entire program they have to comprehend all the different parts of the program put it together. So they're doing a deep analysis of the program without actually doing any nitty-gritty calculations. Asking a question like this is in fact a very useful way to help students be able to trace code and understand the logic behind the code. Okay so now let's look at another type of question each of these type is an example. In the previous one you were asking students to count iterations in a for loop or a while loop here you're asking students what the code does in another type of peer instruction question you ask students to predict the output of a program. So you give them a small piece of code or maybe don't give them a very large program but a few lines of code and ask them to predict the output what actually is will the computer output at the end and here are the choices. So this time we will let you take a look at the slide first make sure that you choose your answer and then we will launch the poll on the clicker so that you don't need to look at the clicker for the question itself you can use the clicker only to send your answer. So now this time what we will do is we will not go to the summary responses we will not display the summary responses but we will go into the peer discussion phase. So you can please turn to your neighbor and check whether your neighbor had the same answer as yours. So right now what we can see is about 40% of the people have got the correct answer and 60% of the people have got the wrong answer. So what we want is that discuss with your neighbor what is the answer that you have got and why your answer is correct. We'll give you another one minute for the discussion to take place and then we will come back and talk about the actual answer. So what I'm going to do is I'm going to launch the same quiz again. So this time you can, this time for only 30 seconds because you can, you would have already discussed and now let us see how many answers are there in which category. So it's the same quiz now. So this is the post discussion phase. I've launched the same quiz again. Okay, so what we see now is the fact that initially we had 40% of the people getting the correct answer and 60% of the people getting the wrong answer whereas now the numbers are almost flipping around to saying that 60% of the people are getting the correct answer and 40% for some reason are still getting the wrong answer. Now what you can do as a teacher is you can have students do a hand execution of this program. You can have students say that, okay, create a box which has A, create a box which has B and you can say that let's create a place for a pointer. Let's create an arrow for P pointing to and A. Let's create, let's see what happens when we say star P is equal to 13. So when we say P is equal to Q what happens is initially P is pointing to A and then when you say P is equal to Q now P is pointing to B and when you say star P is equal to 13 it's the contents of what P is pointing to so which is actually B which is getting modified and so that's how you can have the students hand executed have them draw the memory locations and thereby you can get them to see that the correct answer is two. You can always ask a question like this in any topic in programming. Give a part of a code and ask students to predict what will be the output of a code. So a question that follows this format is something that you can use regardless of the topic and it's again one type of question where students actually can go through the program understand its logic and this time they're also being made to execute it. Okay, next question. So once again we'll do the same thing. We will display the question through the A view window first for you all to take a look at and then we will launch the quiz through the clicker, okay. So the question here is there is a function definition which takes one parameter value X as the parameter that is passed to the function FUN and inside the function all that is done is it sets the value X to five and then in the main program there is a variable which is set to three and the function is called with that variable and then you're checking what is the output of the variable. So now the question is there are two types of function calls in the first case the function is being called as FUN int X which is being passed and in the other case the function is being called as FUN int and X which is being passed, okay. So what I will do is I will now launch this quiz for the same two minutes, okay. So we are starting to get the responses and a large number of people have said that C is the correct response. So this question is a good example for discussing with students the validity of option three. So why is it that option three is the answer and why is it not option four is the answer. So we see that many people have got option three as the correct one and some people have also got option four which is not a small number it's very few have got A and B as the answers. So now we look at the options and we look at the program and we can use this as an example to reinforce the concept that when we are passing a parameter by value versus when we are passing a parameter by reference what happens. So you may have taught the concept of parameter passing in the previous class and this is an excellent question for you to start the next class with which is actually a revision of what they did in the previous class of ensuring that they have understood the concept of parameter passing by value versus parameter passing by reference. Okay so we will do one more question which I think is the last question for the ticker pole. So this question is as you can see on the slide there is a value which is being assigned and there is a switch statement so you can ignore the capital I for the int and the capital S for the switch assume that there is no syntax error in the program. Now the only thing that has happened in the program is that the break statement in case five has been missed. So I am going to launch the quiz and give you a minute to do this quiz. Okay so once again what you can do is before you see whether your answer is correct or wrong you can just turn to your neighbor and tell your answer to your neighbor and see whether both of you have the same answer. If you don't have the same answer see if you can convince your neighbor that your answer is the correct one and why it is the correct one. If both of you have the same answer then turn to the other neighbor and see whether that neighbor has a different answer. Okay so once again when we see the responses this is what is being displayed we find that about 67% have got the correct answer 67% of nearly 2000 people have got the correct answer while 32% have got the wrong answer in the sense that most of them have opted for D as the other answer saying that it will print five four default while some have in fact also said that it will give a compiler error or it will print only five. So now this is a good time for you to go back and talk to the students and recap what is the effect of missing the break statement. Basically what happens when you miss the break statement is that the execution just falls through into the next case. So since the break statement at the end of case five has been missed the execution will continue to case four and it will also output four after which there is a break statement. So now the switch will break it doesn't matter whether the break statement is encountered in the case that you want or in some other case. So after that the switch will break out of the switch and so that's why only five and four will be output. So this is one way in which you can incorporate clicker questions or peer instruction questions into your classes and these are usually they are very useful for beginning the class discussion with where you talk about either some recap of what was done in the last class or you can use it as an initialization for some topic that you want to do in the current class. The other reason why it's important for you to think about these questions is let's say you do a question and 70% of your students get the wrong answer. So at that point you can actually do a little bit something different than what you had planned. You can explain the concept again or you can try to find out where the students are having difficulty and help them through the difficulty because see finally it doesn't matter as much as so much what you actually tell them but what really matters is how much they're able to do. So if we find and if we get evidence right away that they're not able to do what we want them to do then that's the best time for us to take corrective action. So having said this and having shown so many examples I think what you can do next is actually a little bit of experience writing such questions right away. So here is an activity. I think what you can do is pair up with somebody in your center and together choose a topic in the intro to programming course. You saw a variety of topics and a variety of questions and pick any one topic and pick any one of these question formats. I'll just run through those again so that you remember what those are and write there's a typo here. Write just one peer instruction question in that topic for any of these formats. Right now it's time to think more in terms of a teacher and see how to write these peer instruction questions. So choose one topic first choose a partner then choose a topic and write one peer instruction question in that topic. When you write the question you need to include the main question such as this part and you need to include choices. So you can include anywhere between three and five choices. When you write the question make sure that all the criteria for effective peer instruction questions are in fact satisfied. So we encountered these in a previous slide and I've put it up again here right now. So this is one example one instance of a PI question that you'll write what we've done is posted a moodle assignment where you will write one PI question for each of the formats that you just saw. So we saw five different examples. In some students were asked to predict the output of a program and others they were asked to debug the program and so on. So in the homework assignment you'll pick a topic and for each of those formats you'll write a particular question. What you can do now is pick any one of these I'll just go through all those five so that you get an idea and write the questions. You can spend about 10 minutes or so writing the question and after that we'll go to the concluding part. So find a partner and start writing your questions. Let me simply go through these slides once again to give you an idea of what they look like. Counting iterations and for and while loops or other loops. What does the code do? Show a code and ask what it's supposed to do. Predict the output of a program. What will happen if something is changed in a program to something else? Finally some case where something is forgotten or something was written wrongly. So it's like debugging a program. So pick any of these and write one PI question. Spend about 10 minutes on this activity. Right in the morning we mentioned several times that you will be able to benefit from this workshop when you actually do the activities. Similar to when you do active learning strategies in your class your students actually should be doing those activities and participating in them. So right now is the time for you to be doing the particular activity shown on the slide where you're supposed to be writing a question and then you can actually use this question in your class when you go back. So please make sure you spend time writing this activity it's not a break yet. You'll get a break in about 20 minutes. After the evening session what you can do is go to Moodle you'll see an assignment where you have to write five such questions and that's the assignment that you can upload. So keep this question with you right now and this can be part of one of your five questions that you will upload in Moodle later. Okay so we talked about how to implement your instruction questions and also about how to write these questions before you come to class. So the next thing is when to implement these questions in your class. So let's say you have a one hour class. Should you do it in the beginning? Should you do it in the middle? Should you do it in the end? How many should you do and so on? Because as a teacher you need to have guidelines about this. So it turns out that here instruction questions can be implemented before you actually start the main part of your class. So you can do it in the beginning of the class or the beginning of a module. You can do it during in the central part of your instruction in a class or you can do it towards the end of the class. But if you look at this diagram it tells you that the type of questions and the purpose of questions you write whether when you do it in the beginning, middle or end is actually different. So if you're giving questions in the beginning of class the goal of the questions and the type of questions should be such that it helps to motivate students or it helps them to recall the knowledge from the previous classes. A lot of questions we saw today can be used at the beginning of the class after you do the instruction on that particular topic. Or it can be if you do the instruction in the first half an hour you can interrupt your one hour lecture and give a peer instruction question on getting students to predict the output in the middle of the class itself. When you're doing questions related at the end of a module or towards the end of a topic then you can try to write questions where multiple concepts in the topic are synthesized together. So the questions in the beginning can be easy and straightforward and the questions in the end can relate to the entire picture. So you can use these questions anytime you want you have to plan them however before you come to class and depending on where you decide to use them the type and the goal of the question is different. So finally what are some problems you may face and how do you address it and what are best practices for using clicker questions because what we expect is that at the end of this workshop you will already have written five peer instruction questions. So hopefully you will be using at least one or two in your classes when you go back to your teaching. So I'll just walk you through some challenges that people have reported and also some solutions to them. So these are common challenges where nobody in my class is speaking and a parallel challenge is that too many people in the class are speaking because when you have students to a discussion it can get a little bit noisy, not a lot but just a little bit. So what the recommended strategies are is that if the class is too quiet in the beginning it's okay people do need some time to get used to it and that's why we impressed very often this morning that this is not a one time thing you do. You won't get the benefits and your students won't get the benefits if you do this only once. Instead you do this more or less in every class but spend a small time spend a short amount of time five minutes in each class doing one PI question or if you think of doing the other strategy that you learned like a flip classroom you can do that maybe once a week or if you're going to do a flip classroom as all of you will likely be doing you can use these strategies there. You have to be a little patient initially and after let's say two or three instances students get used to it and they in fact enjoy doing this. If you as a teacher feel that the class is too noisy you're coming to the second problem that's completely okay. Usually this noise is good because students are engaging on the content and if you want to be sure of that you can just walk up and down the rows and make sure that students are engaged on the task. What will you do if students don't participate? You may find this in the beginning in fact and again this will improve but it's useful to keep explaining to students why you're doing this and why this is important and how these interesting questions will benefit them. Like we said in the morning explain often why you're doing this and also demonstrate it often. And finally what will happen? Suppose you give a peer instruction question and there's a lot of animated discussion what do you do then? You can occasionally use a cue like you can clap your hands you can use a bell you can do something like this do any cue you want and if you make it a little humorous people will get back again that's and if you repeat that students will know that that's the cue for them to come back to the lecture. So each of these PI questions can be about three to five minutes if you do two rounds of voting it'll take a little longer maybe seven minutes or so and it's recommended that you do I would say one to three questions per class so even if you do one or two per every class you've actually gotten students to do a lot of active learning in your classes. Best practices I'll leave these on the slide there are some practices on how to write questions what to do and what to avoid the main thing is even though there are multiple choice questions do not give questions where students can answer using memorization instead give questions where students have to reason and work through the program and maybe have to calculate something and so on. In the classroom itself the main part is the discussion between students it's not only the voting so don't skip this part because that's where the benefit lies. Focus on students reasoning even if you tell them this is the right answer spend a little time explaining why the others are not the right answer and for learning benefits it's recommended that you don't assign marks for these questions at most you can assign something like participation marks but don't assign marks for right answers or wrong answers in which case students will only start thinking in terms of the rightness or wrongness and they'll stop thinking about the reasons. So what we do here even in large classes because we've implemented this in classes as large as about 250 is when students are doing the discussion we walk around we help the groups come up with the right reasoning and that way you can give individual attention even in a large class. All this happens in the one minute or two minutes that students are in fact voting. So this slide we've already put up earlier today morning and if you want to use this in your classes there are plenty of resources available both for how to implement it which is on the first bullet and also some examples. These days even many CS instructors have posted questions and secondly by doing the Moodle assignment right after this you will be creating a bank of questions which can be used as community resources by the other instructors in this community. So as you go back to the lab log into Moodle do that assignment and we'll take it from there. Right now the next step is a T-brick.