 Good morning and welcome back to all of you on day 3 of the workshop on Pedagogy for Effective Use of ICT in Engineering Education. This is the last day of the first phase, the first three synchronous sessions. And as we had discussed in the question answer session yesterday, we will be beginning with aligning, discussing about how to align our assessment questions with the learning objectives that we have discussed in so much detail over the last two days. So, what we did in the last two days was not only look at various cognitive levels that a student can and has to work with, but we also saw how that can form a basis for our own course planning and also saw how it can form a basis for the technology tools that we choose and the learning activities that we assign to our students with the objectives. The third pillar so to say of this learning where we have objectives and strategies is that of assessment it is as we all understand and accept it is important and we cannot get away from it. So, we will begin with how do we align the assessment along with the learning objectives and the activities. So, as per our usual format, we will be doing this in a partially flipped mode. We will begin with a video of Rinal Patwardhan who is a faculty member in electrical engineering in D.J. Sarangvi College. She is also a research scholar in educational technology at IIT Bombay. So, she will explain how this alignment can be done from a theoretical perspective and then we will work through a number of examples including some polling questions in between on how this can happen. Today's session is about continuing the pedagogy for effective use of ICT in engineering education. Let us look at the next topic that how we are going to do the alignment of assessment with the learning objective. First, let us try to understand the importance of this concept. You might have already gone through this scenario, so let us revisit this scenario. Imagine that this course on digital electronics is being taught to second year electronics engineering students. Now, as this course serves as a fundamental platform to understand higher level subjects like digital system design or microprocessor, the instructor has set very clear goal or learning objectives in mind. So, therefore, the instructor thinks that the learning objective should take care of preparing students in developing the ability to design digital system using logic. So, one of the topic from this course is logic gets which we can consider as basic fundamental blocks which can be used for building up more complex systems, digital systems. So, before you go for designing such systems, it is essential that we understand these basic building blocks very clearly. So, based on that instructional strategies were planned, lectures were delivered and as a part of curriculum, let us say that the midterm test was conducted. This midterm test mainly focused on writing truth table for the gates and finding out output for the given surface. So, for example, what do you mean by truth table? What kind of behavior or functional description a typical logic gate will deliver? For example, if it is a AND gate, how it will deliver or if it is a OR gate, what will be the output of this gate? That is what we mean by truth table or suppose there is some circuit which is made up of these logic gates, then what will be the output of this circuit? So, let us assume that this kind of questions were put into the midterm test which was conducted. If you are not from the domain, the way you can interpret this question is that there is some system and the inputs are given and the students have to calculate the output. So, it is a fairly straightforward question, a calculational question and in terms of the cognitive levels that you saw yesterday, it will be at the understand or apply level somewhere there. So, if you are not familiar with logic gates, it is ok, but what you want to take away from this scenario is that the content was taught in questions of simple calculations of input and output of outputs given input were given on the test. So, let us continue further. Now, those who are instructors, you will clearly understand there seems to be some problem hold. So, what kind of problem we are trying to find out here, let us see. So, since the learning objective said by the instructor was students developing ability to design circuit. Obviously, students got training or they studied the content to develop this ability. This is actually very much in line with maybe course level learning objectives or unit level learning objectives. You have already learnt about what is the difference between course level learning objectives or unit level learning objectives. So, what has happened is students studied the course content. So, as to develop ability to design systems. However, when they appeared for the midterm test, they were not assessed for the same. So, here the objective or the outcome that was expected from students was that they should be able to develop or design a system instead of that they were assessed only for finding out some functional aspects of the system which is already designed or already given to them. So, what is the impression generally students will carry? The exams do not assess what we learn. Many times we come across this scenario as an instructor. So, many times we get to hear this comments from students. Whatever we studied did not appear in the exam or it could be the other way round also. Whatever was asked in this exam that we did not study. So, now it is quite possible here that though the preparation of the student or whatever was the expectation of the student that was comparatively at higher level, but the assessment test that was set that was comparatively at lower level. So, it is quite possible that students might have scored very good grades in this exam. But unfortunately here the scenario is good grades achieved by the students in such exams they need not indicate good learning. And this is very detrimental to the process of teaching learning. We want not just good grades, but we want those good grades to be indication of good learning also. So, now what is the solution? How can we avoid these problems? So, let us try to understand. Is there anything as an instructor that we can do to find out solution for this problem? Because this problem is very severe as it is affecting the success or effectiveness of the teaching learning process itself. So, in fact, you know the answer lies with us, we already know the answer. So, if we do a quick overview of last two days session, we will automatically come to the solution. So, let us try to go through various aspects that you have been studying for last two days. This is what we call as triangle of effective learning. As you could see the entire teaching learning process here is you could consider this in the form of tripod and which is resting on these three legs that is learning objectives, instructional activities and assessment. Now, I guess we all are quite familiar and comfortable with this phrase called as learning objective. It is a very scientific way of defining what are our expectations from students. How are we going to make the students learn a typical topic? And that is how we try to write this learning objectives which are student centric, specific and measurable statements in the form of students will be able to do whatever we are expecting them to do. The second aspect here presented is instructional activities. As an instructor you might have come across the situation that we do not teach in the same manner every day for every topic. So, we have these variations. So, maybe sometimes you would prefer making use of blackboard. Sometimes it could be by making use of visualizations with appropriate strategies or it could be some active learning strategies in terms of peer instruction or think peer share and many more. If it is a project you would like students to debate or give presentations related to that. So, appropriate instructional activities are also important part of this teaching learning process. And now here we come to this aspect the another important aspect of it that is what we called as an assessment. Now, before we formally get introduced to what do you mean by assessment just let us try to understand that the success or effectiveness of teaching learning process is completely dependent on these three important aspect learning that is learning objectives, instructional activities and assessment. So, let us try to understand more about this assessment part today. What do we mean by assessment? Now, we are absolutely not unfamiliar with this breeze. We try to assess students many times to give you some examples. You are going to start new topic in the class and you ask students do you know this? So, what are you trying to do at that moment? You are trying to evaluate or you are trying to assess students prior knowledge so that they can understand the new topic that you are about to introduce. You conduct test or you are assessing their experimentation abilities when students are performing certain experiments in the lab. If it is a project based examination you are trying to understand how students are developing solution for the problem that they have identified. So, in short we have been using making use of assessments every now and then. So, if at all we have to summarize we could simply say that it is just a process of collecting evidence of how much students have learnt. Let me just slightly rephrase this statement or let me say assessment should reveal how well students have learnt what we want them to learn. Let me emphasize on this last few words of this statement that is what we want them to learn. These words are very important because that is what we are going to assess. Now, here is the question how do we know what we want them to learn? In fact, we have already answered this question that this is nothing, but this is what we have defined by means of learning objectives. So, learning objective are clearly trying to tell us that this is what we want our students to learn. So, just a quick recap of what you have already studied for last two days. So, we know that these learning objectives could be at different levels starting right from recall level up to increasing order going up to create level. Now, when you look at this or what you have you must have experienced this in earlier sessions. So, let us answer this very simple question. Are all learning objectives at the same cognitive level to make it more simple? Is it expected from students that they take same amount of efforts every time they try to answer some question? I am sure the answer is no and everybody agrees to because within a unit, within a chapter, within a syllabus, within a course also we try to take students right from lower level to higher level. The efforts that students are putting we try to increase the incremental order. So, we know that these learning objectives certainly are not at same level. So, now, let us tie up all these things together. We know that students should be studying or they should be focusing on learning objectives or learning objectives are the one which are making us understand what students must be studying. So, and when we know that these learning objectives are not at the same level, here is the question that we need to answer. Then how do we assess students for the task at different cognitive levels? So, if you try to answer this question intuitively, the answer is going to be when learning objectives are at different cognitive levels. Our assessment questions should also must be of different levels. Now, please understand here we are talking about two different aspects. The first aspect that we are trying to highlight here is that all assessment questions should not be at same cognitive level because basically they are trying to assess the knowledge that is gained due to learning objectives which are at different levels. So, look at this scenario, you have learning objectives at different levels, you have assessment questions at different levels. So, I guess it is very important to understand the second aspect that there must be matching between the level of assessment question and learning objective. So, this is what we are trying to highlight in today's session. So, here are our learning objectives set for covering up a certain content. We are trying to come up with good quality assessment question. So, what is more important is alignment between assessment questions and learning objective. Okay, now that we saw in detail what is meant by the triangle of effective assessment. Let us actually dive in and look at examples at various cognitive levels of learning objectives in order to do this alignment. So, the learning objectives for you for this particular session are that by the end of this session you will be able to classify different assessment questions as per Bloom's taxonomy very similar to what you did with learning objectives. You will be able to generate your own questions for your own course at different levels of Bloom's taxonomy and all the while make sure that the assessment questions that you wrote are aligned to their respective learning objectives. So, in order to do this let us look at the next two videos. I will play them back to back and we will go along the ladder of Bloom's taxonomy. If you recall it from the last two sessions, this was Bloom's taxonomy from below to above. So, we will look at two videos for the lower levels, we will pause, do an activity and then the next two levels and so on. So, the first here you could see that there is a question that has been posed. Now, how do we do this alignment can be very well understood if we try to take some example and from some domain and we try to generate various questions and learning objectives within that domain or for the given topic. So, the topic which I have selected for this understanding is a very basic topic that everybody must should be able to understand that is Ohm's law. So, I guess irrespective of your background everybody will be able to understand this topic contained and how we try to align the assessment questions to the learning objectives that we have set. I guess there might be some faculty members from humanities also, but when I am explaining the question I will try to give you some general description about the process. So, that you will be able to apply that effectively in your own domain as well. So, let us look at this question. The question says what the unit of resistance is that. So, it is a fill in the blank kind of question where it is expected that learner should write down or student should write down what is the unit of resistance. So, let us first try to understand what is the cognitive level of this question. So, I think it will be very easy for us to understand the cognitive level of this question that this question is at recall level. The very first level in Bloom's taxonomy. So, where what do we expect students to do? It is expected that students have already learnt the matter or the content and by posing this question we are just trying to students to recall it from their memory. So, it is basically we are just trying to make students remember what they have already learnt. So, here maybe it is expected that students should just write down that the unit of resistance is 0. So, if you read this description for cognitive level you can understand that it is basically recalling the facts remembering previously learnt material. Now, what you see here on the slide in front of you what we call them as action verbs. So, whenever we post assessment questions obviously, we try to make use of appropriate verbs. Now, there is a good way of understanding which cognitive level the question belongs to. So, here are some examples that have been given here. For example, level name reproduce. Let us try to make use of some of these verbs and try to find out whether we can create questions at the first cognitive level as per Bloom's taxonomy. For example, state Newton's third law that could be a question at basic recall level. Now, if you go back to this you might be wondering that here there is no verb that is action verb that is happening. So, in a way it is silent here, but I could have just simply framed the question like state what is the unit of resistance. So, I guess this should become clear to you that even if you find absence of action verb you could understand the intention of the question and that will give you clear idea at what level at what cognitive level the question is or maybe you could say name some of the systems from human body. So, where you could understand that basically the cognitive activity that students will be doing while answering this question is recalling something from their mind. So, right now we did this exercise. For example, I posed one question for you. We try to understand at what cognitive level the question is and based on that we are trying to come up with some generic list of action verbs which can make a help us in creating questions at that cognitive level. We will be doing this exercise for all the six levels before that let me just give you a very important instruction here. This list is not exhaustive action verb list what you see here for example, here some 10, 11 action verbs have been put up. So, this is not an exhaustive list. What I mean to say is it is not necessary that every question which has this action verb must fall into cognitive level 1 or vice versa. If there is one question which has an action verb which is not in this list that means the question cannot belong to cognitive level 1. So, basically you when you try to find out cognitive level of the question these action verbs certainly do help us, but we also need to understand what has been the intention of this question. What kind of cognitive challenge we are putting in front of students or what kind of cognitive activities students are expected to do for answering this question. Now look at this question. The question says which of the following will cause the current to an electrical circuit to decrease? Choose all that apply and certain answers have been given. So, there could be one answer or there could be more than one answers for this question. Now let us just try to analyze this question. Now you can really appreciate and understand that this question seems to be little at higher level as compared to the first question. So, here when we say which of the following will cause the current through an electrical circuit to decrease and the in the answer you find either voltage or resistance. That means here we are talking about some relationship that exists between voltage, current and resistance and beyond that we want students to understand the interplay between these parameters. So, we are moving beyond just remembering something from the memory. We are trying to understand or we are trying to assist students comprehension about it. So, suppose if ohm's law is one relation which tries to understand or judge the interplay between voltage, current and resistance. We are really trying to understand by means of this question that has students understood what exactly will happen if you vary one of the parameters or one of these three factors, what impact it will have on another parameter. So, this is something is what we can say that this question as you could say this question is at understand cognitive level. So, it moves beyond just simply memorizing whatever is the mathematical formula for ohm's law. It moves beyond that it makes students think about it, it makes students understand the relationship that these three parameters have with respect to each other. So, here as you could see in the description they are trying to understand meaning of this concept of ohm's law and not only that they might be able to paraphrase that in their own words or they might be able to explain some phenomena, some behavior by taking help of ohm's law. Now, let us come to these action verbs for understand level question. So, for example, here you could see illustrate, rephrase, substitute, convert. So, for example, if it is a something is given to you let us say in polar coordinate system and you have been asked to convert that into some rectangular coordinate system. So, if at all student is going to do this exercise or if at all students is trying to understand this two coordinate system it is expected that student must have understood basic what do you mean by coordinate system. Okay, so at this point you saw two examples one each in recall and understand level question. So, let us do one polling question at this point. So, I am going to first show you the polling question please do not show anything by chat. After I enable the poll, the A view poll you can send your answers using A view poll. So, I will just quickly explain the question. There are four choices you should not answer any of the technical questions in these four choices. The actual question you need to answer is the one given in red which of the questions below belongs to understand level. So, read the questions here and I will enable the poll shortly. The first question has some symbols and there are four choices totally. So, I am going to go ahead and enable the poll at this point. So, let us actually go through the choices here and you can do a self-assessment of your own poll. Let us go through choices one by one and see if it is at an understand level or if it is at some other level. So, if a student is asked to identify the symbols some symbol is given and they are asked to say what element or what does the symbol represent. What a student has to do at this point is in their own mind remember which symbol belongs to which element. It is like a match the following and there is not much a student can do at this level other than memorize and remember. If a student does not remembers it is not possible to try to figure out on their own how to do it. So, the first one identify the following symbols is a recall level question. Let us look at the second choice two wires one of copper and the other of iron are made of same length and same radius which will have more resistance. So, what does the student have to do in this question? They have to try to interpret what is it about copper and what is it about iron that makes the two different and how does that feature translate into higher resistance or lower resistance. So, this question actually quite well belongs to an understand level where the student has to grasp the meaning behind what is asked. They do have to remember something about resistance that resistance depends directly with respect to some variable and inversely with respect to some variable, but they have to interpret that from the question. In a way it is a translation of the formula that they have memorized into the actual question. Let us look at the other two choices also write the mathematical expression for Ohm's law or it can be any law here. This is purely a recall level question and the final question construct a voltage divider that produces an output voltage. This is a much higher order question because the student is actually being asked to construct of quite a complicated circuit. So, this is much higher than understand. So, in this question B would be at the understand level. So, let us move on I am now going to play two examples of the apply and of the analyze level and then we will take one more poll at that point. Let us now move on to this third cognitive level. So, let us see what is this question. Use Ohm's law equation to provide numerical answers to the following question. So, the question is an electrical device with a resistance of 3 Ohm will allow current of 4 amperes to flow through it if a voltage drop of dash volts is imposed across device. Now, let us try to spend some time in understanding this question itself first. So, as you could see when student is trying to answer this question, it is already known to the students or students already have this linkage that resistance is measured by or the unit of resistance is zone or current is measured by means of amperes. So, the question what we had earlier put for remember level or recall level. So, that level of knowledge is already expected in this question. Not only that, suppose when student is trying to answer this question, it is also expected that student must know what is the relation between these three parameters. So, obviously here one has to make use of Ohm's law. So, that Ohm's law should be used and how that should be applied in this new context that is something is expected from the students. So, Ohm's law is a generic behavior of some phenomenon. Now, here a typical scenario or situation is given for example, an electrical device with resistance to and so with current and so and so and you have been asked to find out something else. So, whatever knowledge student has gathered in the process of understanding Ohm's law now need to be applied in this new situation. So, we can understand that this typical question not only is taking care of the very first cognitive level that is recall, but it is also expected that in order to answer this question student must have come up the second level that is understanding. So, then what is this cognitive level of this question? I am sure you all must have guessed it correctly. This question is at apply cognitive level. So, let us read the description that itself will make the positioning very clear. So, what it says use knowledge in a new situation that is what precisely we have seen in this question. Knowledge about Ohm's law was used in this new situation where certain parameters were given to us. So, it may involve rules, methods, laws and principles. As you know rules, methods, laws these are generic description of certain behavior and they are trying to apply them in the new context in the new situation. So, these are some of the examples of action works that we can use while posing apply level questions or the questions which are at apply cognitive level. So, you could see here that you can understand when student is trying to do a task at apply level, student has already achieved you can say mastery into the first two levels that is remember level and understand level. So, that is how we say that lower cognitive levels of Bloom, they are subsumed into the higher cognitive levels. So, far we have covered first three levels of from Bloom's taxonomy that is recall or remember understand and apply. Now, let us move on to higher order levels that is analyze evaluate and create. It is definitely going to be very interesting exercise for all of you. So, here is the question. So, four circuits have been given and what is asked is find out the what method has been used in each of this to control the current in the circuit. So, let us read the question. In the circuit shown in the diagram A, B, C and E what method has been used to control the current in the circuit. So, if you try to understand or if you try to look at this question, it is definitely obviously expect students to know Ohm's law or the relation between voltage current and resistance. It moves beyond that that there is some typical scenario that is given to us in which we need to apply them, but I would say it further moves beyond that because here is in order to answer this question students need to analyze this situation very minutely they need to understand how this things have been done differently all the four circuits every situation has to be properly seen and then only this students will be able to answer this question. So, this brings us to the next cognitive level. This is what is analyze. So, as the description goes while analyzing a typical scenario or while posing a question or answering a question for analyze cognitive level, we separate whole into parts until the structure of whole and relationship between part is very clear. So, maybe that exercise will be required here to find out how much it has been the current in the circuit or what will happen in scenario A and scenario B or how scenario C or D has been different from A and B. So, all that has to be worked out very clearly to answer this question. So, this is what is the fourth level of from Bloom's taxonomy that is analyze. So, here are some verbs action verbs which might help us in posing questions or writing questions for analyze level. Now, sometimes specially in engineering domain we find that this apply level and analyze level they slightly merge or they slightly overlap and you therefore you might find that some of the action verbs might be common in apply level as well as analyze level. So, again as I have said previously it is you really need to understand what has been the intention of this question and based on that we have to take this decision which cognitive level the question belongs to. So, let us do one more question at this point. Here the question you need to answer is what is the cognitive level of the question below? You please do not answer the actual question itself, but what you need to answer is this red question what is the cognitive level. Now, if you do not understand this domain very clearly what you can do is try to see what is being asked in the question. Is the student being asked to interpret something make meaning of something or is the student being asked to calculate something or as Munal mentioned with respect to analyze level is the student being asked to look at a complex system delve into its different parts and try to make sense of the relationship between the parts. So, see what the question is actually asking. So, let me just explain this question a little bit. There are two graphs A and B which are shown it can be for any quantity one represents one some system in this particular case if you know the domain it represents a series question if the other graph represents a different system. So, the student is being asked which of the two represents some a particular system. So, think of it in the abstract manner I am going to start the poll at this point good looks like most of you have are getting the idea that here it is an analyze level question and let us just quickly understand why it is analyzed and apply does not seem to be here see the thing is we are in order to figure out which system each of these graphs belong to. First the student has to look at the system itself the series combination then break that system apart think that well in a series combination this is what happens and the resistance is lower or higher because of this reason in the parallel combination something else happens. So, the very first step in a in doing this question for any student or anybody who wants to answer this question is to look at this complex entity and break it apart look into what are the pieces inside that system. So, anytime you are asking a student to do this they are working at an analyze level. Once they realize that a series combination has these elements called resistors then they start doing the calculation about how much is the resistance is it high is it low and so on. So, as we mentioned earlier any of the higher level objectives actually does involve tasks at the lower level it is also. So, you will see that the higher level questions are actually very well framed they are good examples for students to apply all their skills right from recall up to the level it is at. Let us now look at the last two levels we will look at two examples and then we will look at one more series of questions. This is again another interesting question let us look at it. Which of the given circuits can be used to drive seven segment display? I guess many of us are familiar what we mean by seven segment display or maybe what we see on railway platform as an indicator. So, where we are able to see or the alphabets can be seen by means of a display that is creating using LEDs. Now, so in this there are two circuits which are given and assume that we already know what is the value of voltage out resistances are known to us and what is asked here is which of the given circuits can be used to drive seven segment display. So, it is very obvious that some kind of decision is expected from students. But what is more important to understand here is on what basis the decision should be taken that is not very obvious. So, that also has to come from students. So, when we say that the circuit is to be used to drive seven segment display that means, in a way though it is not explicitly mentioned here in the question the people from domain can understand that it means that it should not be allowing more than what is required the amount of current that should flow from the seven segment display. Otherwise if it is exposed to excessive amount of current the display is going to get burnt out. So, so that means, we are trying to avoid this situation. So, this is the sort of criteria that we are setting for evaluating or taking the decision of this evaluation. So, based on that one has to work out how much current circuit A will produce, how much current circuit B will produce and then based on that we will have to find out that which of these two circuits will be a safe option to drive seven segment display. So, what is getting highlighted when we discuss about this question or from the discussion about this question that it is some judgment or some kind of decision is expected in this question. And also as I have said the criteria also has to be explained or also has to be thought about it. So, it is not very obvious that the student has to think about what kind of criteria should be applied in order to take the decision so that this question can be answered. So, that is how it leads us to the next cognitive level which we call as evaluate. So, as you could see in the description here it is expected in this kind of question that there must be some judgment based on some criteria or suppose somebody is saying or that circuit A is a better option it is we are trying to ask or we are asking for justification what has been the justification for this decision on what basis this decision was made. So, these kind of cognitive activities are expected from students when students are answering this kind of question. So, again I guess it will become very clear to you that we when we are at evaluate level we are taking care of all lower levels of blues. For example, it is right from recall then understand, apply, analyze and then we are coming to this evaluate level where students are going through all these lower levels and they are becoming now expert to answer the question at evaluate level. So, generally you will find these kind of verbs when we are trying to pose question at evaluate level. For example, compare, evaluate, criticize, conclude. So, where you could see that from literal meaning of these verbs also tries to indicate that we are trying to come up with some kind of judgment, some kind of decision by making use of these words. Let us come to the next question. It says using Ohm's law construct a voltage divider to obtain x voltage from battery bank of y voltage. Now x and y could be just numerical values. So, what we are trying to do here is we are trying to understand the general nature of the question. So, the question says that it is I would say it is a little open ended question. It says using Ohm's law construct a voltage divider. So, here is some new creation that is expected from student. What kind of creation? It is a voltage divider circuit. The situation is given to us that here is a battery bank from which we want certain amount of voltage. So, now it is completely students freedom that students will have to design this circuit. So, whatever knowledge they have gathered so far related to Ohm's law, applying Ohm's law in new situation, making decisions about whether x circuit will be better or y circuit will be better. All that will be taken care and using all these things it is expected that students will make this circuit which should fulfill the requirement that is given in this question. So, here you could see that it is a completely new creation that is coming out of this cognitive activity. So, no doubt we try to put this question at the highest order or highest level from Bloom's taxonomy that is at create level. So, in a way you could see here that it will generate new ideas, products right and it will be sort of a synthesized activity. So, number of things will be put together so as to make this product. Now, many times we find that the answers to create question, create level of question they are not straightforward or they are not unique. When you give this question to a class of 40 students, it is not necessary that every student will come up with the same circuit. That is a freedom or that is a sort of creativity that you are you want in this kind of question. And therefore, these new ideas whatever will get generated they need not be unique and every student might come with different kind of solution. The final object or final goal of this question has to be achieved that whatever is the task that is given to them that must get fulfilled. So, here is a list of action verbs which we can use for posing or creating create level questions. So, the very obvious ones are for example, design, combine or synthesize, develop. So, all these questions or all these action work they try to take care of the questions at the highest level of as per Bloom's taxonomy that is create level. So, let us revisit our triangle of effective learning and in what order we should do things. So, until now we have seen how to write assessment questions that are aligned to learning objectives and before that you saw some learning objectives. Now, there is an educational theory called backward design and I will explain what the theory is and we will also discuss why it is called backward. So, in backward design we first talk about learning objectives that is we write learning objectives for our course like we did in the course planning and concept mapping sessions yesterday and the day before. After that unlike our typical experience where we go and plan the next day's lecture we do not think of what strategies we should use, but as soon as we write learning objectives what we do is in fact write assessment questions before we plan what to do in class and that is why it is called backward. Why is this a good idea? And the answer is that once we align our objectives to assessment we know clearly what is to be done in the class or in homework. We have already set our expectations not just for in terms of this is what students should be able to do, but also how we as teachers will be able to assess them. So, in backward design what we do is we create a bank of questions, a large bank of questions which actually help us later pick and choose various questions at different levels when we want to set a question paper or given assignment. So, what is done at this point is we will do I will explain one exercise, we will do it partly in this session and partly in the next session and part in one of the assignments. We call this exercise as blooming all the way. What it means is that we will write a series of questions, assessment questions either for an exam or for a project or for an assignment, but a series of questions in a particular topic for your course which go from recall level to create level. And there is another objective of doing this exercise and as a community of college teachers what we want is to create a question bank in various topics which we can reuse and we can share with each other. So, we have been doing this activity for many types of topics and for many formats of questions where one teacher writes a particular question, somebody else reviews it and then it becomes a part of this common question bank and which we as teachers can use. So, before we actually do this exercise what I would like you to do is show three examples from three different domains of questions that bloom all the way. The first topic is from is actually a topic from 7th or 8th standard mathematics and I have chosen this topic so that hopefully everybody will have an idea will be able to relate to this topic and the other two are specific I think one is in machines and one is in computer science. You can pick any to understand this idea. We have just posted a number of examples so that at least one of them will be familiar to you. And the first one I will walk you through in a little more detail and hopefully this is something that all of you will be able to relate to ok. Here the questions go from top to bottom so unlike the others. So, this is something like a 7th or 8th or 9th standard high school maths and the topic is coordinate geometry. The recall level question is state the formula for Euclidean distance between two points. It is very clear why this is a recall level question. It is the memorization and stating of a formula that a student has to do. This can be given as a short in class exercise or maybe if you just want to warm up question in a quiz you can give such a question to a student. You can consider all of these as test or exam questions. Let us look at the understand level question. Here the student is being asked to draw a diagram to explain the meaning of Euclidean distance. The student has to not just grasp the meaning but also explain it and not only by words but also by a diagram. So, they are making some connection between the formula for recall level with the diagram and that is why it becomes an understand level question. The apply level question in this particular topic is asking students to calculate some number. So, the question is what is the air distance between Rastrapati Bhavan and Red Fort given their coordinates on a map. So, they have to actually apply this formula to these points and realize that in fact even before applying the formula the students have to realize that these coordinates have to be taken as the two points and the air distance means Euclidean distance formula, shortest distance, distance that a crow flies. All these are different terminologies that we use but it is a calculation problem. If you go to analyze level now things get more interesting and more challenging both for the student and for us as teachers that here the student is being asked a word problem. Let me just read it out. Ram flies in a helicopter from points A to B then from B to C then from C to D as shown on some diagram. How far is he from the starting point? So, some of you may think well this is also a calculation problem and it is okay to consider this as apply. But the reason we consider it as analyze is again that in order to apply the formula, in order to make calculations with the Euclidean distance formula the student has to draw the diagram and break apart this four point diagram to realize that the distance he should be calculating is in fact or the two points he should be using are in fact A and D. They would not have too much meaning for this particular problem. So, there is some breaking apart of this I would call it a system or some complex whole before the student has to apply the formula. So, when you want to do analysis level questions think of questions like this. Give them a circuit and ask them to analyze it or give them some system and ask them to determine the relationship between its parts and so on. Evaluate question is a classic decision making question. Decide if you should use the chessboard distance or the Euclidean distance to calculate the cost of the road connecting two towers. So, the student has to first analyze what is the difference between these two formulae and then make a decision that for this particular goal cost of the road which of these two formulae makes more sense. So, evaluate level questions should be as challenging as this question. And finally, a create level question where the student is asked to develop a program in scratch. Scratch is a programming language it is a visual programming language without syntax that even 5th standard students can master. So, the student is being asked to create a program that takes values of the coordinates between two points and outputs the distance. This can be done in a lab or as an assignment. So, one more point a couple of points I would like to stress. One is just note that for any higher order question the student really has to be able to do all these levels. They are embedded within the higher order question. Second point is that while the action verbs are important, the action verbs themselves do not really specify anything. So, for example we got a question yesterday and even on day one that write a program, is it apply or create? So, what you have to see is the task that the student is doing when he or she is doing is answering that question. Is the student working cognitively at an apply level or at a create level? What are the sub tasks that a student has to do in order to answer that question? So, action verbs are indicators they are good hints, good tools for us to write questions, but they are not an end in themselves. One more example of the same action verb being used for two levels is you may ask a question which says compare and contrast. So, if you say something like compare and contrast living and non-living beings, you may put it at the understand level because the student simply has to think about what are the properties of living beings and the properties of non-living beings and write down the answer. So, it may be a simpler question. On the other hand, if you ask a question compare and contrast these two methods for building a road, it becomes an evaluate level question. So, what I am trying to say again is that use action verbs as clues and indicators for you to construct these questions, but just because a question has a particular action verb, it does not indicate its level. So, here is an example in manufacturing engineering. I will just keep this on for some time for you to read it if you want to if you think it is relevant to you. If you do not understand the domain again read it and see what is being asked. So, in analyze determine the optimum number of something is being asked. So, in many of your domains a question that starts with the stem determine the optimum number of something is a valid analyze level question. Not just determine the number of, but determine the optimum number of. So, there is some analysis that is needed here and one final example this is in logic gates in programming in CS subject. So, just take a look again we will post all these examples on Moodle. So, you will have access to a lot of examples. Okay, what next? So, this is an assignment that you will do in groups you do not have to do it here. You have to pick a topic you have to first find a partner in your domain choose a topic a complex enough topic that you will need two or three classes to teach and you will be this is a Moodle assignment that we will post you will be writing six assessment questions very similar to the examples that we saw. Note that these are not merely learning objectives while the stem from learning objectives they are actually actual questions that you can give students during a quiz or a test or exam or assignment and so on. We will break for T and there is some homework for you over T. So, find your partner and over T discuss one example of a create level question. Why? Because in the next session when you come back after T we will look at how to assess these open-ended create level questions. So, I want you to actually think of a create level question before you come back for the next session.