 Okay, I think we'll get started. So this talk is about a new method of teaching, it's called flipping the classroom. Before I get into the details of the talk, in fact, there's a lot I wish to convey. So if you have any questions, if it's going to block you from continuing with the talk please by all means ask, otherwise reserve them to the end. So our goal as educators is to provide high quality education. So we have both parties here, I see faculty, I see students. Okay, so let's start the talk with what I'll call grumbles and groans because we have a lot of complaints about the way classes are conducted. So let's start with the instructor's outlook. So I'll tell some of my own experiences. So often an instructor spends some time thinking about how to teach a specific concept, he says, should I teach it this way, should I teach it that way? And maybe he spends a lot of time animating something. So constable amount of time is spent. And then he comes with a sprint in his leg to share his excitement with the students. But then what does he see in a class, out of a class of 100? How many do you think are present? 30, 40, of which maybe 10 are already dozing. Others are sleepy-eyed, and somehow it just kills the excitement. So that is one of the complaints which many of the instructors have. So from the audience, any of the instructors you can share, any other complaint, quite happy with the way classes are organized. Too many complaints, they don't even want to start. Whatever the students, what is your wish list? If not, complaints, how would you like a class to be? Just share two, you like a lot of discussion in the class. So course work should be guided by the discussion that happens in class. So that is one thing, do you think that doesn't happen right now? So a few courses it happens. Okay, not at all. Okay. So I would like to see more of it. Okay, any other, one more from the students? Don't feel shy, I mean, flexible deadlines. Flexible deadlines where you can work whenever you want. Okay, so the reality is, I mean I'm coming more from the instructor's perspective, which translates into some aspects of translating to the student's perspective also. So the reality is we are now having to deal with very large class sizes, especially the undergraduate level, it has hit 100. Even at the postgraduate level, some of the courses, it's exceeding 100. So basically we have very large class sizes. So what this translates to is there's a lot of difficulty in giving personalized attention. So especially when you want to have something like a lot of discussion where you pose questions, you want students to give their feedback, you can only target a select few students. You really cannot involve everyone as part of the discussion. And often there'll be few active students who hog all the attention. Right, so that typically happens. And there is also the problem that you can't give enough think time for weaker students to solve a subtle problem. Maybe they need 30 minutes or maybe they need 20 minutes. So you really pause the class. I mean you have to cover certain amount of material in the class. So you really cannot give enough time for the weaker students. And this discussion also, there's only so much questions you can ask because again there is a specific time limit. You need to convey certain amount of information. So you can't have a free for all discussion and going on. So these are some of the problems that we face in the large class sizes. Another problem which is this problem of poor attendance. So there is a mismatch between students waking hours and class timings. So 830 class is like lot of groans. I mean it's, even I personally can relate to them because I also hate the 830 classes. And what happens often is once a lecture or two is skipped which becomes inevitable because you get involved in so many other activities, catch up is not easy and that's at least what the students think. I mean if you are really dedicated, I'm sure catch up is not that difficult. But once you skip, then that cycle continues. You just tend to skip because you don't know what has happened earlier. You can't follow what's happening right now. So this is again something that happens. So the question that this talk is going to address is how should one conduct classes so as to facilitate effective learning. Specifically we'll deal with large classes as well as the problem of poor attendance. So the solution, this flipped classroom has been gaining traction in recent years. So the idea here is basically you flip a traditional classroom. What happens in a traditional classroom is the instructor lectures and then the students go home, do the homework and then again come back to listen to the lecture. In a flipped classroom it is the opposite. So you watch lectures outside the classroom. So these are basically videos which students watch at their own pace and time. And in the classroom there is an activity that happens. So this could be solving some problems, discussion, whatever it is. So it has basically flipped. So that's the basic idea. Now when it comes to executing flipped classroom, each instructor has their own ideas and how to go about it. So what I'm going to walk through hence forth is I'm going to present a case study of a UG course which I had conducted using the flipped classroom model. So this had about 95 students. The course is a computer networks which is a third year undergrad course. This is a compulsory course. So that's where the registration is high. So here is the outline of the talk. So as an instructor I had certain objectives that need to be met as part of this flipped classroom model. State what these objectives are. And then I'll tell how I went about implementing these objectives. Then I'll also show a demo of the platform that I have used for conducting this flipped classroom. Naturally, since this was an experiment, I did a very detailed student survey. I'll present some results from the survey. I'll also share with you some of the insights I had gained as part of conducting this particular class. And then I'll talk about some ultimate vision I have in this space. So that's the outline. So here are the main objectives. So naturally when you're moving instruction outside the classroom setting, so whatever you want to teach is now going to happen via the video. So the goal is you should mimic in classroom settings as much as possible. So what happens in a classroom setting? There's a lot of discussion as mentioned. You ask questions, students respond, you tell something. So that same setting to the extent possible you want to capture it. So what this translates to is the videos that you want the students to watch have to be very interactive. So for that, you need to embed questions, activities as part of the videos. And when someone solves a question, you need to provide feedback to the students also as to, oh, you may have missed something like this. That's why your answer is wrong. Some kind of feedback has to be given. So the way students proceed is they watch, they pause, they think, understand, answer, and then again continue. That's the way in which the instruction happens. The second objective I had is you have to retain student focus. So in a traditional classroom setting, if some student is playing Angry Birds on his smartphone, I can just confiscate the smartphone or kick him out of the class. Whereas here, there's no control. I mean, I don't know what he is doing. He may be playing. He may watch one slide and then go ahead, play some game, come back. So you really need to retain students' focus when he is watching. So for that, naturally, exciting material is the way to go. But it's often not always feasible. The next best thing you could do is to keep the video short and sweet. Anything about 20 minutes, students tend to get distracted. So you need to keep it simple. In fact, many have requested to keep under 10 minutes. But that's a bit of a, I mean, you really can't convey much. Another objective which I had in mind is this provide complete learning. So again, in a typical classroom setting, you tell something which can span multiple lectures. And then there is an activity, either a demo or reference material, which all happens outside the classroom. But in order to learn a concept properly, it may be that you listen to something and then maybe do a demo of it, then come back and continue. So that will give you a better understanding of that particular concept. So you basically want interspersed video viewing as well as study time where you reference this material, do some practice problems, do a demo, whatever it is. So you move ahead only after mastering a concept. So you want to provide this complete learning as well. So in order to achieve these three objectives that I have listed, what it has resulted in is what I call a multimedia textbook that is going to be hosted on a web platform. So what you do here is in a regular textbook, if you see there are chapters, there are sections, there are review problems, practice problems, so on and so forth. So you have a game idea, except that instead of text, you now have a video. So what you do is you come up with these concepts. Typically a subject, you have many concepts you want to cover. You package these concepts as modules. Typically you try to set the video length less than 20 minutes. If not, you try to split it up. And then you neatly organized all these modules as chapter sections with all the supporting material in one place. So that is what forms a multimedia textbook, and this is what you're going to host on a website for the students to reference. So that's outside the classroom setting. What happens inside the classroom setting? Again, this is the tutorial portion of it. And my objective here is to provide some kind of personalized attention. In order to provide some kind of personalized attention, especially if there's a large class, it's a good idea to divide it into smaller groups. So each tutorial should contain no more than 30 students or 35 students, ideally even lesser if possible. And then as part of the tutorial, you need to excite, challenge students. You need to be able to facilitate group, learning, discussion, so on, so forth. So some of the discussion aspects which are really not possible to do in the outside classroom setting when you're watching videos, you try to do it inside the classroom. So the end result of these objectives in implementation, it resulted in me dividing the 95 students into groups of 32, three groups, groups of three consisting of 32, and three one-hour tutorial slots for the three groups. So each group has one-hour tutorial slots, but since I'm managing three groups, there are overall three tutorial slots. So this is the typical weekly plan I followed in running the course. So outside class hours are about five hours, which consist of video time totaling two hours and study time which is three hours. This is interspersed so they can do whatever they want, watch video, do practice problem, do a demo, whatever it is that they want. And then there is a tutorial session for one hour where we meet in small groups of 32, three slots over all the three groups. So one of the first things I do when we meet is there's a five to 10-minute recap quiz just to ensure that they watch the video. It's just a review quiz, just so that they're on top of things. This is followed by discussion, clarification of questions, some practice problems, whatever it is. I'll talk more about what happens in the tutorial in a later slide. So you can think of it like the regular slots are three, we have three slots in a week, right? So each slot I meet a group. One group. Sometimes I meet if I'm conducting a quiz or some such makeup classes. So in terms of grading, the way this ran was there were periodic quizzes just to ensure that students stay on top of things, which constituted of weekly tutorial quizzes of 10% review quizzes every three weeks and then the remaining grade was made up by the mid-sum and final. This is how the course was organized. So outcome of this model is, this is basically the advantages of this particular model is the PFC learning, which I call personalized learning. The first P stands for Personalized Learning. So when a student is watching a video, it's as if the instructor is talking just to him, so you feel special, hopefully. One very important aspect, which is not possible in a traditional classroom setting, is all get to answer the questions in the video. So in a typical traditional setting, I can only ask you answer this question. I can't target everyone, but here everyone gets to answer the question. So that's something very important. Again, you answer without fear of embarrassment. So mostly UGs are not that shy, but I see a lot of shyness in the MTech students, even though they know the answer, they just keep quiet. So here there is no such fear of embarrassment. You could go ahead, express whatever thoughts you have. And because we are dealing face-to-face in a small group setting, the focused small group tutorials also give a kind of a personalized feeling. So that is a personalized learning aspect of it. Then there is a flexible learning aspect of it which students loved as you can relate to. So students' choice of time plays group. So they could watch whenever they want, middle of the night, 2 o'clock, whatever it is. So that's the flexibility. And importantly from a learning's perspective, the students' space. You can take as much time as you want to solve a problem. So for example, something like in a classroom setting, if I ask a question, I hardly give one or two minutes for you to give an answer. Here you take 10 minutes, you take 20 minutes. It doesn't really matter till you understand what's happening. So you can do it at your own pace. And if something is not clear, you can even watch multiple times. And of course the flexibility, you have to be careful. This can to some extent be avoided due to periodic quizzes. So you really cannot procrastinate watching because you'll be quizzed. And then there is complete learning. Each concept is complete with all the materials. That way you can interspers your watching and study time and only move ahead once you master the concept. So this is the outcome of this model. So I'll show you a demo of the platform that was used in running this particular course. There were many contributors. All these, the top are all, are recently graduated BTEC 4. They've done a fabulous. I'm really proud of them. So now going forward, Mayank Megwanshi, who is a third year student, our own student. We are working on version three now. This course was run on version one. Version two, Bhaskar is actually using it to run his course. Sake Choudhury is in fact outside the department chemical engineering student who is helping me in developing the platform. So let me just show a demo of the platform. So this is the webpage. In fact, those two are the guys who had developed this version one, Saif and Alankar. So this is the course. So you can see there is some description. Viki is where I put up the schedule as to who can watch when. In this week, you watch these videos. This week, you watch these videos, so on and so forth. So that schedule is put up in there. And here is the organization of the content in the form of you can view these as chapters and within, for example, each chapter, these are the sections which cover different concepts. For example, if you go here, so this has slides. There is a quiz associated with this. This is a quiz outside the video because this is a practice quiz. So it lasts certain questions you have. Well, when I log in, there are bugs because I try to modify things when I do it, but students don't have such kind of bugs. So here is the video. So I'll just quickly, so as you can see, there are questions as part of it. I don't think there is a sound card here that plays, but as you can see, questions pop up. You have to answer the questions. And based on what you answer, it will tell something or the other. So you can give any kind of feedback here, not just correct. So this is how a video looks. And then there is a discussion forum for the students to communicate any questions. A student can also track his progress. So in each chapter, how many points did he score, so on and so forth. And what is interesting, this is another thing that is interesting about, I don't know, they'll blame me for displaying this, but I'll quickly show and take away. So basically what the scoreboard does is it shows how the students, how many problems they have solved and how many points they scored. As you can see, this is, so overall it's about 191. 191 is the total number of problems. As you can see, majority. It's unbelievable. I've never thought they'll do it, but they have in fact solved problems. Many of the problems that I had put up. So that's part of the scoreboard. These are the problems that are outside the quiz. Within the quiz problems are also there. So there are outside ones are more of a testing your understanding of the concept. Inside the quiz questions are about directing you to the right solution. So there are more like the kind of questions you ask in class for facilitating understanding. So based on, after the running this course, I had done a very detailed survey. In fact, I'm only going to present a subset of the results. I'm happy to share with you the results of the survey if you are interested. So there were 54 respondents. Not easy. Had to coax them a lot. So the positives, of course, everyone loved the flexibility, convenience, freedom. So the words were peppered all over the good things about the things. So that's what they love. Another thing which they appreciated is the fact that they could rewatch videos. Some watched tooth teeth rimes. So they watched once before the quiz, before the Midsum. They, in fact, again watched. In fact, they had a marathon of watching all the videos just before any exam. They also liked the fact that they had small tutorial groups where they could get more attention. Another thing they really appreciated is this outside the video quizzes where you solve these problems and you get instant feedback. You solve something. It tells you whether it's correct or wrong. So that's something. So it's kind of a competition in trying to solve all this problem who came first. The scoreboard, as you see, many of them attempted it because the first few people got a treat from me each periodically. So there was some kind of competition to be at the top of the scoreboard. So this flipped classroom model is no magic one, but will make every problem disappear. It has, in fact, a few negatives. One major complaint, in fact, again is something that is there all over the survey form. No immediate feedback. Basically, doubts are not cleared right away. So you're watching the video. You have some question. You really cannot get it clarified right then and there. You really have to wait for the tutorial to get it clarified or you can initiate a discussion forum something, but it's not an instantaneous feedback. This is something which many people did not like. No easy solutions for this, but there are three solutions that are feasible. One I would call is an instructor watch. Basically, you specify a designated time during the week where you are available. These are like office hours, say, you are there. You assemble in a class. They all watch. And whatever questions they have, they can come get instant feedback. This needs some kind of discipline. I offered this rather a bit after the Midsim, but no one really, even though they had a lot of complaints about it, they really didn't take up this offer. So that's one. So the other thing which instant, again, is not an instant solution, but as people watch, they can actually tag the videos that this portion of the video was not clear to me. And as more and more people tag, I can redo the video or provide explanation at that particular point. So over time, you can really maybe two, three years down the lane after sufficient batches go through it. Maybe the quality of the video itself can be improved to an extent where the confusion or clarification needed will be quite less. So that is the second solution. The third solution is which they did, is group watching. Four or five students should come together, watch, and this peer-to-peer discussion that results. I think there's a lot of learning to be had when you do that peer-to-peer discussion when you're watching. So that's something I have to see how to encourage them to do this group watching. So that's one negative. The second negative is that the flexibility kills discipline. So there's a lot of last-minute watching. So you watch right before the tutorial or just right before the quizzes. So that's why I had very periodic quizzes just to ensure that they were watching continuously. Again, solution is weekly and periodic quizzes. And again, the instructor watch also enforces some kind of discipline on them. This generation, again, has attention deficiency. So it's called the cell phone generation. So to address this short videos, I think, are important. Again, group watching or even an instructor watching can help. You can even design technology where you can ensure that they watch. You basically log down the browser, make it very difficult to exit. You have to, until you watch, you cannot exit until they reboot the machine or some such thing, then they're just forced to. So I put a smiley that is not really a solution because I know no one will like it. But you can potentially, if you want it, I can provide the technology for that. Another complaint was this reduced discussion in class interaction with the instructor tutorials. To the extent possible, I've tried to make them as much to facilitate discussion as part of them. But tutorial is something that has to be carefully designed to ensure, without proper design of tutorials, as I said, that interactivity part can just be killed. So I'll talk more about this sometime soon. Some complaint that's really boring to watch videos by self. As I said, group watching is the way to go then if you're finding it boring. Taking notes, some complaint is difficult because the videos are rather information dense. In 20 minutes, what I cover in a video, typically in a traditional classroom setting, it can range anywhere from one hour to one and a half hours because there are no pauses. There's no writing on the board. There's no looking to see what the feedback is, so on. So they do tend to be dense. So here, again, technology can help. Some things which are missing in version one which have been rectified later, they weren't keyboard shortcuts to pause play, so they had to move the mouse. But this can be ensured. So you can also make available transcript of videos which you can mark, highlight by the students. You can make available a note taking annotation tool. So all this, I think, technology can easily fix. So here are some of those negatives, as I said, have come out from the students themselves. So in the survey which I've conducted, I asked them what are the three things you hate about this model. So they have, so many students replied. I just, so I didn't show you the count of what. So the, yeah, I'll come to the instructor perspective shortly. So I'm just telling the students perspective, all this is students perspective. As I said, the first one was the major complaint. Rest of it is very few students, maybe no more than two, three. So that's the thing about it. So in the survey, I also collected statistics like did you feel the need for instructor presence for explanation or clarification while watching the video? As you can see, very, I mean, there is a good number who did feel the need for the instructor. So that's one thing to take away. I also asked, within video, the quizzes, how many questions did you attempt to solve seriously? A good number did seem to solve it seriously. So within the video quizzes, there is in fact no marking, but outside video, it's a function of attempt. So you do it in first attempt, you get full marks. It's divided by two based on, yeah. So quiz outside video, did this help in overall learning? So majority have said yes, it had significantly helped. Again, approximate time spent on a 20-minute video. So a good number have spent more than 20 minutes. Okay, this is the question which I'm sure everyone wants to see. So if your overall score for this new model of teaching in comparison to the traditional model of teaching, so zero means they are at the same level. One, two, three means the fifth classroom model is better. Three means best. The negative means they didn't like it that much. So as you can see, 60, so I've removed the zero, the neutral people. So there are 66% in favor versus 28% that were against this model of teaching. So that's the student's perspective. So I'm getting into my perspective. So the first offering, I would say, is a step close to wellness. I've lost so much hair as part of it. I've never worked this hard in my life, I should say. The main problem that arose was each concept, which is just a 20-minute video if you have to prepare. When I started it, I just had a cache of 10 videos, but overall I had to make some 60-65 videos. And then I had to do these videos while running the course. And each video takes at least two days. I'm saying to continuous, there is no break. All I'm doing is focusing on the video itself. I haven't done any research last semester. So you have to do slides from scratch because there will be some copyright issues. You also wanted, since you're doing it once, you want to do it the best you can. So I did want to refer. Naturally, I may not refer to multiple books when I teach, but since it is going in recording, I just wanted to ensure that I'm doing it properly. So I had to refer to multiple books, really think about what I wanted to present. Then there was a question of recording, editing, and then quizzes generation was another major. Every concept I had to put quizzes. As it is generating quizzes for an exam itself takes up so much time. Now here, I had to populate every concept with quizzes. So that also took up a good amount of time. Something which I think, which if you do it, you will realize, sometimes I just found instead of editing a badly recorded video, it was just easier for me to do the recording again instead of a cut-paste and do the stuff. So some of the lessons, those were learned. But hopefully the following offerings, once you put in that effort, the following offerings will be a piece of cake. Maybe that will take you a step closer to fatness instead of baldness. Okay, so there is both the tutorials. As I said, they're very important component and need very careful design. The first 10, 20 minutes, definitely you will spend on the quiz to ensure that students have watched the videos and some clarifications or some questions they have. The rest of the time, you should do an activity that excites, challenges students and leads to discussion. Then again, this is very course specific. So something you could do is you could motivate a problem and discuss challenges involved in it and leave the solution approaches to the video material. So that may lend itself to certain courses. So within this, you can employ the think-pair share of Shreedharayur. I won't get into that model. So basically that facilitates group discussion. If there is a course that has some experiments you could do, but it doesn't have a lab, you could facilitate some hands-on experiments as part of the course where they again will feel quite excited. You could review challenging problems, basically apply a concept in a new setting that challenges their understanding. I mean, if some of you have attended the other MOOC stock, the Takami principle he was mentioning, I mean, you think you understood, but then when you throw a new problem at it, you realize you didn't really understand the concept. So you could post certain such things and do. There is another model which is employed in a flipped classroom model. This is called the what summary question. So basically you watch the video. Each student is supposed to summarize what he watched and then ask a question in the class. So the time is spent in just discussing some of the questions. Naturally, if you ask very lame questions, you don't get credit, but you ask interesting questions. It leads to other discussion. What you do within the tutorial, I think is custom fit to a specific course type. So that is something which you have to think carefully before you follow this model. The challenge in tutorial is unprepared students can ruin the tutorial. So basically you expect them to watch the videos and then your discussion, your initiating is a function that they should know the concept. And thereby, for example, you're trying to review a challenging problem. They should know the concept. If they didn't listen to it, if they didn't watch it, then they won't really contribute. So it's very important to assign weightage to preparedness. My quiz there is a result of this, but even this what summary question is also a result of some need to do to assign some weightage of the marks that they are prepared to before they come to the tutorial. So that's very important. So face-to-face interactions when you're teaching is something very special because facial expressions convey a lot. For example, you teach a subtle concept, you are explaining and everyone is confused and then suddenly it clicks and you see these ooze and ask. It's very gratifying for an instructor to watch those ooze and ask. Tutorials, as I said, can help to some extent in this case. And another thing is this confuse. You see confused faces. You know things. People are not getting it. Again, you can tell the same thing in other ways. Still, finally, you know everyone has understood it. Again, these are some of the things it's not... When you push instruction away into the videos, these are some things which you're going to miss out. Discussions to some extent they help, but it's not like a whole lot they can help. So student performance, well, there's just one sample. Whatever I tell you, you can easily dismiss it, but still I'll give the data. So if you see, I have taught this course in spring 2009, spring 2012, as well as fall, which is the previous semester. The average marks have increased. Maximum marks have also increased. Number of As, well, there were marks here. The people who scored more than 80 marks out of 100 is about 12. If you look at it, I don't know what to make sense of it, but there is... Well, this increase from 9 to 12, maybe it's because I'm teaching it the second time. There are many causes for it. Maybe this batch was great, or maybe I put preferred, which I did, or maybe the flipped classroom model really helped. One doesn't know. But my gut feeling is that this did help to some extent because as I said, if you saw the scoreboard, many students, in fact, 80% of the class has attempted a lot of the practice problems. So it shows that they were trying to learn something as part of the course. And I also spoke with some students, and many said that they've watched videos multiple times and it kind of helped them understand the concept better, so on, so forth. So my gut feeling is this increase is likely due to the flipped classroom model, but as I said, one sample doesn't tell much. So finally, I guess the question is, should I flip or not flip? So let me put together again both quick comparison. Most of the points I already went through, just flashing it to you. The flexibility, fixed time, focus, whatever, all those things, I mean, at times, I feel we go, we bend backward to cater to the students. Students do need to have some discipline. You can only do so much in that regard. So I won't focus on all that. Learning, let's focus. I think from the learning perspective, as far as the flipped model is concerned, the fact that you can watch multiple times, I mean, you attend a lecture, not everything goes into your head. Typically, textbooks help you to revise, but this way you can watch the video multiple times. This, I think, can help in the learning. Student pace as opposed to the instructor pace. Again, this is something that can significantly help. You can ask many questions. When I did a traditional, I could hardly fit in two, three questions in a classroom just because the ensuring discussion, I can't cover much. Whereas here, I can ask many questions. In fact, I love to follow the Socrates model where anything you lead to, you drive up a question and then go in there. So each video, which is 20 minutes, can have 10 questions, 15 questions, whatever it is. If there's a possibility to insert a question, I insert. So a lot of questions were asked. Again, this, typically when you are asking a question, you target only few students in the class, whereas flipped classroom model, every student gets to answer the question. So that's another advantage. In contrast, I think the traditional model, the immediate feedback is something you cannot, that's something missing in the flipped classroom model. The rest of it, as I said, if you look at focus, in a long one hour or one and a half hour lecture, you may lose focus in the middle. But when you're doing flipped classroom, if you keep it only 10 minutes, 20 minutes, the 10, 20 minutes, you can pay a lot of attention and then walk out, do whatever you want, come back. So focus, I don't know. Maybe it's same in either. So again, this is my take. There is no strong basis for it. In a given class, what I've often observed is there is 10% who are very interested in the subject. There's 40% marginally interested. 40% will do something to pass. And 10% are the least bothered. They're just in the system for whatever. Maybe their parents ask them to come here or whatever it is. They just want to get out. So who do we cater to? I think for 10% who are very interested, I think nothing beats the traditional classroom setting. I think if you want to cater to them, I would say go for traditional classroom setting. But a well-designed flipped classroom is not bad either. It's not like a major fall. But for the remaining, I personally felt that flipped classroom is better, both from learning as well as exam preparation perspective. So that's with respect to the flipped classroom. As I said, I have an ulterior motive for doing all this. So these are the wheels within the wheels. So IITB students as such are a privileged lot. We are here. Many of the faculty are great both on the research front as well as the teaching front. They can convey very effectively. But if you compare this with many outside students, the engineering education is really bad. I mean, I have talked with many people in many schools. In fact, colleges, the teachers don't even show up. They just learn by themselves. Or the recent graduated students teach the next batch. So some such things happen. So there is definitely a need for good video material for the Indian students, preferably ideally since I'm in CSC. The entire CSC curriculum should be prepared with focus on employment skills. Why not textbooks for the same reason why we conduct classes? Because there is something in the way we tell. So textbooks are very comprehensive. There's a lot of material. You don't know where to focus, where not to focus. So when you teach in face-to-face, you kind of take the important points and convey. So that's very important. Now there is CD, PENPITEL. So why can't you just go with whatever exists? I think in CD, PENPITEL, I have seen some of these videos, but I won't dismiss them outright. But one thing which I feel personally very important is this interactive videos. Videos have to capture as much as possible what happens in a classroom setting. So they have to be made very interactive. And you also need to organize and put together these concepts in fashion, much like a textbook. And another thing which is quite important in an Indian setting is what I would call a playlist feature. So for example, each university will have a different syllabus. Now you want to cater to that syllabus. For example, here I develop many concepts. You can just create a playlist of syllabus for the different universities and they can pull whatever they want and run that particular course. And material alone as such is not enough. This is something like when you are writing a textbook. This is as good as writing a textbook, except that it's in a video form. So you have to pay attention to a lot of detail. What question should I ask? When should I ask? What are the kind of feedback I should give to the students? So all that is not something where you just go, teach something, you record it and put it up, it doesn't cut it. So the goal is to cater to engineering education material to an average student who wants some skill set to get a job, not do research. So often times when we teach, we try to teach to cater to the research-bent mind because we teach all these concepts, subtle issues and all. But I think outside in, I mean, if you're going to outside engineering colleges, they are not much interested in research. I think we should just cater to them achieving some skill set to get a job in that particular discipline. So the platform, so what I have shown you is version one. We are right now at version three, version two Baskar is using. But the goal of the platform is the following. To create three interfaces, one interface is what I would call is for the content developers. These are people who create material, the videos, the quizzes, the reference material and organize it in the form of that multimedia textbook that I'm talking about. And then there are instructors who belong to other colleges or industry which will pull the content to run a course. They are given the opportunity to track student progress, facilitate discussion, do this in-class tutorials. All that will be facilitated by this platform. And then there is a student interface who basically enroll under an instructor. Now there are these edX, course array, et cetera. There are many such platforms that help. So the platform wise, I don't particularly care. I mean, as long as the platform facilitates an interface whereby you can conduct flip classroom models, whether it is edX, course array and all, it's rather irrelevant. We started developing the platform because at that time there was no open source. edX wasn't open source at that time. So we have already had the expertise. We were already in the middle of it. So we continued to develop it. And I also am picky in some, I wanted certain things done certain way which course array or edX, nothing supported. So I wanted my own ideas. So that's, but as I said, platform wise, it's easy to change it. It's not a big deal. I think what is more important is implementation and the content. I think that's where ITB can play a big role as a local facilitator. So this content you are generating has to be custom fit for the Indian market. So I'm saying, I was telling about the syllabus. Syllabus is one such thing. They are not going to create content based on our syllabus. Accent also which I've heard from others, some of the engineering students do have a problem trying to follow the accent of some of these Americans or Europeans. So that also is something which hopefully our accents are more accessible to them. But if need be, you can always leverage on existing content from any of these places. It's the putting together that is important. The certification trust is the biggest concern. I think since there are resources available in each of these universities, personally, I think it should be pushed to them. You can, with this blended model, you can leverage all these university resources, labs, face-to-face interaction. The groundwork I feel is the challenge. Basically, you need an interface between the content developers and the universities because they want some changes to be done. Or they want certain things done in a certain fashion. This implementation, I think course RA or edX is not going to do for you. Someone who is here, who understands their needs, who understands what they want, something has to be done here. So I am hoping that this will help in that regard. So IIT is B's role, again, my very personal opinion. So I feel we have a role to play in the following regard. We can develop this content. Basically, these interactive multimedia books that facilitate the split classroom setting. So this can cater both to other universities as well as industry. We can also do this teaching workshops where you can train teachers to do this split classroom model. We can help in question paper setting and I do not see any reason why we should restrict our BTEC200 or MTECH200. We can serve 1000 degrees where all the teaching happens through this thing and they just come here, give the exams and to some extent, maybe you have to see the tutorial part of it is a bit difficult to handle but to some extent, this can be achieved because the platform definitely facilitates something like that. So, cliche, but let's work for a better India. So teachers, I see quite a few. Please help create multimedia textbooks. I will provide any help you need in that regard including how to edit, how to, what tools to use whatever it is and as I said, first time is difficult, next time on it will make your life really easy. Not only that, you are going to help if you are okay with it, the content if we give it to other engineering colleges, you are really going to help a lot of these students. From the students perspective, I guess please continue to be our guinea pigs for a few more years, provide feedback on this particular model and you can also, we are actively often looking for people who can use the platform. So if you are good coders and want to contribute, most welcome. Right to me. I will, well more or less that finishes it. I also wanted to mention a few other pieces of work I am interested in that is again involved in this classroom setting. So I thought I will mention those as well. I think the instructors again will relate to this. 100 students meet some final when it comes to grading. I don't know about others, but I just hate the constant grind of having to grade 100 papers in a very short time span. And often I notice is that there are definitely design questions which you have to ask which you cannot grade automatically. You need manual grading for it. But I also have felt that maybe 40% or even 30% of the questions can be graded automatically. You really don't need an instructor to sit and grade. So all those that portion that's with respect to exams and when you do periodic quizzes like I was telling you every class you have to conduct a quiz. And there are 100 people to grade overall. It becomes a very tedious. I mean you are constantly having to force the TAs to grade something or the other and so on so forth. So the idea with this exam is basically whether you can conduct a proctored exam on smartphones. So people come to an exam room. They sit with their Akash tablets or smartphones and you push the exam paper and pull the answers. So it's not pure online. So they'll also have a regular paper which they can write. Whatever can be done online you kind of push it to this particular platform. So that is some work that's happening. There's some wireless challenges there because pushing, pulling happens. If there are 100,000 students by 100 students then things will collapse. The wireless infrastructure will collapse. There are also some android security kind of issues involved. The other thing which again I'm working on is this lab grader. So a lot of the CS courses they have database labs, networks lab, the data structures, labs so on so forth. So if you want to do a lab course as part of this platform you do need to provide the requisite infrastructure for that. So most of the time for example in networks lab to set up the environment I myself have a lot of difficulties. I have to break my head sometimes to set it up. So I cannot even expect the outset to set it up and do the lab. But if I can set up the lab and provide a virtualized environment for them to use it kind of makes things a lot easier. So when you do that how do you auto grade their virtualization challenges scalable many people are using it some networking issues using a low band link. So what do you do about it? So that's another project I'm also looking into. With Aakash tablets and smartphones taking over so there's the energy issues especially if you're using your Aakash tablets to watch my videos or other course videos they drain your battery within 20 minutes I think your battery is gone. So what is it that you can do in terms of energy optimization you can do some traffic shaping you design the web page also can play a lot important role in draining the battery and so on. So there's some work that's happening in the space as well. So with that I conclude and we can take up any questions questions, issues, expectations before they come to the class or do you ask them to do at the beginning of the class? Summary that model I didn't use. So I think you're referring to that watch-summary question kind of a model. That's not in my mind because they're not sitting but my concern is that there are 32 students come for tutorial potentially they could have 32 issues. Yes. How could how do you how do you play alpha and I thought it is better to organize them before they come to the class they give you the feedback or the issues you can classify them and then only cover those. So did you do that? As I said last semester was so busy I didn't even have the time to think about anything other than the things. So my tutorials didn't work out like the way I wanted them to work out. I mean it wasn't so the outside the platform wise I was very happy with the way things have worked out. Tutorial wise I couldn't give enough thought to how to organize the tutorials because I was just swamped with too many things. So the way I organized the tutorial is I didn't do anything. So what I've done is I connected with then I had an open clarification session. So whatever questions they had they watched the videos and then I used to pose my own challenge to them. So I used to say okay you learn to go for all this apply it in the setting. Discuss among yourself figure out whatever is this one. Then I just pick on a few people ask them to tell what is it that they found out and then kind of correct. Sometimes they come up with the right answer sometimes they don't. So it led to that kind of a discussion. So I didn't really organize my coming. Because I gave a thought. He was suggesting that this that you give all your audio video before and then collect the material. So I tried it. It was not a video I just gave him notes. I just asked him to read. I gave him a week's time and by Sunday afternoon you emailed your questions and I think it is the 10% who are very much interested because I see it is only those students who ask questions. Multiple questions came from those students and those are only 4 or 5. This class is 120. So that's what I am saying. You make 20% weightage or 30% for the preparedness when you will see people putting in the effort. I mean 10% So you are asking some questions but this also should be the last question is when you started this 2009? No, no I did it just last No It was your first time. So at the beginning of semester you found that few students actually were prepared for the tutorials but as you moved along towards the end after the mix-up you found most students actually watching the tutorials and coming to the class. So what happened was originally I didn't insist there was no quiz as part of the tutorials so I just used to clarify the questions and initiate some discussion on some problem. So what I found is many people were not watching the videos. So then I introduced the quiz and I said I will account for 10% of your grade. So for 20 minutes it is 2 days worth and then if they don't watch So they do it's not that they don't watch they do watch right before the periodic quiz so that happens every 3 weeks one thing comes but tutorials they don't come and that as I said ruins the tutorial because you are asking something which they should know for the discussion. So that was a major problem which as I said I assigned 10% so it improved I wouldn't say significantly improved but it definitely improved they were at least over 30 to 10-15 who would watch it now for the 10% So how many months did they get in their quiz in that preparing this quiz? So again it's a range a good member I mean I have to really look at it but I do think they would have gotten I mean I think the final marks out of 10 would have ranged from maybe 5 to 10 something like that because these are very these are recall very simple if you watch the answer is there as part of the slides in fact you just need to have gone through the slides or gone through the thing it's just that recall So probably 75% maybe coming to class having watched is that I would say more like 50% the tutorials having watched or even less 40% So Abhiram, the average for the quizzes for the minor course I am conducting I have up to maybe that's 20% I increased the average first quiz was 3.6 second quiz was 3.3 third quiz was out of 5 but it's roughly about 50 to 60% average I have an interpretation they are not wasted and of course you can object to my interpretation so one possible interpretation of a positive is that the students are practicing a unique process which normally in the traditional kind of and that may in some sense explain the the issue then is normally in the past people used to try and do this to assignments solve these problems and over a period of time basically for example you are looking at the number of quizzes they have so let me be clear the ones they have done as part of the online web thing is not counted towards grade at all so that's just healthy competition as well as preparedness for the exam they have done it solely out of their own thing because it doesn't count towards grade basically you are learning a lot of honest issues then so the ones that are part of the plan may be some kind of work is already taking place which would be of course good in the sense of the numbers that you actually showed maybe 3 spoons have worked together so much and my survey although they don't get any weightage for it I did a survey so my survey also questioned them how many are working in groups that percentage is very small majority have worked only individually group is maybe 5-6 people or at most 10 one last question so solving more questions more problems it would be better it could also be that the style of asking the problems of questions in IIT setting it is very handy and students have this know-how say ok this professor ask this kind of questions so maybe one way of looking at the effect in this community is to pose the questions so that familiarly with your style of asking questions and they are getting more familiar with it they will not give them an advantage and then you will get better see what kind of quality you are improving sure I think that's a very valid in fact the idea was also that see now that in networks at least the material is there I am hoping going forward Bhaskar is going to use it and they can put in their own spin to whatever the material is there so to make things so you can collect that kind of statistics based on their experiences instead of the very video script for this purpose hope you were to record your own class in the language and test them similarly in the language and instead of the tutorial you get a blend of maybe somewhat advanced class in this kind of that way would it make a difference see there are many factors forget about you do the traditional style of teaching but whatever practice problems I have put let me say I put all the practice problems on this platform and where they get a lot of practice would it have improved they are learning very likely where you are not even talking about video based stuff it's just make them practice practices maybe whatever whether you made them practice practice and it improves your score so that is a contributor so in order to really distinguish which is actually contributing to supposedly that is the single statistic of what can you make out of it what is contributing to the learning one really doesn't know but I personally didn't want the general regular recording and all because I wanted the videos to be very interactive so that socrates model whatever you follow where you ask a lot of questions you direct them, guide them that you cannot really capture like in a regular traditional class I mean you can pop up but it is a retrofit it doesn't go smoothly and things like that so if you want to do it in that as I said it's like writing a textbook more or less but it may increase some marginally I mean the learning may increase to some extent, we use the platform use the recorded videos but populate it with a lot of the pieces I mean it's a long term thing someone has to do the analysis to figure out which is contributing best to their work so this is fantastic I appreciate that I sense a bit of anguish in this tutorial but I wish it would flip that's because we are minus-minus that's not doing this tutorial thing I mean when you say that the student is too much wait what would go wrong with just any individual and say it's pure responsibility to watch these things I've done my 20 minutes two days in a situation you come take the exam I mean how much are we going to go after this student and force them to pop up two days, two days, two days wait is this the right thing to do no students may have questions they may have to clarify you need that person but most of her you cannot just leave everything online and ask them there's a cloud of questioning which happens during the tutorial I'm not saying that it doesn't happen don't worry about the questions if they are not willing to watch then we don't get to ask questions don't try them come to tutorial ask your questions I'll answer them don't do anything extra don't really sense the instructor having a little disappointment about this tutorial that's the tone I got from you no disappointment I did this it's their problem yeah finally yeah that's how I see so it is true that you could have a tutorial where you come if you have any questions I'll clarify other than that in fact I don't do anything extra so that way it can also be run in the course but at the same time I think it's a personal thing I get a kick out of discussing can you put some challenging things some discussion and ensuring whatever and try to make them think about new dimensions and something which I think it's just a personal thing to see whether what they have learned they are able to apply in a different city just for I mean they don't have to show it they can show it demonstrated as part of the exam itself but it's just that I like the process of the discussion something which I wanted to see yeah there it probably doesn't have any only clarification is enough you don't really need to do the extra because for them anyway the increase in quality is going to be substantial that this may be just incremental in terms of yeah I mean as I said research if you are really trying to motivate them to think out of the box research kind of a thing then I agree all this we just want them to know something they apply the concept in a regular setting provide some skill set so that they can be as network administrators or understand some aspect of network so that they can debug some networking problem that level of the thing you want to do then you need a different design maybe the content is the same but how we do the list of the stuff maybe you focus more on the hands on the lab portion of it two questions from one from perspective and another from instance in IIT all the students stay in hostel and the networks speed is good but in rest of the college students it's not totally all students stay in hostel so there is one problem is networking everybody doesn't have the network so your model is way best so student needs to log in and they need to watch the video in the evening watching video they actually answer the question so let me answer that see one is so my videos again right now I have done high definition but you could scale down the videos it's not going to take up too much and lot of these universities do have very high speed internet connection and I think most of I mean good universities within India their networking infrastructure is not bad at all it's pretty good but if it really comes to it there is nothing stopping from this is just a server that we are posting here you get into some kind of agreement with the university just take the server put it locally there will be a local area of agreement may I just add to what you said this model is going to be scaled up now in the future offering for the regular students will happen in the next academy there in the pilot you may restrict in numbers about 1 to 2 lakh students only all out of those colleges so I have put around the country in terms of institutions and so on you are very right that many students who come from outside do not necessarily have a internet connection and curiously what these colleges are really able to do is to reserve specific slots after the regular class in the classroom for such groups to view video in groups and what we have decided to do is while there will be a center server here we will be pre loading the contacts particularly the review and the others so that in the local area network all the students taking that course from a particular college that is one way or another and I would like to also comment on the tutorial Sharad when you are basically valid when you do it this way everything can be done online anyway that is what the books do but the books have very poor completion rate and the books are not known to have improved a lot all the studies have been done over large data have consistently said that a branded model is a good one there is a lot of material on it but a face to face interaction the standard philosophy is that in any group setting if one student asks a question everybody has benefits in that whereas in an individual setting the friend may ask a question on a forum but it is not necessary that all students will see that answer and the teacher can select which questions to answer secondly a face to face thing gives you a facility to address distinct components is here so on for example I used to handle these distinct components differently very poor students I will run a special lecture after dinner and for the smarter students like I will be under water I will call these bunch of 20 people on a Sunday or something and challenge them with other problems now that you cannot do in a normal class but in a flip classroom model permits you to sort of distribute without making it clear there is both the challenging to some people and helping the other people but because your interaction time increases so that is the model we are planning and for the lakhs of students the plan is that the regular teachers who teach that subject will conduct the tutorials but they will be trained for 4 weeks before they conduct such tutorials so telling them the particular problems telling them how to answer questions see the problem as we rightly said the teachers are so weak in many of these colleges one question that has asked when I proposed this trip model in the last workshop they said sir excuse me but we have never solved IIT style problems in our life so if Udaya Gaya doesn't take you there is no dynamic problem they themselves have never seen in their life how they will understand in most colleges the average student quality is better than the average teacher but that is a problem and we propose to try and see that basically what we are saying is a trip classroom model will anyway do something better for the students than what is happening to them today in the larger world even in IIT the observation is that this model has resulted in something better it is said that Karnan has been running a trip classroom model 4 times this time this time he uses aakash tablets to conduct online visits he started conducting the quiz in the beginning to just ensure that they are studying something but now he runs three or four visits in a class which is possible now because you have an instant feedback so I think this is a very exciting thing for possibly making a positive thing and the long lessons are in that class using your model of free record is there it just doesn't work incidentally there is a major effort now with lot of funding to revisit each and every NPTEL course each and every NPTEL lecture to break it into modules interspersed with these kind of questions precisely her model is being adopted by the nation lot of funding is being given with request made to either the same teachers who recorded lectures are there or other teachers to help interspersed so that effort will start probably from the so we have divided 96 students into three groups of the revolution is it a positive pattern for all or it's just a random institution so how was how do you think the interaction between the students among the students was it good so I mean I have done I didn't explain I did one more thing which was you know the Harry Potter the house where you earn points for your house there is a sense of pride and so on so within a tutorial as I said 32 students let's say 4 houses or 3 houses they belong to different houses between the things there is also representation and when you do this discussion whatever you assign house points like if someone comes up with an excellent answer then his house wins so that was I mean I tried following that model but as I said this semester my tutorials need a revisit I mean I have to going forward because I was focusing so much on the content generation I really couldn't devote as much bandwidth for the tutorial thing so next offering I think is when I need to address the tutorial part in its completion so right now I just divided the class into randomly into the things I assigned them to groups and there was some kind of competition among them to earn house points you are going to have some groups which you call houses so as I've known how good is interaction among students so again it's a function of so if they are well prepared based on the tutorial whatever videos they have watched there is a lot where people are coming up with something they do discuss and contribute something meaning to me but if they don't watch as I said it just results because they have no idea what question they are asking and so on and so forth but more I mean Sridhar has experimented with this in the think, share, model he has found I think he has one in a one a good amount of interaction I mean it has to be handled with care in other words that interactiveness is very important before they come they should have watched then I think since they have watched if you throw exciting material at them I think they do discuss and do some things especially if you have this house scheme to to participate then they do tend to do but it's not as I said I am going back to share Sridhar is going to try this there are a lot of teachers in the training program so there is think, share there are about 40 teachers in each remote center there will be over 50 to 300 more so we will have first time to go back on how this will work with grownups but knowing that majority of teachers are just passed out students from last year the experience should be notable in one way or the other way but the main thing is that they will accept the final grade from this individual as their individual so students have to do that sir from the instructor point of view just like there are so many quizzes so that will increase the overlap what you are talking instructor that may which quizzes all the quizzes once you put as a content developer so the tutorial class that I know I mean as I said it's a tutorial you can have a discussion you are just it's not like a practice problem kind of a thing I mean you are told that some concept and then you extend the concept in some other city how would you do it it's just a discussion I mean it's not and it happens only once every week it's not that you have to come up with so many more there will be a burden of correcting the answers there are no questions during the tutorials which are marked towards the grade no actually no so there is a quiz which is a recall quiz based on that practice carries like in a tutorial there will be like what I mean simple questions online kind of a thing so right now grading then for 100 hardly takes 3 hours so the TA's do it it's not like a in IIT there is a facility of TA but rest of the I mean no that is precisely why the TA exam or whatever may help in other models most of the evaluation will be online you can as I said put the same things what is it called online and also do it it's very easy to say as well as grade does that limit the kind of questions that can be asked recall quizzes recall quizzes are so easy to set otherwise the question that you can embed in the tutorial things are videos isn't that very limited I mean close definite questions online as I said that one is good so in fact in lot of the quizzes within the video where I am trying to divert I will just say if you were to design how would you go about it it's more like a pause point pause at this you think they come up with a design there is no grading or everything happening it's just pause point for them to think how would they solve this problem because you can't grade so that both all to the extent possible if it's possible to capture it's a fixed answer or an artificial choice otherwise I give it as a descriptive thing where it's just a pause point for them to think about and then I explain then they compare their own thing with what my explanation is I have two questions first question about tutorials once again you mentioned that your tutorial places carry 10% review places carry 40% 100% and 100% carry 50% so all your evaluation is synchronized mode the students have to come and write it whether they are ready or not so regarding the tutorials with a patch of 30 was attendance in some way compulsory or made compulsory because of the quizzes so it's an indirect thing what was the result where the students all reporting on the tutorials so majority did so you don't have to set up make up no I told them you missed so all the quizzes all the testing was taken by everybody no not everybody so they do miss so it's 10% how much cut you what I mean maybe I can give this to three quizzes because whatever it is but say you give the quiz in the beginning answer to be called in the between sort of linger with that quiz no no five minutes give collect the papers so no discussion of those questions in the quiz as I said it's a very simple level no discussion so you will discuss material which you were interested in or questions have been posed by students early things like so some kind of similar as learning if I may call you for seeking quiz whether the schools were up to it or not these folks said something and they had to take it so that was the question second question I'll start with a remark because it's here so I keep sort of switching between one is IIT here I will bother about the IIT schools the rest of the country the second is the other way I don't bother about the IIT students because they are not paying attention they are not going to engineering there are other students outside who are not getting good teachers who are willing to go into engineering to go into engineering so I keep flipping between these things but I have the IIT so they say there is one issue that nobody wants to think about because they think it is impossible we say our students are special I am not just thinking of V-tex I am also thinking of N-plex most of the time we are worried about V-tex but it is slightly worried more about N-plex too many of us are worried our students are special can't we expect something more out of them that this business of you know mismatch between teachers and classroom timings in other words it is a question of which is first is motivation first or is discipline first it is not clear that you know motivation can be built up without some discipline you have to start attending a lecture or start looking at a video you have to bring yourself up to it then only you could get interested we are not talking about discipline at all we have simply resigned ourselves to the back then the students may not get up on time so if it is 8.30 class they will not come but if it is 8.30 tutorial they may come things like that so I can't imagine the whole semester class were being filled I don't know whether it will work because the students will organize their time in some way in fact to a large discipline in some way it is not self discipline it is some amount of force discipline which we know we think that they are known ups and then in some other places we say they are not known ups so I think this thing as I said there is something it varies by everyone everyone requires external orders I have a question I missed the course number actually is it a course no this is a compulsory course it is a third year compulsory course with an article so everyone is supposed to take it because my next question was based on this question had it been an elective how effective would it be what percentage of course drop would you think was my question so actually I mean I don't know if it really answers but I wanted to mention so CS34 in minors which Vaskar is doing based on this model apparently again you have to correct me even if you don't do well within this course the marks won't appear in the CTI so the way students will of course where the marks count towards the CTI versus where they won't count towards CTI there is a difference because in my class whatever I have seen I have seen again it could be a batch effect also but a lot of people in the minors are also very good students it's not that they are not there CTIs and stuff I have seen majority I was quite happy with this batch more so maybe because periodic quizzes or whatever it is but they did compensate it by watching the videos or whatever it is and overall performance as I said I have not seen in my final no one has scored full but this batch at least two people have scored full and I accept similarly overall I felt that they have done better than the previous batches but I don't know if it was a batch but the same thing I am not seeing in this figure people are taking it easier if you are not doing too well you can drop it it doesn't come through so that's precisely the every point he is making yeah I mean there are too many things wrong in the system in a way that I can speed the terrible motivation levels whatever it is so let me just say that this is we just take one last question and then those who are officially ended and those who want to continue can continue so we have there are lots of novel things in this but with this word if you instead of sending two days to prepare and have videos you just made a manuscript of it in other words you put out a text document which contains the material that you have chosen last very important question that you put in this does that work how well does that work so the reason I ask this is in many many courses all the way from first year to post graduate courses I feel that people don't really follow any testing at all so might say like go and read from this but I am going to teach in my own way so if I am going to teach in my own way does it really matter whether I am going to does my video experience matter is my text experience good enough this is the question you you know the answer actually the voice which is most important voice which is most important there has been statistically good so I will tell you this is a slight shock to me because I thought that my third eye is what people like if you look at Khan Academy lectures most of all are without any person most of all lectures that we are making are also without personal media so what I am planning for example is to have a 30 second one minute clip at the beginning but rest of all as if I am teaching but I am not seeing so I develop programs for example using verse 5 rather than presently put whatever so there is a transcript of the audience and that transcript is a traditional but the fact of like without a person people watch it more than they read every class is exactly the same way that there is a textbook people do not read a textbook unless if I but there are no but all their experiences developed 10 marks so the tricky is not audio is important audio is important in the context that all the lectures are interactive in fact 20 minutes lectures are not permitted otherwise you have 9 minutes or less they have done a study over 1 lakh participants 100,000 participants and any video larger than 9 minutes is watched for less than 9 minutes 20 minute and 40 minute videos are watched for 6 minutes only that is fine so I think the carrot of the cinema is really yeah officially this is an exciting topic but can I just make one point I should make my mind I think the point is you know it's not just the point is what she has has a texture material as well as voice no so the carrot of the cinema is just a good idea I got if you were actually somewhat better the voice was the next yeah and the carrot is the next I mean how much that quantification is difficult but that said when you talk you know there is lot of modulation so for example stress certain point I mean there is something to be said about using the modulation to contain information so sometimes when you do and sometimes and also I mean I get something which I have tried doing not really tried I also try to keep it like in other words throw in a humor here once just so that you know it's when you are doing something very serious you just want them to relax feel some such thing all those things can be played as part of the video how much additional improvement they do I mean maybe I don't know without any doubt it's not just the modulation the point is that the sensory model of the brain works there are some independent challenges one is a visual challenge and one is a texture challenge so if I can explain to you I am going to take one okay I mean it has concluded but I will be around for another 5 minutes you got a big hand