 Welcome back, I hope you had a good two day labs. Today's schedule is as follows, it is very similar to yesterday's schedule. So there is a talk about flipping the classroom, experiences, etcetera. So this one is a talk on how I have used the Bodhi tree platform in running an undergrad which course here at IIT Bombay. So what are my experiences in doing that? After that talk I will be available for some question and answer session specific to that particular talk. Again there is a, we will break for tea and we will reassemble. So again I will get into the details of the next lab in the afternoon and we will also do some course content clarification just like yesterday and afternoon is the lab. So right now we will go ahead with the talk. A very good morning to all. So the title is flipping the classroom, I will share with you my experiences in doing this experiment. This I did last semester within here at IIT Bombay. So this was an undergrad class on computer networks, it had about 100 students. So I had done it in a flipped classroom mode. So let's get into it. So before I get into let me also make the statement very clear. So this talk is based on my experience as a teacher at IIT Bombay. It's very much conceivable that the kind of problems we face is very different from the kind of problems you face as teachers. But nonetheless I think it's good to understand each other's problems. So I'll, so there may be, so when I make some statements you may be wondering where am I coming from. Probably because I'm coming from an IITB perspective. You can add on to your perspectives as we go ahead. So as teachers we all grumble and groan when it comes to teaching. So let me start out with the instructor's version. So I think maybe some of you have faced it, the enthusiastic teachers, you spend a lot of time thinking about how to explain a concept, you spend a lot of time maybe making an animation out of it or PowerPoint presentation out of it, whatever it is. You put in effort and then you walk to the classroom with a spring in your step saying okay I'm going to tell the students this outstanding thing I've kind of put together. And then what do you see in the classroom? Well at least at IIT Bombay out of the 100 students who are there, so are registered in the class about 30 are there in the classroom. The remaining 70 you don't know what's happening with them and it just totally kills your motivation. Like you come there deflated, like you just tell something, you lose your enthusiasm, you somehow tell something and come back. Now you may have learnt a lesson, maybe this is because at IIT Bombay the attendance is not compulsory. So what happens is it's optional who people you assume they are responsible, I mean their certain past, certain age, so they should take responsibility and come. So then we decided okay let's make attendance compulsory. So again you do the same, you go to the classroom, attendance is compulsory and then you expect the entire class to be full, yeah the entire class is full. But maybe 50 are sleeping, the others are like dozing off looking at you as if they are like zonked out they have no idea what's happening in the classroom. Again totally kills your spirit. So this is one of my, I mean I have experienced this as an instructor. So that is one of my complaints when it comes to tackling the students, there are few other complaints as well. Let me spend just maybe few minutes, two, three minutes, are there any complaints from your side when it comes to tackling, I mean taking classes from the, I'll just take two comments apart from what I mentioned, what are your groans and grumbles. I am Neeraj, I am taking final classes in my college and the main problem which I find out with the students, every new day new batch comes. So whatever I teach the previous day, there's no link for the next day. So I need to teach the same thing again. So I introduce introduction topic at this five times. Thank you. Yeah, it's an attendance related issues. Yeah. Okay. One last, anyone else? I found some student doing some other assignment for lab or like that. Yeah, that is also very common at IIT Mumbai, especially at this project times or exam they have some quiz and some other they'll just come. And because attendance is compulsory or whatever they'll sit, they'll work on something else they're totally not clued in as to what is happening. So that's all from the instructor's perspective. I mean you have to also take students viewpoint also, I mean why is it that they are not able to motivate themselves or what is happening. So again from IITB perspective, one of the major complaints is the class is at 8.30, 8.30 in the morning. I mean what are they thinking, what is the faculty thinking that conduct a class at 8.30. I mean it's a very weird, I mean my daughter who is just five years old she goes to school at 7.30. I mean and here are like these grownups complaining about a class that starts at 8.30. But I mean I can understand I was also going through that phase during that thing. You do have late hours, like you do sleep at 2 in the night or 12 in the night or whatever it is for whatever reason and then they just cannot get up in the morning. And 8.30 classes are the worst sufferers, like I mean you have a class at 8.30 you are assured that majority will not show up because it's too early for them, 8.30 is the equivalent of 5.30 in their schedule. So it's something that they kind of complain about, their timings just don't suit. And it's not easy because there's so many classes, so many classrooms constraints. Some classes are bound to happen, but majority of the undergrad courses have a class at 8.30. So that's one major complaint that they have. Any other, like I mean I'll tell more things but from your perspective again, three comments, any comes, things from the student perspective you feel. Everything is available online and we have books everything, why should attend your class? It's true. So they have this thing of, I mean I don't know, it's a displaced confidence or whatever that they feel, yeah, the textbook is there, syllabus comes from the textbook. So what's the reason for attending the class? Very valid. Any others? What is that we teach, the equations? It's very different from what the visualization is and by the time we teach all the theories in the semester, the time ends and we're not able to show them practically how things are happening. So the student gets very bored, like seeing the equation is different and how the paintbrush works in graphics becomes different. So there's a major difference in the theories and actually the student perceives. So what this kind of boils down to is the content that we present is somehow not motivating for the students. So that is also, it's such a boring class, that's how they talk, like such a boring class, why should we even go, we don't want to go. So that's another major complaint. Any one more? If the subjects are theoretical subjects, it is very difficult to motivate them, but if the subject is a numerical kind of subject, then may they take a little bit of interest and taking a one-hour theory class, it's not only boring from the perspective of a student, but as well as for teachers. Yeah, actually, so let me. So as we saw, there are definitely valid viewpoints from both sides. The ground reality is demotivated students. Somehow the students of late in a class, I think it's probably true across many engineering colleges, including IIT, Bombay, including IITs, is in a class of 100, I think the people who are motivated will be under 20. I mean, these are motivated enough that 830 class also they'll show up, which is like a big, it's an achievement for them. So there are very few motivated students. Majority are, they're interested provided you do something about it. Of course, there'll also be another 20 who no matter what you do, they have given up, they don't want to learn, so you really cannot do much about them. But there's a good chunk of maybe around 60 students who if you do the right thing, maybe you can make them interested in the subject. Another issue that we face is this large class sizes. Again, as I said, we deal with 100 students in an undergrad class. How is it other places similar? 60 around. So in fact, there is a thing of increasing the strength from 100 to 120 here. So when it hits numbers like hundreds, it becomes a very difficult thing to engage the students. So the problems you see when you have large class sizes are very difficult to get personalized attention. And lot of this motivation comes with some kind of personalized attention. Then they feel more motivated. I mean, if there's someone talking to them telling others that maybe they'll pay more attention. So in a classroom setting, when you ask questions, only a few get to answer the questions. The rest of it are like passive observers. And some of these active students just hog the attention of the teacher. I mean, they are forever raising their hand and no one else gets a chance. And others also probably are not interested in taking that chance. And as an instructor, I also find that in a classroom, when I'm teaching in a traditional classroom setting, I cannot give enough think time for the weaker students. So what happens is I ask a question. I just give maybe because I have to cover certain amount of stuff in a one hour class. I ask some question, then I kind of give maybe two minutes. Maybe the smarter students can figure out within two minutes. But there are quite a few students who will take time. I mean, there is nothing wrong in taking time also. I mean, instant solutions don't have to be instantaneous. And you really cannot give them this think time for the students to solve this problem because you have to finish the material so you kind of hurry up. And I personally like when I teach, which probably you have seen also as part of the videos, is I like to ask a lot of questions. So this model is called the Socrates model. So what it does is before you explain a concept, you ask questions that show why is it that, I mean, the solution approach you're coming to this particular concept, you re ask quite a lot of questions. Why didn't we do it this way? Why did we do it that way? Finally, then is when you appreciate why the solution is the way it is. Because there are a lot of other questions people have asked when designing protocols, for example, there are many ways of designing. Why didn't we go along that route? Why did we choose this specific route? So there are a lot of questions to ask. And these kind of questions, if I start asking in the classroom, just to cover maybe five minutes of content, it'll take up one hour of a lecture. Because once you start asking questions, you have to take in feedback from many students. And then there is some discussion or the other. So you really cannot ask too many questions. And this problem is compounded in a large class setting. And there is a big issue of poor attendance, naturally, if. So here, again, I'm assuming that the teacher is enthusiastic. He has prepared very good material. I know theory, it is tough. But let's still assume that the material is interesting, exciting. But even with this, you still see these problems. It's not that if the material is great, people will start coming. To some extent, yeah, they probably show more interest. But at IITB, again, I know many teachers who are really good teachers. And the kind of content they present also is pretty good. But attendance is still a very difficult issue here. So even if definitely having good lectures, good content is important. But there is this attendance problem also. And from the student's perspective, one is, again, this waking hours and class timings is a big issue. Another issue they face is once a lecture or two is skipped for whatever reason. Like I think he was also mentioning that if you don't attend the first lecture and you want to attend the second lecture on, you cannot follow. Because it's a continuation of what happened first. And invariably, everyone, even the good students, tend to bunk one or two classes. And once they're bunk, that's it, over. They cannot follow what is happening from that point on. So the class starts out, let's say, with 100 students. By mid-sum, the thing drops down to some 50 students. Progressively, it becomes 20 students. Finally, you have hardly 20 students attending your lecture. I think this is one of the contributing factors. So the question to answer is, how should one conduct classes such that it will facilitate what I call effective learning? So finally, you want students to learn something from your class. So what should we do about it? And specifically deal with these two issues of large classes, size, and poor attendance. The solution, some people have experimented with this. This is from Wikipedia. It's called a flipped classroom. So what is a flipped classroom? They basically inverse the traditional teaching method. So in a traditional classroom, this corresponds to here, where you have the lecture. And then homework is there, where you go home, do some practice problems, whatever. That's how typical traditional classrooms work. What the flipped classroom does is you watch the lecture online, outside class hours. But when you come to the class, we'll do an activity. It could be solving some practice problem or doing some discussion or doing some lab, whatever it is. So you're doing a face-to-face interaction with the teacher, not in understanding the, or rather getting to know the concept, but in doing an activity related to the concept whereby you strengthen your understanding of the particular concept. So that is the flipped classroom setting. So what I'm going to show is a case study of an UG course, CS348, which is the computer networks that I have conducted. So I did it in this flipped classroom setting. So I'll get into the details of how I conducted the class, what my experiences have been. So here is the outline of the talk going forward. I will specify what my objectives were, how I went about implementing them. I'll demo actually is probably not needed since you have already seen the platform. I'll, I conducted a detailed student survey to get the student's reaction to the flipped classroom model. I'll also provide my own insight and what the ultimate vision of this setup is. So these were my objectives. So when you're trying to replace a traditional classroom with this online thing, you shouldn't lose out on the benefits that a traditional classroom offers. Traditional classroom, there's a reason why we have a face-to-face interaction with instructor. So it should mimic this in-classroom settings as much as possible. And what happens in a classroom? There's a lot of interaction. So you need to capture that interaction. Without the interaction is just going to be very dull. And I think I've heard from some of you also personally that there's NPTEL and other videos. They're kind of monotonous. You just, it's just a broadcast. And when you don't have interaction, it, I mean, if you're really motivated, fine, you can still follow the lectures and do. But for an average student, it's kind of difficult for them to sustain viewing when there is no interaction. Interaction is like putting you on your toes. You have to follow what is happening so that you can answer or follow the next portion. And some question is asked, you just don't want to keep clicking wrong answers and going ahead. So interaction is very important. So how to make interactive basically by embedding questions, activities as part of video. Some things, I mean, as I said, this is not a fully developed platform. So you may not have seen some of this. You can also embed, like for example, as a video is going, you can put a pause point, say, do a demo, do a lab exercise and come back. So I explain, let's say, ARP. And right at that point, when I'm explaining, okay, go open Wireshark, look at the ARP packet header, come back and then follow. So these kind of activities can also be embedded as part of the video. And you could also provide feedback. So for example, there were a question asked and there were some choices and the student clicked choice one. And let's say choice one is wrong. You can actually tell the student, I mean, you are thinking, you click this choice because you thought in this fashion and that is why what you have thought is wrong. Again, this feedback feature has not been incorporated, so maybe you didn't see it. But you can potentially provide feedback for why the student has selected that particular thing and why he is wrong. Or if he's chosen it correctly, you can give like a pat on his back, oh, great job, you have done really well. So whatever, so this feedback is also quite important to the student. So that makes it feel as if there is an instructor kind of doing a personalized tutoring for him. So again, another thing that is important is the student should be able to pause, think, understand, answer and proceed. So you want to give the student enough time such that he can do this particular activity. Because as I said, I was mentioning you weaker students specifically need longer time. So that also should be enabled. And naturally, again, someone mentioned smartphone generation, right? Their attention spans are like really, really small. So they get distracted very easily. So you need to retain students' focus. And for this, the material has to be pretty good. You have to make the material exciting. And it's also important that the video should be short and sweet under 20 minutes. If I, in fact, there was some survey done. In fact, 20 minutes is itself is a larger stretch. I think EDX or Coursera, I don't know which of them has specified that anything about nine minutes, I think people don't watch. So there is some magic number. I've tended to keep, in fact, originally, I did create some about 20, 25, 30. Then a lot of complaints. They said, no, no, we can't watch 30 minutes videos. Keep the videos pretty short. So what happens is, especially when you're doing something online, I myself am guilty of this. Like, you watch because you're on a browser. You keep clicking, checking your Gmail, checking this thing, coming back, and then kind of continue. So it's kind of difficult to retain that focus. So you really need to keep it in smaller chunks. Another objective, which I personally had, is to provide complete learning. In other words, all material related to a concept, be it videos, slides, reference materials, practice problems, have to be in one place. So what happens, again, in a traditional classroom setting, is you cover, at times, you cover many concepts within that one hour. And then students go back, work on the problems some other time. And then their feedback for the problems that worked maybe come after another one week, where you post the solutions. So it doesn't happen all in one go. Ideally, you have to master a concept before moving on to the next concept. That is when your understanding strengthens. So the idea here is to provide complete learning where the study or the practice thing is interspersed with the video time. So both happen hand in hand. So you only move ahead after mastering the concept. So that is also a specific objective I had. So the end result of all this is, what I would call is a multimedia textbook that is hosted on a web platform. So you are familiar with regular textbooks. So this is identical to a regular textbook in organization. But except that instead of text, you now have a multimedia content. So you organize everything into chapters, sections, with all supporting material in one place. So what you have seen to some extent as part of the computer networks course is that multimedia textbook. So this is outside hours. So what happens inside a classroom, this tutorial, you need to provide some kind of personalized attention. So what I have done is I decided to divide the large class into smaller groups. So there were about 100 students. So I divided them roughly into 35 groups of 35. And I had a tutorial session for each small group. So I'm dealing only with about 30, 35 students at any given time. And I use this tutorial to excite and challenge the students. So I used to throw some advanced problem than what they have seen in the videos and ask them to come up with a solution. So a lot of discussion happens in the classroom. And while I'm doing this, I also try to encourage group learning discussions as part of this tutorial. So what we have done is for the three groups, there were three one-hour tutorials. So in other words, each group got about one hour of personalized attention in a week. So basically outside class hours, they are supposed to watch for about five hours in a week, all this video, time study, time, and so on. And then they had one hour tutorial session, so leading to about six hours. So one other thing which I had to do is a recap quiz within the tutorial. This was very important. Originally, I didn't do it. I thought people will come. And I realized that tutorials are most effective only when people come prepared to the tutorial. They should have watched the video. Only then they can do this advanced problem that I have solved. And what was happening is someone not watching the video. So if you had a quiz where they're supposed to, it's a very simple quiz. It's a recap. If you watch the video, you will get full marks in this particular quiz. And this quiz carried some percentage. It carried 10% of their overall grade. So people were motivated enough for this 10% that they used to come and at least watch something before they come. And then we used to do some discussions, clarifications, question and answer, practice problems, so on. So the way this course was graded was there were periodic quizzes which accounted for 50% of the grade. By the way, this period, so quizzes were made up of this weekly tutorial quizzes, which happen every week. That is 10%. And there was a review quiz, which is a regular exam, like a mid-sum or a final, which happens in a big classroom. Students are spread about. You give them a paper. They write it. So every three weeks, I had to conduct one such quiz. So basically what this ensures is students watch the videos on time and stay on top of things. And then there was a mid-sum and final, which accounted for 50% of the grade. So what was the end result of all this? This is what I call PFC learning. So it's a personalized learning. So what happens is it's as if the instructor is talking just to the students. So you're watching a video. There's no one else as if the instructor is talking. And all get to answer the questions. So when you are doing in a classroom, traditional classroom setting, you can only pinpoint, OK, you tell, you tell. You can't ask everyone the question, because it will take forever. So here, on the other hand, everyone is getting to answer the question. And sometimes students have this fear of embarrassment. Oh, if I ask this question, maybe the instructor will think I'm stupid or whatever it is. So they have this fear, which kind of stops them from asking, here there is no fear. It's kind of a personalized. You can do that. And this focus small group tutorials give a lot of personalized attention where it's particularly useful for large classroom settings. And there is a flexible. In fact, this is the most appreciated feature by the students. Students' choice of time, place, and group. So when they want to view the content, they can decide when they want 2 o'clock in the night. Perfectly fine. In fact, majority watched at 2 o'clock in the night, according to them. They can choose wherever they want to watch, hostel rooms. And they can also do a group watching. So come together as a group and watch it. And very important from the learning perspective is the student's pace. So when you are running in a traditional classroom setting, I dictate the pace, because I need to cover this material within this one hour. But when student is watching the same materials, the good student may be able to finish it in 30 minutes, whereas a weak student can take one, one, and half hours. No one is stopping him, because he can pause, rewatch portions, finish the take as much time as he wants to solve the problem before proceeding. And you are ensuring that they do not procrastinate this watching through this periodic quizzes. So they have to stay on top of things. And this point I've already mentioned, each concept is complete. Thereby, you master the concept before going ahead. This also strengthens your learning overall. So here is the interesting aspect of what has happened. So I conducted a survey. So 45 had responded after much coaxing. So the positives that they have conveyed, I think, majority loved the flexibility, convenience, and the freedom of it. I mean, the fact that you don't have to come to class at 8.30 was like a big, big thing for them. So many didn't really, they liked majority who commented positively, commented on this flexibility convenient. And a good number also said that they have watched the video many times. So they watched it, they didn't understand, they went back, they watched it again and again until they understood. So that also people appreciated. They also liked the fact that the tutorials were much smaller so that there was a lot more personalized attention. They also liked the fact that these practice problems had given instant feedback. So when they saw something, many of them, again, in a traditional classroom setting, you post some homework, people solve it, and you post the answers after one week or two weeks or whatever. And by that time, the student, the problem is out of his mind. He has no idea whether what he did, what the solution is, so on, so forth. So what they liked here is you solve it, you get feedback on what you did wrong, what you did correct, everything about it kind of instantaneously. This feature also they liked. And another thing which I did as part of the thing is the scoreboard feature where you get to know how your peers are performing. So this tended to produce a healthy competition among the students. So what I used to do is almost every two, three weeks, the top scorer got a treat. So there was an excitement to be the top scorer so that they can get the treat. In general, it's like a bragging privilege, like, oh, I'm the top scorer. So that's something which also some of them enjoyed. The negatives, so those are all the good things. Naturally, they are, it's not a perfect system. No immediate feedback. I mean, this is a very valid concern. The fact that you're watching a video, you get stuck and you don't know what to do. And unless you understand, you don't really cannot proceed further also. And there is no instructor right there and there to give you feedback. So it is definitely a problem, but there are a few solutions to that which can potentially be addressed through this platform. One is, which I have not implemented it, I would like to implement it, is what I call iWatch. So what you do is the lectures are all there. So just like a classroom, whoever are these people who want this kind of immediate feedback or whatever, we all assemble together in a classroom. They continue to watch the videos, but the instructor is there. So whenever they get stuck, they come to the instructor, get their questions clarified and continued watching. It's not that the instructor is sitting and delivering the lecture again. It's that they are watching on their smartphones or their laptops or whatever it is, but the instructor is around them. So you have, this is a designated study time, you fix one, two hours like that, you are around, whenever they have questions, they come to you and get them clarified. The other thing is, you can always tag portions of videos with less clarity. So as people, for example, many people tag some portion saying, this portion I did not understand, that is an indication to the instructor that some explanation is missing. He may have to redo the video or add some explanation to that. So over time, the videos will become more and more clear such that the students will have less difficulty understanding some of this concept. Another thing which is also very useful is what I call G-Watch, which is encouraged group watching. So you may be stuck, but maybe your friend knows the solution to it. So watch in small groups or four, five. That way, whenever you're stuck, you can discuss with them and through the discussion, maybe the solution will become obvious to you. So that's also something which will facilitate peer-to-peer discussion and better learning. The other negative that some students have complained is this flexibility kills discipline. So in other words, the content is already there. So lectures in a traditional classroom setting happen at designated hours, so you have to attend at those times. So you are kind of more disciplined. Here, there is a lot of flexibility, so you just push things till the end and then things are in a rush. So to some extent, ensuring that you have a weekly periodic quizzes helps. This instructor watch can also help. That way, it's like a regular classroom, except you're watching from your laptop rather than listening to the instructor. So attention deficiency is another big thing. Said two long videos, they tend to browse around instead of watching. So videos have to be short, as I have mentioned, and this G-Watch, I-Watch will also help when they're watching in a group, you can tend to, it's more disciplined because it's a group, they tend to focus better. Well, technology can also help. We have something called lockdown browsers. Basically, you open this content within a lockdown browser until the video finishes, it will not let you quit. So maybe people will not appreciate it unless you shut down the mission and reboot it, you cannot get out of that particular video page. So that's something we can do, but I don't think many people will appreciate it, but the technology is there to do it. And again, another negative is because the content itself, I mean, there are discussions as part of tutorials, but the content itself, majority of it is covered outside class. So this means reduced discussions in a classroom as well as interaction with the instructor. So tutorials have to be very carefully designed to fill this gap. So those are the positives and negatives about the thing. Again, some said it's so boring to watch the video by themselves. Group watching is the way to go. And taking notes is also difficult because the video is information dense and you have to pause the video, make some notes. Again, technology can help here. You can have keyboard shortcuts to pause and play and the transcripts of the video itself, again, this is something we'll make available soon, will be made available so you can mark there itself, whatever you want to do. You can also make available a note taking or an annotation tool. These are all features to be added as we go forward. So that's with the positives and negatives. There also like a survey done on, did you feel the need for the instructor presence while watching videos? As I said, 9% felt very much, 67% felt sometimes and there was a 24% who said, no, we didn't feel the need for instructor at all. Quizzes within video, how many questions did you attempt to solve seriously? Again, some of these numbers, like a good number did solve kind of seriously. So many people like the fact that there were quizzes outside video for practice. So these are the practice problems. This is what helps them score well in their exams. So this feature majority liked. As I said, each 20 minute video is very dense and if you're doing a good job of watching this video, it can take a really long time. So very few people actually could finish the video. Majority took 20 to 30 minutes and there were quite a few who have spent a good amount of time on the video trying to pause, think, answer the questions before proceeding. Well, this is the ultimate question. So your overall score for this new model of teaching in comparison to the traditional model. So the way I've said this three means I love it. Two is like, I like it as one is fine. I'm fine with it. Zero is like, I don't, I'm neutral. Minus one is again the negative scale which says it's okay. I mean, it's, I don't like it that much. I don't like it at all. I just hate it. So minus three is that. So as you can see, there are a few who have not liked it much, but a good number. I mean, if I'm just counting three to one, not even the neutral responses, there were 66% in favor of this model, whereas 28% were against it. So what's my intake after doing this? The first offering, whatever I've done is a step closer to bottlenecks because I've lost so much hair, such a stressful thing conducting this class for the first three, four months because I was also preparing the videos in parallel and it was like meeting a deadline. Like, I don't know how many of you have written papers. Paper deadlines are extremely painful. I mean, they are very stressful because you're working till the last minute and this was like meeting a paper deadline every week and this was like stretched out for an entire semester of close to four months and it really, okay. So each 20 minute video just to prepare is at least two days is like a minimum thing. It easily takes two, three days because you have to prepare slides from scratch because you don't want to get into copyright issues. You have to refer multiple books just because you're doing it once. You want it to be more permanent. You don't want to do a shoddy job. You have to think what you want to convey and then you have to record it and recording itself is a kind of a learning experience, even though I have given lectures so many times and I'm a pretty decent speaker, but still recording does take, like you record, you're not happy with it and you have to rerecord. Then the more rerecordings you do, editing is a nightmare. So overall, it was a very tough. Preparing the content is kind of tough but what's good about it is the following offering is will be a piece of cake. So maybe it's a step closer to fatness because you can just take it easy so once you have everything ready, repeated offerings later will make life a lot easier. Tutorials I feel are very important component of this particular flip classroom model. It's very necessary to pay a lot of attention to it. They need very careful design because you can kill a class if you are not properly designed them. And again, as I mentioned, quiz is very important as part of tutorials. The first 10, 20 minutes is quite spent on the quiz and some of the clarifications and the rest of the time, we should do an activity that excites challenges students and leads to discussion. So things that one can potentially do is you motivate a problem, you discuss challenges and leave the solution approaches to the video material or you can do a hands-on experiment or you could review some challenging problems where whatever they've learned, you're applying it in a new setting so that they get a better understanding of that particular problem. There is this thing called WISC, WSQ model, which some of this flip classroom do. What they do is you're supposed to watch a video, you prepare a summary of it as well as a question specific to that video. And in a classroom, you discuss the summaries and discuss some of the questions that students have prepared. Whatever you do, this tutorials have to be custom fit for a given course type. So for example, the way I do CS348, which is computer networks, may be very different from the way I do, maybe a programming data structures course or a databases course. So it has to be custom fit and across physics course or chemistry course. The biggest challenge to tutorials is unprepared students can ruin tutorials. If they don't know the content, then the tutorial is all about discussion, then not much will happen. So it's very important to ensure that the students come prepared to a tutorial. So you have to assign a good weight. So your grade, some good portion of it, maybe 20, 30% of your grade, that weightage has to be assigned to this preparedness before you come to the tutorial. Otherwise, it's just kills the tutorials. I also, I mean, those were the, maybe the good things. What I personally didn't like in this flip classroom model is the face-to-face interactions are special. I mean, when you convey something interesting, you can see those aha moment from the students where they realize something. Or if you're telling something and they do not understand, you can see the confused looks and then you can re-explain the concepts, can explain the same thing in multiple ways such that they understand it better. So, I mean, tutorials didn't really cut it for me. That face-to-face interaction is something special, which I think many of us really like. That's something, tutorials didn't really, I mean, they did help, but it was only because normally, I interact with them for three hours in a week. Now I was interacting for only one hour. So it's like they cut off two hours of that special interaction bonding which a teacher has with a student. So student performance. So I mean, I did this experiment. Now did I, did the students learn better or they didn't do well? I really cannot tell it with any confidence because all I have is one statistics. And what can you, I mean, if any statistician was here, he'll just who far what I'm saying. Because you can't really make conclusions from one sample, but nonetheless, let me present some results. So Spring 09 and Spring 12 are two offerings of mine, which I've conducted in a traditional, that is face-to-face in a classroom, a lagboard setting. So these are the average marks that the student got out of 100, and these are the maximum marks. And the number of AAs, which I have assigned is in this particular fashion. So as you see, I mean, between Spring to 12, which there is an increase, the number of AAs, of course, they didn't, in fact, they have decreased. The average marks are so increased. And this was when I did, which is the fall 2013, is when I've done this flipped classroom setting, average has increased, maximum marks has increased. Another thing is until now, no one has scored full on my final, because I tend to at least put some question, which is tough. But this time around, which was in fall 2013, the flipped classroom model, two people had scored full in my, which I felt was a good achievement. And the number of AAs was also significantly high. Overall, I did feel that students did well, but I don't know what the cause for it. Maybe the students of this batch were inherently better, or I definitely put in a lot more effort. I've never worked that hard in my life, I would say, in that semester. So I put in a lot of effort, maybe the fact that they had so many practice problems to work on, there all these are contributing factors, or maybe the flipped classroom model worked. I have no idea. I mean, unless you do it sustained for a few years, you don't really get to know. But my gut feeling is that the model did help to some extent based on the scoreboard. Scoreboard told me that, and normally students are very lazy, they don't do assignments. But when you look at the scoreboard, the majority of the class barring three, four people had worked on all the practice problems. So that shows that they must have watched some videos and worked on it. So they were taking it a bit seriously. And from talking with students also, many of them said, the ones who liked the model said, see, we really watched your video many times, many things became clear after that, so on. So I think it did help, but as I said, I can't conclude positively. So the final question after all this is, should I flip or not flip? In other words, should I do a traditional setting or should I go for a flip classroom setting? So let me just pull together a comparison between both of them. What is good about this? What is good about the flipped? Traditional model, it's a fixed timing place, flipped model, flexible timing and place. Focus is an iffy thing in either because here the traditional model is a one hour lecture. You lose focus somewhere in the middle, that's it, it's gone. Flipped model, it's a shot, it's only 10 minutes or 20 minutes videos. During that time, you can stay focused, but then you are doing it online where all this Facebook, Gmail, everything is hanging around YouTube. So it's easy to change your focus. So focus, I don't know, which is better. In a traditional classroom setting, you can only watch the lecture once because the instructor just gives the lecture and goes. Here you can watch it many times, which is helpful. Again, the traditional model, the space is dictated by the instructor because he will say, I will cover this much material in a class, whereas here the pace of learning is dictated by the student. He can take as much time as he wants in watching the video. Traditional model, very few questions you can ask here. You can ask as many questions as you want by interspersing them as part of the video. And again, traditional model, you can target only a few students when you're asking questions. Here you target all students when it comes to them answering the question. The big advantage of traditional model is immediate feedback, whereas that is missing. So what I've shown in the underlined thing are the advantage. As far as learning is concerned, maybe there are flexible timing place, it's a convenience thing. It's not crucial for learning, but as far as learning is concerned, what I have underlined, I think is very important for learning. That way, flipped model has four advantages and traditional model has one advantage, but that immediate feedback is also very important to learning. So one has to wait and accordingly do it. So again, these are my observations. In any given classroom, there'll be 10% very interested, 40% marginally interested, 40% will do something to pass 10% lease border. So for this 10% very interested students, I think traditional classroom is the best bet because you can really excite them, you can coach them, guide them in whatever fashion. But a well-designed flipped classroom may not be bad either for this 10%, but I think from remaining flipped classroom in general is better, both from learning and exam preparedness perspective. Just the fact that they can watch something multiple times and take their own pace, I think tends to be better. So why did I start doing all this? I mean, there were these wheels within wheels. My overall goal was something different. So the goal was to make good video material for Indian engineering students, preferably the entire CSE curriculum. So I started with computer networks. Hopefully in the next few years, one year or whatever, there is a plan of doing it for computer architecture, maybe data structures and some programming kind of software lab kind of a thing. But the overall goal is to expand it to the entire CSE curriculum with a focus on employment skills. It's, see a lot of times when we teach content, especially this is true at IIT, is we try to throw in a research, because we are all researchers on the side. So we tend to make all the concepts slightly difficult. Everything is design oriented because we come from a research background. But outside people, it's really not essential that they understand all this researchy business. For them, if you're doing computer networks, maybe it is with a perspective of getting a job as a network administrator. Or, so for the majority, I'm not talking there will always be exceptions. So the focus, when you prepare this material, by the way, when I prepared CS348, this wasn't the focus. I did it for our students. But in general, the focus should be more on creating more hands-on stuff as part of this so that they can use this hands-on learning when they go for employability purposes. Now, why, I mean, these are some of the questions that may be passing through our head. I mean, there are already so many textbooks. Why this additional multimedia textbook kind of a thing. But the same reason we conduct classes face-to-face is quite important. Now, how is this multimedia textbook differ from, different from the CD, NPTEL? There are a few fundamental differences, which I think are very important. One is the entire content is organized into the sections, chapters, and so on. So NPTEL is just like one video after the other, they have just put it. More importantly, the videos are interactive. This, I think, is extremely important to capture the students, to motivate as well as keep them on their toes, interactive videos. Another thing which I personally felt is important is what I call a playlist feature. Because the engineering colleges across India, they have different syllabus. So what is followed in one engineering college, there'll be only a subset intersection match. So the goal again here is to create content that kind of is a superset of all the syllabus that is out there. So whenever someone is running, what they need to do is an instructor needs to prepare a playlist of the concept that corresponds to their syllabus and use just that for running the particular course. So the instructor has that flexibility. So that's also something which the current interface currently doesn't support, but will soon support. And material alone is not important. In other words, some of these NPTEL, you kind of just went ahead and recorded your lecture. But often when you know that you're going to run this in a flip classroom setting, you have to pay a lot of attention to the execution. I mean, am I explaining it in the right fashion? Will people really, it's like writing a textbook. It's not like just, so for example, when I'm preparing for a lecture to deliver in my class, since I know the material, it's more or less like I may spend half an hour, just go and deliver it. But if someone were to ask me to write a paper based on what I'm going to deliver, I cannot half an hour preparation is not enough for me because I know once I put it on paper, I have to be very careful of the editing syntax. What exactly am I conveying? So I will easily spend few hours to make it right. This is also kind of similar. You really need to pay a lot of attention to the details when you record the video. So there's no different from writing a textbook. So the goal here is to cater to this engineering material where an average student who can get a job, which I have already mentioned. So the provide quality students to Indian industry. So this platform which you have already used, I won't get into too many details, but at a high level, what you have used is the student interface, but there are two other interfaces which have not been exposed to you. One is a content developer interface, which is the people who create the material, like the people who do those videos, quizzes, reference material, who will organize the textbook in the form of a multimedia. So there is an interface for those who will organize the multimedia textbook. And then there is an instructor interface. This is for other college teachers as well as academic industry, also whoever wants to use this material. Their interface will be that they are exposed to what is there as part of the content developers and they can pull the content from this content developers, prepare the playlist, run it, and as part of it they can track student progress, they can facilitate discussions, they can do this in-class tutorials and all. So for this instructor, there is definitely the trust and certification is the instructor responsibility nor the content developers responsibility. So for example, the grades that you assign or the certificate you are giving that someone has finished this material. And all the student does is enroll under an instructor. So there are already plenty of other online platforms like edX, Coursera, so on. Platform-wise I don't think, I mean all these have sophisticated features, so does our platform. I mean, it's not, we started developing this because at that point when we started the development there was no open source. edX was open source later, but by the time we have spent already enough time on the platform that it didn't make sense for us to back off because it was already, and it was also designed keeping in mind this flip classroom setting, whereas edX was not, it serves a general audience. But more than that, I felt that implementation is the key. There's a lot of ground work you have to do. You have to custom fit whatever you are doing for the Indian market. You have to leverage on, you can leverage on existing content from edX, Coursera, but to make it work you have to do a ground work and this is the challenge because there if you say someone from Stanford or MIT saying, okay, there's this engineering college somewhere in remote India, you prepare some content or modify some interface for me, they are not going to do it for you. So there should be someone within India who understands the issues corresponding to the instructors and accordingly redesign the system or make some additional content available or whatever it is. So you really need someone as a local facilitator and I feel ITB or for that matter any of the IITs can play that role. So the goal here is one can develop content that facilities flip classroom settings. You can conduct this kind of teacher training workshop. Question papers can also be set and we could potentially expand our own BTEC MTech program via the distance education. So I'll skip. So the high level thing is cliche, but nonetheless I think we should, a lot of problems in India, education is a big problem. So I think we should work together for a better India. So in this context, definitely your feedback and cooperation I would really appreciate. I mean, this is something which I'm kind of passionate about because I feel of the many things you can do in life. I think you should do things that are impactful. Whereas people are using what you are developing as part of their day-to-day life and education is something I think is very important. So if something you can do in this space, I felt it's a gratifying thing. So that's why I got started here. I'm quite enthusiastic. So if you have any comments, feedback, or any help in whatever I'm more than happy to collaborate and take your feedback and cooperation. So I also wanted to mention a few other things that I'm working on. So one is something called eExam. So what happens is, for example, in a classroom of 100, you want to conduct this tutorial quizzes. And I also mentioned that there are many quizzes you have to conduct just to stay on top. Otherwise, students will slack off. They won't watch the videos. So if there is so much, if there is a class of 100 and you're almost every week you are conducting a quiz, grading becomes a major. I mean, earlier I was preparing the videos, now I have to sit and start grading all the papers. So it's a painful thing. So what we are looking at is an eExam where you conduct proctored exams on smartphone. There are a lot of research challenges in this space. So if there are, BIFI, if you generally use so that people start logging in and pulling the exam paper and pushing their answers, BIFI doesn't work. Because beyond 2030, more logins, BIFI crashes. So there's a lot of issues there. So multicast is something that can potentially be done. And there are also Android systems issue. When you're using a smartphone, things don't work out. You have to custom, you can't say I will hack around the Android driver. Nothing is, it's not easy. I mean, if I say you have to route your Android device to contact this exam, no student will be ready to route their device for taking my exam. So I have to work around within the constraints of the Android system. So there's a lot of research scope also. So there's something that we are working on. There is also a lab support. So the CS348, some of the labs that you're seeing, as well as other labs like as algorithms lab or data structures lab or databases lab. Some of the, in fact, one of, there was this very excellent tool that we use in the CS348 lab called VNUML. It has now advanced to VNX. It's a virtual, it's a user mode Linux that works at the user space. So whatever commands you use in general for setting up of the networks you can use in there. And we demonstrated many aspects of networking like setting the routing table, setting up bridging, many things on that lab. It's a very interesting lab. But I cannot really, I mean, I would have liked to include it as part of some of these labs, but installation of that is a nightmare. I mean, we ourselves have spent weeks to figure out how to install it. So it's not an easy installation. So when you have things like this, even NS2 installation, some people will have difficulty when you don't really know how to go about the installation. So for some of these, it's very important to provide a virtualized environment so that people can log in to the lab exercise and kind of log out. So there's some effort in virtualization and auto grading of lab assignments and security scalability networking issues in the space. And naturally, when you're using smartphones, tablets for classroom settings, their energy consumption is again a big concern. So there is some research going on in the space of how to shape videos, traffic, web video player designs, such that the life of these tablets and smartphones is enhanced. So that's about my talk.