 So, welcome to this second lecture in the series of lectures on flip classroom. So, today Professor Bhaskaran will be sharing his experience about flip classroom method and also about two tools that he has he and Professor Kameshwari from computer science department have developed to facilitate flip classroom model. So, these are one is Bodhi tree. So, this is a software that allows you to create your own lectures which help you to assess the students as you as they progress watching your videos and safe is a platform or is a method is a software tool that helps you to conduct exams. So, this is something which is unique because he has been involved in developing it and simultaneously been deploying it in his teaching. So, this is I think journey for four and half years for him and almost it is a six years five year work together on development of the students. So, I hand over to Professor Bhaskaran. So, since this is a lecture on flip classroom seems like a oxymoron because in flip classroom you never you do not have lectures. So, we will try to keep it interactive and also I have a demo planned in which you will be able to do something hands on. So, you will get a feel for what we have used. Thank you for inviting me to this session and thank you for the introduction also. As Professor Sachin said today I will be talking about two of our tools Bodhi tree and safe both of which have been developed completely in house by our students various student projects M tech students B tech students R and D projects and so on. Bodhi tree and safe have a joint existence they also have an independent existence independent of one another. You will know what I mean by that once we get to learn about them and as you mentioned all this is joint work with Kameshwari and a significant part of Bodhi tree another faculty Varsha Abte another colleague of ours she has also been involved that has been a later addendum to Bodhi tree and the number of students who have worked on this is too many to fit in here. We have been very fortunate in terms of getting very committed students and they are so committed that even after joining job even now there are many students who contribute to this. So, it has been very encouraging that way. So, this is a intro slide if you look at you know how technology has affected various parts of our lives you will see that we think back it has affected all aspects of our life. So, this slide shows you know transportation pre industrialization in early days of industrialization and today it is completely changed as you can imagine. Similarly, communication technology pre industrialization and in the early industrialization days and today is supposed to be the information age although it is fast becoming misinformation age. So, that is how technology has affected how we communicate. However, if you look at technology in teaching it is pretty much remains the same. We had Gurukul in our pre industrialization in our Indian mythology Guru teaching a bunch of students similarly it is nothing much has changed throughout the world actually in terms of teaching use of technology in teaching. In fact, the last slide there shows how things have worsened because of technology you probably cannot see but if you zoom in lot of people will be browsing on their phone and maybe open laptops and so on. So, technology is actually a hindrance to teaching in many ways. There are many colleges which completely ban cell phones and laptops in classrooms many teachers do not like these kind of things coming into their classroom because it is a distraction it comes in the way of learning. Similarly, exam technology has not also changed it is pretty much remained the same paper pen. So, in this context we have looked at how technology cannot solve all problems but technology can at least help in addressing some of the teaching problems especially in terms of scale which we all face in our classrooms today. Our department core course class sizes have become 140 and probably with the additional expected increase in numbers it might touch 150, 160 beyond a point it really does not matter. I have stopped trying to remember my students names at one point I used to try until about class size of 90 I had reasonable success beyond that I just gave up. So, I will talk about Bodhi tree at a high level which is a multimedia book platform it is a interactive multimedia book and it is a platform constructed to handle flip classes specifically. And I will talk about our experience and some evaluation also of the system. SAVE stands for smart authenticated fast exams which is the use of smartphone app for conducting quizzes and exams. So, that also I will talk about little bit about the design and design and evaluation. I am going to rush through many of these partly because you all already know partly because I want to get to the demo part of it quickly. So, our starting point was this if you look at academia the education system the stakeholders are the various academic institutions the trainers and then the students who get trained and then the industry which is supposed to use the trained people. Each of these parties face certain set of difficulties if you look at it students face inadequate opportunity for quality learning lot of places insist on road learning and subsequently they have trouble in terms of placement. In academia the structure is poor in terms of what they teach is not what is tested and there are also this mismatch in terms of teaching load versus expectations of what is expected of them. And finally you know industry faces lack of sufficiently trained students who they can hire. So, in the context of this and also in the context of our own experience where we have large class sizes and poor attendance we started looking at how technology can help. And Bodhi tree and safe as a result of this our vision is that Bodhi tree is a platform where we have interactive multimedia books which will be the content will be created by so called star teachers. And this will be used for flip classroom conducting flip classrooms in various institutions including IITs and various other institutions also. This is similar to if you are familiar with the exceed effort in school curriculum they standardize the teaching material. So, that there is lesser variability across teaching quality. So, this is a effort similar to that and with this hopefully we will address the problems faced by all three parties in the system. And we envisioned safe to be a mechanism to conduct exams in a scalable fashion easy fashion using just students smartphones. So, these are the key technologies Bodhi tree itself has two parts one is the interactive multimedia book itself and interface for the instructor to manage a flipped classroom using this interactive multimedia book. So, flip classroom model I think you are all familiar with it is basically one where the role of what happens inside a classroom and what happens outside gets interchanged. So, Bodhi tree is designed to conduct flip classes by organizing material in just like a regular textbook, but in terms of videos divided into various chapters and sections and importantly the videos have embedded questions. So, let me show you an example now I will switch over to the browser. So, I will actually show you the course which I ran the last semester I taught this. So, this was the computer network course and I will actually show you the student view first and this is just like a textbook divided into various chapters and sections within those and let me show one example let us say reliable data transfer this is a section within a chapter. So, it has multiple sub videos we try to keep the video length short and also divided into various titles at a slide level granularity. So, that students face it find it easy to go through the material and importantly. So, let me play the video now you will see something happening. So, this is what happens as the video plays there are questions which are popped this is very similar to an interactive lecture. In fact, it is much better than an interactive lecture because even in an interactive lecture however good the instructor is if the class size is large there will hardly be one or two people who get to participate. But here everybody gets to answer without having to feel shy they get to answer the question which is posed we have a mix of both subjective questions as well as objective questions. Objective questions we keep track of how many have been attempted how many they got it right and so on. Subjective questions are just for them to pause and think. So, in this way the video is interactive as you can see in this outline here on the left you know we try to keep as many questions as possible. So, almost like one question per minute or sometimes even more than that sometimes we have numeric problems also where the students as they watch these videos only when they pause and think try to solve something they realize that maybe there is a gap in their understanding. So, they have to go back in the video and relook at it and then try to answer of course it is possible to you know cheat by answering something and going on. But this is we do not enforce it the system does not enforce that you answer it, but that is the intention the student answers it before going forward. So, these are the videos which are there and Bodhi tree system also has you know discussion forum chatting with others and so I will also show you the tracking of progress with respect to others. So, I will show you here for this course for example there is a score card we maintain and this is so we have a bunch of questions in video and out of video out of video questions are like questions at the end of a chapter in a regular textbook and we track which student has attempted how many and what the score they have gotten by default it is kept open because it is not related to marks really. So, sort of a friendly competition builds and if a student does not want to show this here she can hide it from others view, but instructor gets to see everybody's progress and I do see as an instructor you know I once in a while check as to have they really watch the video have they attempted these questions. So, they have not I ask them as to why they have not and so on. And this material can be accessed via smart phone or a browser okay and so let me show you the flipped classroom interface what I have shown you is the interactive multimedia book view from the perspective of a student from the perspective of the instructor. So, if I go to the instructor view so I can announce a schedule of which of these videos should be watched when okay. This is like similar to announcing some schedule on Moodle this is not enforced and there are many other things like discussion forum if you can see in the discussion forum there will be lot of discussion related to different topics and there is a email archive and so on. So, this is a complete platform for managing flipped classroom by giving material for students to watch during the class and what I do in class is solve these problems that is the way I have structured my course before before it is a flipped classroom yes. So, they are expected to watch it before and during the class what I do is so for every topic so for example here if I go here I will again I will go to the student view for example yes that is where safe also comes in system tracks whether they have answered the questions okay. Now I do not know whether they have just done guesswork whether they have really understood those cannot be gauged from this. So, that is why we have this safe platform for conducting quizzes that I will come to. So, that is where we felt the need you know so if I assign this watching material or reading material it turned out that more than half the students just do not watch before coming and what I do in class is solve further problems. So, this is the philosophy behind flipped classroom right. So, you solve problems in the classroom in the presence of the teacher who knows how to solve those problems and as you solve problems you have the student gets lot of doubts which he or she clarifies and there is a discussion around what questions arise during the in class session and what we found in the initial few semesters of experience is that if you ask the students to watch videos there is a significant percentage of them who do not watch because they just want to do what is minimum possible. Yeah, the question was how are these videos created does the platform have support for creating videos. The platform has support for once you have a MP4 you can upload the MP4 video format and annotate it you can insert those interactive questions in between those can be inserted at any point by the teacher and you can give video markers as I showed in that example. So, here for example so these all these marks here whether they are quizzes or just marker without quizzes those can be edited by the instructor later on. We use the Camtasia software mostly we have also done many just using open source software like Kazam. This is the problem solving session again there is a bunch of numerics and discussion and so on which the students answer in class. So, this is also interactive. So, they can do that they can choose to do it anytime really. In fact, the entire content of the course I usually give it in the beginning of the semester itself. If you want to speed ahead of the rest of the class you can do it. If you want to go in pace you can do it. Whether they have opened the video once we do not track that we just track how many of those in video questions they have answered. It will stop by default it will stop there are ways to cheat it which we do not prevent. For example, I mean there are plugins for any browser which using which you can download that MP4 and watch on some other player which does not have those interactive questions popping up. We do not do anything to prevent such things alright. So, this site is open for anyone and we have a bunch of video material on it. So, anybody can sign up and look at it. Currently we have I mean computer science courses we have computer networks computer architecture. There is a course on Unix and networks Unix and scripting and there is a course on wireless networks which I am actually currently using. So, there are some differences between CDP and NPTEL similar platforms. So, in terms of we view this more like a multimedia book like I said interactive multimedia book. Interactivity is something key and you know the fact that we say that it is a book also means that creating the material takes much more effort than typical NPTEL videos. At least that is the way we have created the content is this effort is more in line with writing a book. So, this started to develop around the same time as all these book platforms like edX, Coursera and so on. There are many similarities, but our focus has been on this flip classroom model whereas the MOOCs have gone in a different way they want open massive enrollment and all that. So, we are focused on the flip classroom model. The other differences also are that we have over a period of time added support for labs. So, there is a portion of Bodhi tree where students can in a typical lab what happens you do some experiment write a lab report. So, we have interface for those things also. And we importantly also have the module. So, this part is relevant mainly for programming. So, we have a module where programs can get auto evaluated. So, this has been used two times so far for CS101. This time also it is being used. This is primarily for programming and we have used it regularly for conducting programming test for project staff hiring also. Project staff hiring as well as MTech research positions. Any programming test we want to conduct we use the platform and I take this opportunity to invite any of you also who want to use it similarly to use the same thing. So, a few things about the case studies. I have mentioned all this already what happens before the class R during the class R. So, the weekly plan goes something like this. We have this three one hour slots and our batch size is roughly about 120 to 140 for core courses. So, what we do is assign about two hours of video watching time one and a half to two hours of raw video. It actually takes much more than that in order to absorb it. So, the expected amount of time is about 4 to 5 hours they are expected to spend outside the classroom and each of these three groups is mandated to attend one of the three sessions in a week. So, you know for a normal six credit course the so called contact hours is supposed to be rather the time spent per week is supposed to be about 6 hours. So, this is how it gets divided in our case and the advantage so the in class session I call it a tutorial session really because that is what happens problem solving happens. There is some amount of lecturing also which happens in the tutorial session for concepts which may have been difficult to grasp just watching the video that happens. And the main advantage is that it is conducted in very small groups much smaller than typical core class sizes. So, I have more or less stopped lecturing for the last three years all my in class sessions I just go around and look at how students are solving the given problems. So, one other thing is highlighted here which is in order to check whether they have really processed and understood the material given assigned to them before the class. We started feeling this need to conduct a quiz and pedagogically also it is known that if you conduct frequent quizzes then it keeps the students on their toes their learning level retention level goes up. So, we felt this need to conduct quizzes and if you conduct a quiz unless you assign a mark nobody is going to take it seriously. So, we assigned grade for it in fact in the early days where when only both entry existed and safe did not exist we used to do this just by paper printouts and so on of course that was laborious. So, this leads to a requirement of being able to conduct these quizzes in a easy fashion even for you know 30 students managing the logistics of printing and distributing and collecting it too much overhead for every class ok. So, the end result of all this use of Bori tree turns out to be what we call as PFC learning P for personalized it is like everybody gets to have an instructor for themselves they can cause the instructor there is no fear of embarrassment in answering the questions and so on it is flexible flexibility is something which the students like a lot they can watch the videos at any time or students are all their brain is active mostly at around 1 a.m. or 2 a.m. So, that is when they watch the videos they all watch it multiple times which is a very useful thing they have marathon video watching session before the exams. So, this is all based on feedback from students and the fact that we have periodic quizzes discourages them from postponing everything to the last ok. And since it is a book it is in some sense complete it tries to cover various aspects and there is videos slides reference materials practice problems in that sense it is complete. So, we have lots of usage of Bori tree in our department as well as outside colleges also several of them have used it. This is a small list of colleges which have used it in the past we also used it for a teach 10000 teachers workshop which was conducted as part of IITB project. So, here is a little bit of evaluation based on a student survey the survey was conducted in a computer networks offering as well as in the teacher training workshop. Some other subjective statements which they mentioned flexibility they like they like the fact that they can rewatch the videos they get more attention in small groups. And the fact that they get instant feedback in those in video questions is they like it the score board is also a feature they like there is a healthy competition of who gets more score. Some negatives are that some concepts are difficult. So, without the help of a teacher it becomes difficult until you come to the classroom session. And for many students you know flexibility kills discipline. So, we try to account for it by having the in class quiz, but still there are some students who ignore. So, you know there will always be some 5 to 10 percent of students for whom this mode of teaching they are not able to adapt to it may be because of this aspect more flexibility you give they tend to misuse it. It can happen that if you do not design the videos carefully it can lead to you know just watching without really absorbing attention deficiency problem. So, that is why we have those in video quizzes. So, that all requires very careful design of the it is basically like you know what a good lecturer would do by asking questions frequently in a class. So, some survey statistics whether students felt the need for instructor presence most of them said sometimes that is the maximum number sometimes they felt some felt not at all. Helpfulness of quiz within the video most of them say they are all very helpful in understanding. Helpfulness of problems outside the video most of them say it is very useful and typical time spent on a 20 minute video is 20 to 30 minutes. So, that is the bloat factor I mentioned right. So, you do need to rewatch many things many times in order to absorb it properly and in comparison to traditional model subject understanding increased is the maximum response we got and amount of time spent it varies as you can see some say it stayed about the same some say quite if you say it increased the amount of time spent has increased compared to traditional model of teaching. Exam preparedness significantly improved that is the response and that is overall score most of whom give a thumbs up for this model of teaching as compared to traditional model. We also did a formal analysis of whether this has resulted in improvement in learning improvement in grades. So, there is some amount of subjectivity in that I mean the same instructor without using this platform compared with same instructor using this platform the exams were different but assumed to be of the same difficulty level because instructor was the same and we found that the percentage of students who got A Bs and B Bs increased significantly. Behind. Yeah. Yes. Is that number increased significantly or I do not know I do not know is the right answer but I do know for a fact for example that last semester one of the core courses was taught by another instructor in the traditional model of teaching. I had taught it in the previous year using this approach and I know that many students continue to watch these videos because they used to write to me when the server was down we have an exam tomorrow please. So, I know that many of the students who repeated the course under a different instructor in the traditional model used to watch my videos for reference. So, this did happen so at least that is one data point. So, the first offering is a lot of effort because it is like writing a book but from the second offering onwards it is a breeze. So, that is how it is been for me for the last 3 years now because the material is all available readily so that is great. So, I am going to move over to SAFE now which as I mentioned you know its requirement came about from use of both e-tree really speaking because we felt the need to conduct these quizzes every class session. So, the end effect of SAFE is summarized here this is the notice stuck outside our classrooms and which turn into exam rooms during exam time and those are my students taking exam. So, while traditionally mobile phones are not permitted we turn that into a system where you can while completely preserving the integrity of the exam that is very important. So, they can use the system only for giving the exam during that time. So, this is of course very different from many instructors use Moodle quiz because you can use it but you are assuming that students do not cheat there is nothing preventing the students from using the same platform for doing Google search or chatting or whatever it is. So, the primary motivation I already mentioned for student participation flip classes and you know there are also attendance woes attendance falls off and you feel demotivated as an instructor. So, regular quizzes having a quiz every session improved attendance although there used to be cases I always conduct the quiz at the beginning of the class first 5 minutes. So, there are many cases where students just come for the quiz and then leave I do not prevent them but it happens. We have also used this for admission test and also project recruitment test pre-interview screening once this platform is there. So, that also formed a part of original motivation for safe. So, we wanted to conduct the exams periodically and in large scale the challenges being the scale is large. So, you cannot do paper pen it just is an impossibility to talk about for even a class for class of 30 you want to conduct quiz every class forget it you just cannot do it. But with this platform it is a breeze. So, our goals were it should be easy to use otherwise people would not use it. It should be easy for both the instructor and the students integrity very very important and robustness. So, because it is an exam with marks you cannot ignore even one or two students having failure. So, then you know as an instructor you want to have fairness. So, even if one or two students have problem it is a problem it is a you cannot use the system and hence robustness in the system is very important. So, the architecture is very simple students come to a classroom just like they come to a classroom for any purpose. They are typically already logged on to the safe app and the instructor simply gives a quiz ID and quiz starts on each of their phones they attempt the questions and submit. Then they can immediately view their marks if the quiz is so configured they can even see the answers instead of waiting instructor can get statistics of the how many students got which how much marks and so on. So, we have further enhanced this architecture by not requiring Wi-Fi during the classroom. So, during usage we encountered many instances where Wi-Fi was unreliable. There will be few students for whom for one reason or another the network does not work for whatever reason they may have typed the password wrong some configuration mistake something. So, this will unnecessarily delay the exam. So, we have rearchitected this to have a feature where the exam is downloaded in the background in the app periodically every 2 hours right now in an encrypted fashion. So, it is already there in each of the students phones when they enter the classroom when I want to start the quiz I just provide the decryption key no need for any network and exam gets conducted if the network is still not there no problem they the app stores it locally and whenever the network becomes available it gets uploaded. So, we built in that kind of robustness also to handle scenarios even though rare where network created problems you cannot open it well you ok. So, you can open it again but we will come to know I mean it will show up in the logs as student has attempted twice is like student are submitting 2 answer sheets you will come to know yeah. So, it is primarily meant for objective questions ok. So, we have different question types. So, let me since you ask that question let me go to demo part of it. So, this is the instructor interface. So, these are the yeah I mean that part is quite simple I will show you. So, let me show you for example, this is the quiz I conducted on Monday. So, it is similar to creating Google forms. So, you just edit and there are different types of questions we support single choice correct multiple choice correct fill in the blanks in fill in the blanks we do a exact match and numeric answer numeric answer is very convenient for engineering setting. There is a last option here which is known as image answer ok. So, this we have to support even traditional exams we can conduct using this student can write the answer on paper and just takes a snap and submit snap using the app and submit. Why this is useful? So, there is a back end support for correcting the question this has lots of uses for one thing there is a record of everything online we can back it up and so on. TA correction management becomes very simple one TA can correct one particular question in an exam as opposed to one particular paper. So, that process becomes very easy and right now we do not have it, but we are also building in support for what are known as rubrics. If you give a question let us say a phi mark question typically as a instructor you will have in mind you know if this mistake is there then I will deduct one mark. If this aspect is there as a positive aspect in the answer then I will give two marks out of five. So, these are called rubrics typically as an instructor is correcting the questions the rubrics are in his or her head these can be formalized once you correct 10 questions the rubrics are all entered into the system then correcting the remaining 130 becomes very easy just select the rubric whether this positive part is there whether this negative part is there in the student's answer and the student also gets to see why he or she has gotten a particular mark. So, that is I am talking about this image answer we call it as image answer you know it can be a paragraph it can be a diagram it can be an equation it can be anything. So, this system is actually more secure than a typical paper pen quiz because we have an option to shuffle the questions it becomes very easy to shuffle the question order. So, over the shoulder copying becomes much more difficult you cannot you know also mime for question number 2 answer is b you cannot do that. So, those kind of things become very difficult option order also we shuffle. So, those are things we do and we also have a feature of question group whereby you know question group is it does not get reordered it is a group of questions it does not make sense to shuffle it with other questions but as a group it gets shuffled with other questions. So, these are the features we have. So, this is a quiz I conducted on Monday actually and then so I mentioned the you know password feature. So, there is a encrypted zip which is downloaded to each of the students when I create the instance and this is a decryption password which is visible only to me of course I would not be projecting this is in classroom I will just write it on the board. So, decryption key they just type it and get started with the quiz. There is a dashboard we have where you know I get to see how many students have submitted whether there are any alerts. So, if student for example tries to go outside the app to refer to a cheat sheet or if the student tries to chat or if the student takes a phone call. So, all these things we track and these come as alerts. So, these are all info level level alerts I hope there are no let us see there are some critical net ok. So, this we need to turn it off. So, earlier we did not allow mobile data network that we need to change in the system. So, that is why it is showing as a critical alert, but right now we allow mobile data network also. And that is about it you can see the submissions you can download the mark sheet and so on. So, you have doing that period you have to look at only one. Yes. In fact, I have to go on saying that I mean the student can force it is like you know you do not physically prevent a student from leaving the exam room. If you leave the exam room without instructor's permission you are forfeiting the exam basically is like that. So, this is one of the reasons why this we need a exam room. There have been there have been such cases there have been such cases of two mobile phones. So, we do need a exam room setting this cannot be done in any setting ok. There are too many other ways of cheating in a non-exam room setting. So, in a if it is an exam room well you are supposed to have the phone on which you are taking the exam in front of you. If you have some other mobile that is equivalent to writing a physical exam with a normal mobile that has to be caught through other means. We do not do anything in the system to prevent that. Yeah. So, since this is simple quiz each week I create three versions of the not three versions three different set of questions and all of them are simple. So, they can discuss, but I mean since the questions are different ok. So, let me come back to the presentation. So, these I have covered ok. So, some technical aspects of what we do to achieve cheating free operation. So, we track whether the user is focused on the app. So, this is one of the key technical mechanism. We have authentication based on LDAP that is there and we log various things and report to the server ok. User in call whether the call was accepted or not, what connection the user is using, the phone's identity and so on and all this logging is robust against disconnections. So, it is not that student can disconnect the phone from the network going to airplane mode cheat and then come back that is not possible. I mean that is possible, but we will track it still. Genuine phone call we will come to know I mean it will be visible. I mean in smartphones app cannot take over the phone it cannot hide incoming phone calls. What we can do we have done it in some cases, but right now we do not enforce it is we can enforce airplane mode. So, if an instructor wants that you can do it that is like you cannot take a phone call during an exam and then you know this is our red screen. If you leave the app this red screen will pop up and there is a timer which will run and the timer expires. In fact even if the timer does not expire a report will go to the instructor saying that app was out of focus. So, this is a laundry list of various safeguards we have for different kinds of threats. We have a paper on it so I will not go into the details of this table, but I will point out one aspect which you brought up which is that you know we do need some non-technical mechanisms to go hand in hand. Everything cannot be solved through technology. So, for example you know we do not do anything to proxy another person comes in front in place of you and gives the exam we cannot prevent it unless you do an ID check. So, that it can be so we are exploring at least giving a selfie photo, but we do not want to use biometrics that is a decision design decision we have made for a bunch of reasons for one biometric is not there in many phones now a large percentage of phones do not have biometrics. Second is we do not want to keep biometric in our server then we have to be like UIDAA denying in denial if something gets broken then I do not want that headache I do not want biometrics in my it is too sensitive photo is ok. Various features apps you can have equations you can have images there is a countdown timer in the app. So, another feature which the students like is that you know everybody gets the same time if it runs out with the timer runs out it will auto submit. So, there is no question of somebody not submitting on time. Various exam settings we have duration of course you can configure negative marks whether you want to randomize the order whether you want to show the marks and answers at the end of the quiz. You can modify the answer in case you make a mistake in the answer key you can modify it and reevaluate. So, these kind of things we have is a live dashboard I showed you the current this is actually you know old one I showed you the current one. So, it turns out that the there are really no other apps which do this there are lockdown browsers and as some of you may know TCS is big time into this business of exam centers. So, there they conduct exams online exams on gate is also done this way they do it on desktops which are locked for that particular purpose only. So, here the advantage is that this is a bring your own device model there is no overhead of maintaining any lab in fact we have made it such that you do not even need Wi-Fi now any classroom you take call it an exam room you can conduct the exam. So, huge list of various courses as well as non courses which have used this system currently it is being used in 10 different courses in 2016 we used it for MTech admission within our department various colleges have used it for conducting national exams and mojo networks a company used it for some pre-interview screening and so on. So, some instructors so safe was originally meant for mental attendance check many instructors also wanted just physical attendance marking currently in our institute we use this biometric readers which aside from the disadvantage which I mentioned the hardware does not work many times there are many classroom is a maintenance issues there. So, on the other hand here physical attendance marking is just like empty quiz. So, we have subset that in order to support that feature also. So, I will just talk about this third one is logistic of how to handle students without any smart phone ok. So, that is a practical question and I will just talk about that. So, there are some students always who do not have our quizzes supported currently only in Android attendance is supported both in Android and iOS, but still there are always some students who do not have these devices. So, we have a bunch of worker on yeah. So, we have turned on a selfie option to handle that and we do a face match in the back end. However, although theoretically it can happen I believe most students will not do it because their cell phone is like their heart and soul they cannot part with it for 5 minutes. They are not going to part with their phone to a friend just for marking a proxy most students will not do it. So, but still we have this selfie option. So, currently some of the instructors are just using this manual attendance marking we have a manual attendance marking on a server for students without smart phones that is the interface they are using. It is also possible to just pass around a spare device which in our experience only about 5% of students do not have smart phones at all that is a very small number typically you either do manual attendance for those or just pass around a spare device. So, for quiz it becomes a more difficult issue because quiz has to happen simultaneously for everybody. So, for that the two options we have used are divide the students into two batches or two or three batches like I have done in combination with both the three. So, then I tell them you know you borrow from your friend and get it. So, that they do sometimes I have also done this printed paper we can print a paper version of it. So, I carry around some 5 copies of it when I come to the class and just distributed for those who do not have smart phone. The other option which we have used also is we have a you know bag of some 20-25 spare devices and that goes around classroom to classroom. We have a staff also who does manage this and that is given to. So, this is currently this is the option being used currently for CS101. CS101 every session has a quiz in the last 10 minutes. So, they are using this option to handle those cases. So, I think I will close with that I had prepared for a demo you can try this out. So, these are some student you can just pass it around. There are some student accounts demo student accounts I have created. So, what I am going to do now is I will tell you how this quiz is run. So, this is a demo quiz I am going to create offline instance now and the password you can note it down password for the quiz. So, that is the login for safe the password for this quiz itself is PM and D. So, you can try it anytime and yeah. So, the instructions for doing it is here it is the quiz works only on Android. So, you can install safe app from play store and login using that those credentials and you need to press sync to server once this happens in the background every once every 2 hours and then you are all enrolled in this course the RT101 I mean those demo accounts are enrolled. We do not support that because again it is a lot of the safety aspects come because of support given by Android for such usage. No, no, well iOS also it is possible iOS has lagged behind for quiz because of logistic issues we have not had a iOS consistent iOS developer. So, do let me know you can write to me by email if you want to try either safe or Bodhi tree safe either for attendance or for quiz. Quiz can be in a class or like I said we have used it for pre interview screening also for project recruitment. In fact, another mechanical engineering prof is going to use it in the next 10 days for that same purpose. In fact, that is an exercise we would like to take it up at some point but yeah, so you can do it if you get an instructor account which will be glad to provide you just need to use the interface to insert those quizzes and so on. I can be used even without those but the real benefit you get only if you use those features. The previous slides on Bodhi tree, you are trying to say a couple of slides as seen this is a point that you are trying to make is careful design of questions in between the video. At the same time, careful design of questions in the class. So, can we just throw a light because you have taken so many courses. What is that you keep in mind when you design such. So, when I prepare a video for example, I think about what are the questions I can ask to really check whether student has understood something and what multiple choice questions I can ask or what numeric question I can ask. Sometimes it is just a yes no question. So, for about every minute or so of the video I keep this in mind so that the video does not become one way. So, this is what I meant by being careful attention. I do not know if I answered your question. You even pointed out saying there needs to be careful in design of questions. So, could you refer back to the video. No, the in class can be really anything. In class can be question. I do that many times. I mean it requires as much care as any question does really, but knows further specific attention for the tutorial sessions. So, are your videos only the form of books or it could be instructor explaining some concept or what exactly is the content of the video. All our videos are you know screen recordings of guide presentations. It does not have to be that way. I mean we do not do anything to enforce. It can be any MP4 like I said really. Thanks a lot. I am sure there are more questions to be taken later and I hope this video will be in effect more later by the faculty. There will be more acres for budgetary and safe. Thank you. Thank you.