। ॐ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ ौ  त्वत्रा warehouse Technologies which is now two-week workshop. रब one-week workshop. बअ الت्रा त्वाल अद 중요ग नव गाड वऔक्चा some part available. अद जो slept over oversee. रब गमना kell हलें एक लुग. वाद hammerwork , तो औगह न알ोई कि layraya उछगई लystem त्या तावा सींजिこれ einem रन pick up bail. वाद Vish, फिलक तेदों, है, जी आध diyorum natuurlijk, कि आध श्मश कि लोगा़। से बदोत बअनी त़्भा. वो क्य की और चण्द तब, जी तब, थो साश्विदरुप विए लग मारते, गर साश्वि� continue on dash न पर कुषके से जगता, हो झे some percent oceans. आध todays electric तो eye क्रदगी कच्रट़े औलाई भीं 好तीकorean ।ού listening to theなく तो एक लिहांगे दे. लिहांगे तो वंष 갑자기 ।oc seven-t-seven people, two-eighty people afternoon online activities is a good statistical sample on what will happen when ten thousand teachers do the same thing you agree now that worries me a bit now suppose when you teach a course and you announce a deadline for an assignment tells me occasions when you will extend that deadline or something like that but for a course वो न्हीजता मच्ट्योंकोई दोंईचाट्या दे नियolution में थेटमानता कि मुत सुतिंच से सी सहांगे मैंZA सुतिच सब मिशुनु ज्भा से आजागाis it that there will be large number of students who will perpetually delay the assignment submission or who will never submit the assignment. य sharply to happen because their marks are considered. नोमे अनको अमाने सचत् दिह्डोंंगा थागाूरि पतिएद Liu we do not have any marks. But here we have teachers who are supposed to be self-portivated learners. So somehow things are not working well. l would like to know from you whether there is any method that we can follow to ensure that people actually participate.. because activities during every week are precursor to the activities in the next week.they are not disconnected or disjoiced.. unless somebody has done the first activity the second activity will be difficult and so on.. अगर स्प्टोक तोरीयम तेसकumer on-line activity तो 10,000 सेवल्टगां. तो तरेंगाएी ख़दी काए लगाएा लगागागे है. और � Terminal idea, श़ब यह आपगका के रवाजा, बच्यात ख़्राँव, इस देफ्याद वावाव्रामक. वहां, बच्यात जनगारे। जब शलतके है। श्तादिखा कर गोया मेंगगैं, बच्यात रवावे बच्यात शलतके है। सब e Tribe । 5 l 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 20 9 1 2 4 2 2 2 b 4 4 message . 2 . 4 3 4 4 4 4 4 4 4 4 4 ॐ अपने वाश्वाव खया । ूर्चत तो सब अईज़ा था। ॐ ूआनििया भो बसल्पता नहीं। । उःप पने श्नावावों । ॐ श्वावावल आप प्रैयावावन आप नाज ड़री भिधाश मेगगी लिया। wir we have been attempting to do that just to recapitulate for some of our new friends the standard cost figures Traditionally, under a QIP model or under a CEP model when a 2-week workshop is conducted for teachers the average cost per participant is between Rs. 12,000-pts that is because about 30-35% or 40-� daar for teachers come together at well placed आप दोज़ ती अर फोर इश्पर्स यो आप पे देम अनरेरियम सम अग देम यो लोग तो इनवाईट शम आप साँईद आप यो लोग to spend money on their travel, their stay, their on the riyam itself, all of that money gets amortized, gets spread over the cost of only 35 or 40 p. that is how the per-participant cost increases, you also notice that most of that money is spent in travel expenses, food expenses and stay expenses. When we decided to do a thousand teacher training workshop, we observed that we can cut down the cost significantly by reducing the travel cost because people would come from nearby places. We reduced the cost to about 9500 per participant per workshop. In 2012, when the minister and the secretary of higher education wanted to know whether they can scale it up and bring the cost down further, we enhanced that model to 10,000 teachers and we said we will embark on a large scale enrollment of remote centers. Thanks to many of the initiatives of the new remote centers, we now have close to 300 remote centers out of each 250, 270, 280 remote centers participate in any workshop. We have further reduced the expenditure on travel and stay etc by suggesting that majority of the people should be from the local city and only a few people from out station should be participating in the workshop from a remote center. As a result, we have brought down the cost to about 6,250 rupees per participant. There is some of the part of the cost that is visible to you because that expenditure is done by your remote centers jointly with you and your coordinator. Some cost is spent here in the coordinator's workshop in n number of logistics, you know the large team is working here etc. Mr. Ashok Thakur, Secretary of Higher Education, wanted me to continue to evolve this model such that this model becomes self-sustaining. Please realize the following. We say 6,250 it is less than half the cost of traditional CEP program. Yet when we train 10,000 teachers in one go, the total money spent by the government per workshop is about 6.2 crore rupees. You realize the enormity in absolute terms 6.2 crore rupees are being spent. Of course 10,000 teachers are being trained but 6.2 crore rupees is a lot of money. And what was observed by the government is that continuously it is not possible for the government to keep spending this kind of money year after year after year and expand it to other dimensions of higher education. So he has suggested that we should work out a sustainable model which is self-sustaining economically. That means the stakeholders and the greatest beneficiaries should be willing to spend money from their own pocket to do this. So if I am a teacher in a small college and if I feel that I will benefit from an ISD workshop of this kind, I should be willing to spend money from my pocket or my college should be willing to sponsor me to learn this thing. Now you will agree that even at 6,200 rupees it is not a very small sum if it comes to an individual teacher. Although the salaries have become better recently but you know salaries can keep expanding, expenditure also keeps expanding accordingly. So normally in our budget we rarely would have an individual as an individual an allocation for learning. We need to prepare the mindset of teachers to do that that if I am benefiting somewhat then I should be willing to spend money. Our movement in the direction of further reducing the physical contact hours is to further reduce the cost. You will notice again that people will still have to travel from nearby places. Some people may have to stay locally but their stay food and other costs are now reduced to only one week instead of two weeks. We have calculated that instead of 6,250 rupees the expenditure is more likely to be about 3,500 to 4,000 rupees. We believe that in future years 3,500 to 4,000 rupees teachers would be willing to spend from their pocket if they see a benefit from such workshops. This may not happen now. A separate Section 25 company is being established by IIT Bombay. A special purpose vehicle was suggested by the Secretary of Higher Education through which all the modalities would be handled so that the government funding will not be required to continue doing these good things. I do not know how long it will take. I have suggested to the ministry officials and to the minister also two things. Number one, teachers are the lowest paid employees amongst the engineering professionals in the country. Second, there is no developed syndrome amongst the teachers that annually they must spend so much money on their own self-education. We need to develop this mindset. For example, even today I find many teachers who do not invest in purchasing good books. They would not mind going to library and getting books issued. One disadvantage of getting books issued from the library is that they are rarely read. I found it out myself when I purchased a very costly book when I was a young teacher and I was not reading it regularly. It was my wife who shouted at me. She said, you have spent my money in purchasing that book. You better read it. So when we spend money from our own pocket, hopefully there is a greater incentive to utilize that money maximum. So I was just trying to tell you how the future course of these workshops will evolve. We have a three-year mandate for which we have to train 150,000 teachers thanks to this modified approach. And IIT Kharagpur will also be modifying their future workshops, not current workshops along this model. What it means is that using the saved money will be able to conduct more workshops. More teachers can be incorporated in this. But this money, this funding will end and when this funding ends, these kind of workshops should not stop because they are found useful by people. And you know that number of teachers to be trained in engineering colleges, if you take that teacher subject combination, there is just too many. And this is only about engineering college. Coming back to our workshop then, one important thing is that we will be training people for two more of their teaching activities. One mode is the conventional mode. See if I am a teacher in an affiliated college, I will have to continue to teach computer programming the way it has been taught all these years. Regular lecture sessions, labs, whatever whatever. And the students will be appearing for the normal exams. That is the normal mode. We must ensure that whichever way we train our teachers, the teachers benefit in doing that job better. So all the 10,000 colleagues should be well versed in doing. But additionally, we also want to train teachers in the semi-online mode, the blended mode or the MOOCs mode. And in that context, we want to ensure that all these 10,000 teachers are very comfortable using online tools for more effective training to their students. Now let us take your own colleges. How many colleges exclusively use a learning management system for all the subjects that are taught by all the teachers in that college? One, two, three, partially. Let me tell you that when we started in IIT Bombay to use the Moodle, it was exactly a similar case. A few teachers started using Moodle, etc. Today, not a single course in IIT Bombay can be conducted if Moodle is not operational. Not a single course. The entire registration of students at the beginning of the semester to the conclusion of the course, only the final grades that the teacher gives are required to be submitted online to an interface which is other than Moodle. Only the main registration of the students is on an academic office interface which is other than the Moodle. But once students register for a course, the registration information along with the information about which teachers are going to teach this course, etc., etc., is uploaded on the Moodle. And then onwards, the students interact with the teachers only on Moodle. Believe me, the teaching learning process has become more effective in the last four or five years in IIT Bombay. That is our experience. And as I said, there is nothing special about IIT, exactly the same thing can be done at each of your places. So let me ask you a more mundane question which I am sure will have positive answer. How many colleges have a good local area network in the campus where students can connect to, not internet, but to local servers within the college? Can you raise your hands? Say all of you. You agree that without that you cannot survive. How many of you have good backend servers which students access for doing variety of things, whether it is computer programming, whether it is something else. All of you have some servers of some kind? Yes. The only thing that is required is for someone to take the lead and say, look we have servers, we have network, we have these students, we have access devices. Many of our students even in small places have started using their own laptops or their own connectivity thing. Not everybody has laptops that is agreeable, but together with the computers that you have in your colleges, it is possible to give access to all the students. Why then, are we not using a backend course management system or learning management? And my reasoning, why we are not doing so, is A, we are not familiar with it, number one. B, because we are not familiar with it, we are not sure how useful it will be. Initially it appears to be a lot of Godagiri. I have to ensure that all student enrolled are done there etc. And then related fact is that many of the colleges do not have a good team of system administrators. Not only IIT, Bombay has a strong team through the computer center and other things, but every individual, for example T10KT project has a strong team of sysadmins. I have seen some people here, is Shweta here? No, I think they have, Shweta is there. So she is one of the head honchoes, the head honcho is actually Abhilash. Abhilash is also here, why? Abhilash, how many sysadmins do we have totally in our team, about six or seven? Seven. Seven people just to manage systems related to our projects. Now the fact of life is that individual colleges have never thought that such manpower is required to provide at the back end to make sure that the system works. The institutions do not mind having ten clerks in an academic office, but they would mind having four system administrators. I would like you to take this back home. If my remote centers, which are decidedly the better institutions amongst 5000 colleges in the country, hesitate in adopting this technology, then how will we be able to tell other teachers to do the same thing in their colleges? Now all of you use Moodle. When you conduct this workshop, traditionally you have been using Moodle. So you have Moodle installation there. People can connect on to the institute Moodle, but you would not use Moodle for local usage. I would submit Moodle is a name, any such thing. For example for this main workshop we will not be using Moodle, we will be using the EDX platform, which my colleagues led by Avinash Aute as I mentioned, they are modifying the EDX open source platform in order to get the entire platform usable by our people. So instead of using the Moodle learning management system, from 12th May we will be using the EDX learning management system. You can call it EDX IITB or EDX India or whatever version, we will be releasing that version in open source because EDX itself is in open source. You may wish to actually adopt that platform or Moodle or any other platform. But I would like to request you, take a clue from this meeting, go back and ensure that you now have a mandate from Professor Fata, that within the next 6 months each of your institutions should start using a full-fledged learning management system for the benefit of students. And believe me if you don't take that initiative, I don't see any other initiative automatically coming. Of course you will have to team up with some of your colleagues, you have to convince your director, you have to convince your principal whatever whatever, but please do that. Believe me you will find things much easier to handle. For example, in the absence of such learning management system, how do you collect assignments submitted by students? Student submit printed the printouts of the programs? Use Google grow. How many of you have teaching assistants while teaching this course, a computer programming course? A few institutions but not a majority of them. In IIT system a course cannot be meaningfully taught unless large number of teaching assistants are involved. I have seen days when there were no teaching assistants but some of the teacher colleagues themselves used to be teaching associates and we used to spend a lot of time in conducting tutorials and so on. Now we have teaching assistants who are our post graduate students and PhD students. But something I would like to tell you which you may try, which we have tried successfully for last five years in IIT Bombay. Second year, third year, fourth year undergraduate students are being used as teaching assistants for the lower level IIT undergraduate courses. In computer programming which I have taught many times, we had a large number of final year students working as the years, a number of third year and a few second year students also working as the years and they turned out to be extremely good years. Now many of you may not have enough post graduate students but all of you have undergraduate students of course. Now you would agree that in any class no matter which college it is, not only you are good colleges but even other colleges, in every class there would be at least ten good students no matter which part of the country you go to. Out of those ten, maybe four or five would be interested in participating as teaching assistants. This is another thing I would like you to initiate back home. That good teaching learning in a class can happen if the senior students participate as teaching assistants. In computer programming courses, how many of you do online assessment and how many of you evaluate the final answer papers? So first, how many of you evaluate the written answer papers in the final and mid-term exams? Can you raise your hands? All of them. Now I can understand if it is a paper on history or geography or sociology where long answers have to be given and computers are still dumbos, they cannot understand the long answers properly so they cannot automatically evaluate long answer. In programming courses, is there a need to ask questions like explain something or describe something or is it sufficient to ask programming problems? It is necessary because we have been doing so traditionally. Interesting observation is exactly what the professor also had mentioned. We have to deal with average students. First of all, let me tell you the so-called average student that you mentioned is termed average because of three important reasons. One, the intellectual capability may be inadequate. Two, the background preparation may be inadequate. And three, the ability of the student to participate in effective communication to be able to understand English lectures etc., etc., may be inadequate. You would agree that these are the only three possible things? Let me tell you such students exist everywhere including in IED. I have personally conducted Hindi lectures after dinner for as many as 50 students every year that I have taught CS 101 for one full month till such time that they were comfortable in understanding English language. I have seen students who have zero programming aptitude. There is also a question of aptitude. Like all of us can learn music but very few can become Latha Mangeshkar or Mukesh or somebody like that. Why? Because we don't have that aptitude. What is our ambition with such people? Our ambition with such people is that they should be able to write some simple programs correctly. That is the ambition. Is our ambition that they should be able to write an essay on doubly linked lists or they should be able to explain how recursion works? If you are also not convinced then why do you say that such explanatory questions are essential for average students? You are actually asking them to recite something which you know in your heart is not going to be useful to them at all. What is instead going to be useful to them is if they can write 20 lines of good code to solve a problem. 15 lines of good code to solve a problem. Let me tell you we fortunately realized this long time ago. So our question papers are still problem 1, problem 2, problem 3, problem 4, problem 5. The way we deal with the so-called average students is that there are enough simple problems which an average student can handle, tackle, comfortable. There are enough hard problems for the smarter people to apply their mind and to prove themselves. So I would like to submit this to all of your consideration that when we teach our students we will have a class of students different in their abilities, different in their preparedness. This will vary from very low capability to very high capability. The numbers may differ but this will be there in every class in every college in the country. You would agree with that? I would suggest one of the reasons why we set the papers the way we set is because again because the tradition which has become a rule. We have never questioned that rule. If we question that rule we will find that it is possible to set questions differently. Now I come to the main point of the computer programming course. In the computer programming course it is possible to test whatever students have done automatically if all that they do is write programs. How many of you have tried automatic testing of programs? Few people. प्रज़ाविनाशावते, the colleague that I mentioned is preparing a testing platform which can be used to automatically test such. These are not new things by the way. For last 20 years people have been writing automatic testing tools. Some of you might be familiar with the CDAC effort, the earlier NCST effort where you submit a program online and it will tell you whether it is right or wrong. Second problem, most of your examinations are closed book examinations, right? People are not permitted to carry books, notes etc. While it may be required and important for students to learn a few things by heart but in a subject like computer programming many of your students have been programmers subsequently. You yourself write programs. Tell me one practical instance where a programmer who has to write a program is not permitted to read a book, not permitted to read a manual, not permitted to refer to syntax, not permitted to test that program many times before he says that the program is read. Any one instance you know of? No. We are preparing our students to fulfill that role eventually. If we are going to prepare our students to fulfill that role why are we not preparing them right from the beginning to do it exactly like that? There are some hard questions. Why is it that we want assessment of the students in the traditional way to be the main force deciding on how it is that subject rather than how the students are going to use whatever they learn later and training them in that way. There is another reason why all our examinations in computer programming course in IIT traditionally have been always open book examinations, open notes examinations. I remember a case in 1980s when one student was a less prepared student carried four big files with about 500 written programs to the examination hoping that he will be able to map a program to the appropriate answer for a question. Unfortunately he spent all the three hours in searching through the files he could not find an answer to a single. But the point is when you make it clear to people that you have no problems if people refer to online manuals, people test their programs, people read whatever they want you will invariably will find that people prepare themselves better once they learn the trick and they actually refer less when they write it. At least the better students do. For an average student as I said we can set average problems. Now here is the crux. We would like to train these 10,000 teachers in setting problems, in setting quiz questions. One of the issues that have resulted from the traditional way of examining students is that our teachers have started taking less and less efforts to set question papers. Believe me setting question papers and setting quizzes is a very very tough task. Setting weekly assignments for the lab is a very very tough task. You know why it is tough? We have about 700 students when we teach computer programming. Every week there is a lab. There is an assignment at the end of the lab which they have to submit. First day evening batch comes they are given an assignment. The same assignment can be given to the batch which comes immediately after them because they don't get a chance to discuss the previous batches sir. But on Tuesday if the same assignment is given practically everybody has an answer ready. So you have to give another assignment. Wednesday you have to give another assignment. Thursday, Friday. Monday to Friday we have these labs. Then there are some people who miss their turn. For them we set extra slots on Saturday and Sunday. But suppose somebody smartly says, अ, पातक, repeats a question from the previous six or five assignments and deliberately misses one's turn and comes on Saturday, Sunday with ready answers for all the sick. So you have to set a sixth and a seventh alternate. For every week's lab you have to set seven assignments. I tell you whenever I teach CS1, the maximum time is spent in setting the assignment. That is what it takes to set questions when you are not asking them to simply explain something or describe something. But when you are asking them to solve a programming problem you have to set that problem. Second issue. Whenever we give programming problems we often ask them to write a program to do something. Consider CC++. Suppose the actual algorithm can be coded in about ten lines of code. How many extra lines you are required to write in order to write a complete program. Hash includes something, using namesplace std. Int main something, return zero, the standard things. Have you ever wondered why we are asking the student to waste his or her time in writing this in a paper? How many of you routinely say in your question paper, please do not write this. How many of you do that? Quite a few. I am very happy. But many who don't. Logic is needed. Logic is needed but using namespace std. What logic does it convey? That doesn't matter. But still in standard university examinations when papers are set, I have seen many papers. Many of you have uploaded standard papers and that was eye-opener for me. Most university papers require students to write complete programs on a piece of paper. If that student was being tested online, do you think he or she, even the average student, is dumb enough to actually type all of that? The student will cut and paste the previous program. In fact, on one hand we teach our students the reusability of the software. We want them to become experts in reusability. The only expectation is they should not be reusing their neighbor's software. They should be writing their own. But rest of it should be reusable. So this is another thing that we have tried successfully that our exam questions typically give a skeleton of a program. And students have to write only a portion which pertains to that specific algorithm. So we clearly state that any additional variables that you will require, please define them. The basic things are defined, the basic structure is given. Don't you agree that the time that the student saves can now be meaningfully spent by the student in thinking about the main problem rather than the frills? I am limiting myself to the discussion on computer programming, teaching and learning. But I hope you appreciate that we are actually responsible for causing a lot of wastage of students' time in assignments and examinations. Instead of letting them concentrate on the main problem of programming, we are asking them to recite other things. And if any question paper has said, four programs to be written. Each of the four programs will necessarily have to have using namespace STD. In fact, some of us will deduct marks if that is not written. Because the university rule number 13 we will say, completeness of the answer. Should we not change this? Now this change will occur slowly in the university system. But it is our endeavor to prove to the Indian world of education that computer programming education can be made different, can be made fun and can be made online to a large extent. And this is where a last point in the assessment or evaluation. If I make an examination open book, if I make an examination online where student has to write a program where I have given a skeleton, since the student now has a choice of writing the program, testing it separately, making sure that the program works and then submitting it. That means every student is likely to submit a correct program. Does it mean that all students get 100% marks? How will you distinguish between different programs written by people? There is one problem. The second problem is partial marking cannot be given automatically. Either a program works or it doesn't work. Whereas if a program is written by a student and if I am evaluating that program, I can think like the student does and I can say, this is what the student was trying and this is where the student made a mistake. But up to this point the logic was correct. So I might give the student 3 marks out of 5 or 2 marks out of 5. But a computer will give him either 0 or 5. There is no way to sort this out. And that is why whenever students get 0 marks, those programs need to be manually scanned by teachers to find out whether there is any partially correct answer. That is why. Second, even those people who submit all correct programs, you would agree that all programs are not equally good, quote unquote good. What are the characteristics of good programs? A, they are better organized and better written. That is an oxymoron. But B, logically they will take care of n number of possible situations which are not even specified in the problem. Take a standard example. A-square plus B-x plus C, find the two roots. Standard thing is minus B plus minus B-square, under root B-square minus 4A say about divided by 2A etc. Most people will check whether the discriminant is positive, negative, whether the roots are real or something like that. But very few people will give a separate answer if A is 0, giving an error. There is not a quadratic, it's only a B-x equal to C. Only fewer people will give an error message if A and B both are 0, stating there is not a proper... Forget quadratic, it's not even an equation. It's not proper. This is a simple example. But you will agree that those people who spend their time and mind to point out these things are better programmers, they are thought through better thing. Take any larger problem, sorting of arrays or doing some very specific thing with arrays. There are boundary cases which many people will miss. There are efficiency issues. Some people will write an algorithm which is far more efficient than somebody else. A past student of IIT who works in Pune, runs a company actually, his name is Navin Kabra I think. His company has developed a platform which evaluates programmers for employment by employers like TCS, persistent systems, whatever. It's very funny. It's an online exam. People are given the program specifications. They are given one or two test cases and they are supposed to submit the program. Obviously 90% of the participants submit correct programs. Or maybe less depending upon the complexity of the problem. Some people are able to solve it in that time. Some people are... But all those people, let's say 1000 people appear for this online test. My company has exactly 5 positions. I would like to interview at the most 15 people. But 1000 people have appeared. Out of 1000 people, 500 people submit correct answer to online program. How do I distinguish between these 500? So what his system does is his system accepts that program that the participant has given. And then applies some 50 additional test cases on that program which are not disclosed to this group. And these test cases go through the efficiency of the program, the boundary values. They even try to analyze how many comments are written in that program. They even try to compare the length of the program. They do variety of things automatically. And then they provide that out of these 500 people have submitted correct programs. These 20 are the best in all the quality. Of course there the objective is different. There may be another 5 people who have actually written the exactly good kind of program. But on some parameter they don't come on the top 20. But the objective of the employing organization is not to be fair to all good people. The objective is to be fair to themselves. As long as they get top 20 people they are happy with it. So their purpose is sir. Our purpose is to fairly judge all individuals and rank them. Is it not possible to develop a platform of this kind? Navin Kabra has offered to make this platform available to us for this course. For the 10,000 teachers training program and I am discussing with him. But my colleague professor Avinash Aute feels that we can actually develop that platform ourselves. What is required that platform is not difficult. Platform is any platform which will actually take a code and execute it. But you want to execute that code with multiple test cases. So the problem lies in setting up different test cases for the different problems. And the problem lies in setting up a problem which is amenable to decipher the quality of the program depending upon how people write it different. Now that's a hard problem. In short we would like to introduce in terms of this workshop a methodology which will emphasize teaching and learning good programming. Not necessarily teaching and learning answering questions which are asked in the traditional university system. So it's a major departure. In all earlier workshops we used to request co-ordinators to bring their syllabus. We would discuss that syllabus, we will discuss the question papers and then decide that the final syllabus for the main workshop will be based on what is commonly required. For the first time we are not going to do that. But we would like to ensure that our teachers are trained in pedagogy and in methodology whereby their students become better programmers than what they become today. We are not ensuring that everybody becomes a great programmer. It does not happen with any student, average student or the best student. That will vary differently. But what I am saying is that if we encourage our teachers to adopt this methodology in teaching and learning of programming, then even the average student will write better programs. Even the average students will be encouraged to write better programs. Even my colleague's professor Sridhar Ayer who is my co-teacher in this course and another colleague's professor Sahana Murthy, she and Sridhar Ayer have been teaming up on a whole lot of research in educational technology. They will be sharing with you their findings on the think payer share, the flip classroom, variety of things and what effective learning teaching. One of the discoveries is that learning happens maximally when participants discuss amongst themselves. I have tried it in some of the classroom settings in my own courses and I will tell you I hit a peculiar problem. I have a large classroom with about 200 students, 120 students of my MTech program. The course is effective communication. It's not programming or something like that. So I said okay we will form groups and you will discuss. So imagine you have a row here. So I said these 10 people will form a group. Now the 10th person in that corner cannot even see the first person in this corner easily. How they will discuss? If he has to talk to the last person he will have to shout at his voice and everybody in between will be talking to each other there will be chaos. Then I said alright 4 people in the front row and 4 people in the back row will form a group. The first row people started complaining about broken necks in about 15 minutes. They are always looking like this. You see the architecture of our classroom is not conducive, is not designed for participative activities by students. A recent talk by professor Eric of MIT at a Berkeley conference. They have, they are completely redesigning their classrooms. Modern classroom lot of money is being spent but the classrooms are round tables with 8 to 10 chairs around it. Much like what you see in many conferences these days. And the teacher is not static at one place. The teacher is actually moving around with a mic here. And whatever he speaks he is heard by everybody. And in the 4 walls there are 4 screens. So anybody without having to turn the head can see whatever is being displayed. How many of us are sure that we will be able to restructure our classrooms very quickly for this kind of discussions. So wherever you have fixed chairs or fixed benches you cannot re-architect. But wherever you have tables and chairs. We have seen in some of our classrooms where we have tables and chairs. We can quickly as the students it takes exactly 3 to 4 minutes to reorganize those tables and chairs in settings like this. Where the discussion sessions can be here. Professor Sridhar Ayer will talk more about this discussion session. But why I mentioned it today is that there will be group activities in this workshop itself. And the groups will have to be formed based on the nearness of the people who are sitting. So I would request you to start considering the following. Every row should be broken into 2 or 3 groups. The ideal size of a group is 4 people. In exceptional cases it might have 6 people. 4, 5 or 6. I know that there are some rows which will have a number of people which you cannot break easily into 4, 5 or 6. So for example I thought there was a row with only 7 people. Or there would be a row with 11 people. Anyway, I'll leave it to you. You don't have to form the groups today. The group assignments will be allocated 2 days later. And these group assignments will be the hallmark of the main workshop as well. Because we have consistently found that groups of people work far more efficiently together than every individual. Since we are not concerned with individual assessment, we don't have to give a grade to a participant. But we want to maximize the experience that everyone takes of it. Now in normal practice you would also have group assignments. How many of you have group projects to be done by the students of a programming course? All of you. Wonderful. Oh that is already a university rule is it? I was wondering how nice things also happen because of the rules sometimes. Very good. Now we have no such rule but we traditionally have a practice of having group projects. Some people also give individual assignments, they give group projects. Our observation is that in every group invariably there are sleeping partners. You also observe the same thing. So how do you ensure that either there are less of sleeping partners or that sleeping partners are recognized appropriately that they were sleeping partners. You only ask questions from the sleeping partners. But how do you identify them? Since we have 500 students in a programming course, these Vaivas are traditionally conducted by our T.S. Even when we used to conduct these Vaivas ourselves and you are very right, we can identify a sleeping partner and ask the sleeping partner about the project. Sleeping partners might be sleeping throughout the semester as far as that group project was concerned. But they are not sleeping individuals, they are very smart individuals. So what they do is before the Vaiva they will go to all the waking partners and they will ask them, what have you done? But some are. The point is you will not be able to decipher whether I have done anything or I have learnt only last night from the limited time that you spend on me. Two minutes or three minutes. So I will share with you an experiment which I did successfully twice in the last two offerings of CS101. I first calculated the average marks given by our T.S. in these Vaivas and I looked at the spread. The average marks were 6.3 out of 10 and the spread was somewhere between 3 or 4 to about 9 or 10. Now there is no way a sleeping partner should get even 3 or 4 marks. But my T.S. were unable to figure out that he or she was the worst case sleeping partner. They could identify a sleeping partner. काम कम किया है, कुज जबाब दिया. This is what used to happen. So I adopted a different methodology for evaluation of group projects. I made larger groups of 12 people, 4 people's group for a group project and 3 groups would form a team. And these 12 people will actually take up a larger project which they will break into 3 parts and these 3 teams will individually do. Then I said they will have to submit a group project report and the individual team project report which will be evaluated. But instead of the Vaiva by T.S. I said the individual contribution marks out of 10 will be given through peer evaluation. And the peers are all the 12 members of that group and the coordinators of the 3 groups will be together form a body of the team coordinate. They were given guidelines on how to evaluate an individual's contribution. Each individual incidentally was required to submit an online diary of how many hours one has spent time in doing what. They are programming, thinking, discussing, writing report, debugging, writing program, whatever. Of course students are smart even the sleeping partners had properly filled up diaries so there is never any confusion about it. But the peer evaluation procedure was like this that I am a member of a 12 people group. I will sit in a meeting with all 12 people and I will suggest that based on my contribution I should be given these many marks. These marks will be debated by these 12 people. I have to convince all the 11 people that I have actually done this much work. For the first time in the history of CS10 and teaching in IIT Bombay we had several students being given zero marks by their peers. This had never happened. One fellow actually had a temerity to come and argue with me that my peer evaluation is all wrong. They are biased against me. They are giving me zero marks. I said come with your coordinator and the coordinator said but you admitted that you have done nothing. But in that group also he has done nothing but he was given one mark by his peers. So these are the kind of arguments. Please understand that students are smart enough to fool us within a 5 minute viva. But I cannot fool 11 colleagues who have seen me day in and day out for that one month doing nothing. And I calculated the average marks given through this peer evaluation process. This was 6.8 instead of 6.3. I spent so much time in telling you that you can do this kind of innovation. You can still be closely accurate and you can be sure that you have not done any injustice. And most important, you save all your time. Of course, how do you ensure that people don't give 10 out of 10, 8 out of 10 or 7 out of 10 to everybody. So here is the fatigue strategy to avoid that. I announced that while peer evaluated marks will be taken as sacrosan but they will always subjected to the following random test. I have the liberty to choose any two members of this 12 member group 12 member team and conduct a rigorous viva for those two people and determine how many marks they deserve for their contribution. Suppose one of the persons has been given 7 out of 10 marks and I determine that that persons efforts are worth only 4 marks then every member of the group will lose 3 marks. Every member of the group will lose 3 marks. There is a lot of turmoil there. How can you do that? It says is your peers you evaluate them rigorously I also evaluate them rigorously. And I said I will never evaluate and so I reduced it to only one sample person. I will choose any sample. But if that sample person gets less marks the only argument was suppose you evaluate that person to get more marks than what the peers have given will you increase the marks for all the students. I even agreed to that. So I would take one sample. The sample need not be the weakest student. I would actually go through the marks that they have given and randomly select and announce. So for 500 students I do not have to conduct interviews for all the students. I have to conduct a sample. I will conduct that sample viva jointly with the project coordinator of that 12 group team. So that coordinator he or she will be sitting and the student will be sitting and will evaluate. The fact that such evaluation is being done very seriously ensured in the second year that there was zero discrepancy in all the marks given by them. Now this is an experiment which is easily doable. If not for the university main papers or something there must be something called internal marks in all your colleges. Can you not at least experiment can we not encourage our 10,000 teachers to experiment with such modalities. There the key problem is how do you design projects for so many students. How do you ensure that a project which was submitted by last year's student is not reproduced by the students this year. Yes what I did. I would make available all the projects done by the previous year's students and say you can work on this project but this is what the last year's students have done. What additional functionality you will add to that system you give that as your proposal and do that. Reusing of the software is encouraged by this practice. They all know that in their real life they will actually have to deal with large programs either written by them or written by others and keep making modifications. That's teaching them very practical skills.