 Today, in the morning session, I am going to discuss first the general assessment procedure and more specifically, setting up of test and quiz questions. So, I begin with discussion on the assessment objectives and methods. First and foremost, we presumably assess the students understanding of the topic or subject. Additionally, for this particular subject, computer programming, we wish to assess the programming ability of the students. There are two kinds of abilities. One is the ability to read, analyze and understand a program. As Professor Ranade mentioned, this ability is as important as is the ability to construct programs. In fact, without the ability to read, analyze and understand programs, it is extremely difficult to develop a very good ability to write good programs. Obviously, to assess the ability to write correct programs and write programs such that they are not only correct in the sense that they compile correctly, they execute with given data, but they fulfill the functional requirement that are specified for that program to achieve. We will talk more about it when we talk about testing in the context of software. Now, the synthesis ability can be tested. That is, the ability to write programs can be tested properly only if students are given enough time. Even for a single, not very large program of about 80 to 100 lines, if you want to test whether students can write such a program properly. When I say properly, it means it has multiple facets such as design of the program, meaning the documentation preceding the program writing, writing of the program, thorough testing of the program would typically take 4 to 6 hours. Whereas, we expect students to write such programs in a small confined time limit of a written examination and where we expect the student typically to write 3 to 4 programs. Please note that while we might be able to test the general competency of a student in writing programs, we cannot gauge the extent to which the student has acquired professional skills to write good programs. Second, in real life, no programmer ever writes program in an exam like environment. Never. Exams are mostly closed book. Even when we have open book examinations, the student has no recourse to testing whatever he or she has written is correct or not. One has to write typically on a question paper, on an answer board by pen and pencil. In actual programming, programs are written partly on paper, partly on the machine. They are corrected on the machine. They are analyzed, evaluated, read on the machine and if we do not make a machine available, then our assessment remains unrealistic. There have been umpteen instances globally where programming tests are increasingly conducted online. That means students are given access to a computer. They are asked to write a program and they are given unlimited time on the computer. They can compile their program unlimited number of times. They can refer to internet. They can find out what else they need to know about the right algorithm, whatever, whatever. At the end it is expected that they write a good quality program which works correctly. That indeed is the kind of training which we wish to impart to our students. Unfortunately, our present assessment methods do not test them properly. Now the conventional time bound test and exams, although they are not ideal for this subject, they seem to be the only way in which we can assess people, at least in coming years and at least in those institutions and those universities which stipulate only written examination, closed book examination, limited time examination as the standard way of assessment. If you really think about it, not only in the subject of computer science or not only in the subject of computer programming but in most other subjects you find that when we teach our students, we teach that the students will learn something, something. We expect them to become something, something but what we assess them through our written test mechanisms is probably not that something at all. While this may be generically true, I do not wish to get into that debate but it is certainly true about computer programming. Coming back to the million dollar question, well given the fact that I will have to conduct exams in this fashion, what best can I do to be able to assess the students to the best of my ability and the assess the best of their ability? It is this context in which I would like to discuss the way we could set quiz questions, the way we could set test questions, the way we could assign course projects as part of the internal evaluation and so on. As I mentioned, a team or a group project done by the students is very useful. I have shown here that typical IIT weightages for the assessments of different components during the semester. There is an end semester examination which is of 50 marks. By the way, several years ago when we removed the computer programming course from third year and fourth year, which was taught at that point in time for the five year degree program and brought it all the way down to first year when the new four year program was introduced, this programming course had 80 percent in semester evaluation based on whatever students did throughout the semester and the paper based end semester examination was only for 20 marks. I personally still believe that that was the right way of assessment. However, subsequent to a senate decision, all end semester examinations have 50 percent weightage. What we do is, we have a mid semester examination to which we allocate 15 percent weightage. During the semester, we conduct several tests and quizzes which totally amount about 15 percent and we assign a group project which is very rigorously executed by people. In fact, many of us believe that students learn programming mostly during the project phase. We allocate about 20 marks for that. You have already seen the use of technology. Many of you would have used the clicker application on Aakash tablet. In IIT Bombay, wherever students have such tablets or such clicker devices, it is not uncommon for a teacher to conduct two or three quizzes in every lecture. When the quizzes are marked quizzes, that means those marks are going to count towards your total grade. You know that every student will be more alert, will work harder to ensure that he or she scores good marks. This is a practice which is ideal for a flip classroom approach which some of our institutions will be adopting sooner than others. Eventually, I believe that with the onslaught of MOOCs, the blended MOOCs approach will become the common pathology for teaching not just this but many other courses. When that happens, the entire assessment process by the way will go topsy-turvy because the assessment will almost be on a continuous basis. However, any continuous assessment done on a weekly basis can generally examine only the concepts that are covered during that week. Whereas, any subject including programming subject essentially requires that students are not only understand individual topics, but they are able to connect these topics together and construct some meaningful software, meaningful programs at the end. To assess that ability, you have to evaluate them at some point in time in the overall sense. Currently, we try to do that using a final examination, but eventually this would be done by a combination of a project evaluation, a rigorous examination of the program that the students have written and so on. I have written these two sentences in bold. From the teacher's point of view, from the university point of view and from the society's point of view, it is expected that whatever grade or marks our evaluation and assessment results in, it provides sufficient and accurate discrimination. For example, if in a particular place, the subject thought computer programming, the assessment results in all students getting, let us say, between 60 marks and 80 marks. Now, the students might feel very nice, the faculty might feel very nice, but the society, the employers will ask you a question. How do I understand how much a 70 mark scoring student is better than a 60 mark scoring student? I do not see much difference in the mark. Is it true that there is not much difference in the capabilities? Then you are saying that either all my students are very good because they all score between 60 to 80 or the marks are not realistic and in fact, all the students are not very good programmers. These kind of conclusions are bound to happen if the range of marks is not commensurate with the actual understanding and the programming ability of the students. That is the reason why whatever be the assessment system that you adopt, it must provide for a proper discrimination. For example, if a student gets 40 marks, it must be understood that the student has just learned the basic concepts, but he or she is not an accomplished program. On the other hand, if a student gets 80 mark or 85 mark, it must be understood that the student is an accomplished programmer, has the ability to construct fairly complex program on one and so on. It is this discrimination requirement which forces us ordinarily to construct our examinations in a graded fashion. That means the questions that we ask, the questions will be a few simple questions, a few questions with some medium complexity, a few hard questions. It is not uncommon in the IIT system at least to include at least one question in the paper. The matter about which, the algorithm around which was never ever discussed in the class and people have to apply their mind, their ingenuity and figure out how to solve that problem. Such questions are hardly solved by 5 to 8 percent of the student, but such questions are put in order to discriminate and find out such people separate. On the other hand, it is equally important that students who have prepared their normal understanding of programming should not feel deprived and should not fail the course and that is the reason why every question paper must contain adequate number of questions which while covering the syllabus, while covering the assessment of understanding are still simple enough for the majority of people who have barely prepared for the minimum amount of understanding and minimum amount of competence for them to pass here. So, assessment should be graded, assessment should be gradually simple, medium and complex in terms of the effort that is required for people to solve. The second principle of any assessment, particularly for computer programming and certainly for all other subjects, every assessment should also be a learning experience. Now, this is not commonly seen in most of our conventional examinations. In fact, most students do not expect any examination to be a learning experience at all. They prepare for it with some kind of a scare in their heart. They prepare for it with an intention to maximize their score and their primary objective is to somehow get the answers right to the question. During that period, they are not straining their minds to learn something while giving the exam. Those of you who have read my paper on the blended books will recall how my own first test here by Professor Isaac ended up into an extraordinary learning experience, although from scoring point of view, it was a very frustrating assessment. But we believe that such assessments are extremely important because even they add value in terms of additional learning by the students. We will conclude this small part of the discussion by repeating these two points, namely an assessment must be adequately discriminatory without scaring the low performing students and without offering a bonanza on plate or 100 marks to the people who ordinarily perform better but are not necessarily the truly top professionals writing extremely good programs. Second, assessment as far as possible should also contribute something to the learning. I now come to the discussion of quiz questions and the test questions. This slide will be familiar to most of us as teachers. They describe the different types of quiz questions that we set. The most common are multiple choice questions. One important point to remember when setting up multiple choice questions is that all choices which we list should be plausible. That means such choices should occur in the minds of the solver while solving that problem that yes, this looks like a plausible solution. Suppose we give four choices and among those four choices only two are plausible answers and two answers we have just written arbitrarily so that four choices A, B, C, D can be given. Then students are smart enough to quickly understand that these two choices do not make any sense and would not even look at them and they will look only at the two. As a result your multiple choice question reduces to only two choice questions and the student spends less effort in properly analyzing the problem and trying to solve it. We will see some of the examples of such quiz questions there. Also quiz questions where a single value is required to be given like fill in the blanks. In the context of programming we often use such type of questions where we give a program and we ask for a certain input what would be the output produced by the program. It could be a numerical value, it could be a character string, etc. One common problem in such quizzes when held in a large classroom is that if there are same questions given to my neighbors it is quite possible that I can briefly look here and there and just copy the answer hoping that that answer is correct. However, in computer programming subject we have an additional chance to actually ensure that the question may be same but the answers have to be different from everybody. In IIT we typically use the roll number as that ploy. We often say this is a program which is given. We would like you to analyze that program and find out if the input is so and so what will be the output. This basically tests the ability of the student to read the program, analyze the program and understand what it does. Except we say the input to this program is the last digit of your roll number or the input to your program is the last two digits of your roll number. It is very unlikely at least it gives me ten different answers if I say last digit of your roll number. The point is how do you correct it? It can be corrected both automatically as well as manually without much problem. In manual correction we used to give our teaching assistants a table. The last digit is one, output should be this, last digit is two, output should be this and such table of ten values would ease the way in which the teaching assistant could correct that table. Automatic correction of course is very obvious just like the multiple choice quizzes can be corrected automatically even such fill in the blank type of answers also can be corrected automatically by the machine. We can prescribe that the output should be so and so, so and so, so and so, so and so if the input is so and so, so and so, so and so. Now the input if it is dependent upon the roll number in any automatic system the roll number is captured by the system as a part of the identification and therefore, every machine which automatically grades a question like this already knows what is the roll number, already knows what is the last digit of the roll number. The third one is fill in the blank fill in the blank is in the context of programming is a question where a program is given with one or more missing portion and you are required to answer writing a particular code which has to be written in the missing place. Let me show you an example, these lines represent pre-written program code. Then I will draw a box here, students will be required to write some lines of code here which will perform certain functionality that is prescribed in the question paper. In simple quiz problems, I might have programs which may be not very big programs and I might leave only one such small portion where may be a single iteration or a small piece of code can be written. For example, I might in the top portion here I might declare int i n sum, I might say input n, I might say sum is equal to 0 and I will give this place and ask the student to write code which will find out the value of the sum of the n natural numbers. Obviously, we would expect the student to write here something like for i equal to 1, i less than equal to n, i plus plus, we would expect the student to write sum is equal to sum plus i and then I would have pre-written here already probably a statement like c out sum. Notice the advantage of asking such a question, from the teacher's point of view since all variables etcetera are defined uniformly when I want to evaluate this answer manually it is very easy for me. I will not find different answer books containing for k equal to something to something, total equal to something to something etcetera, etcetera I will get to see exactly the well defined names and variables. From a student's point of view, the student will find it much easier to write because he or she is not spending unnecessary time in writing the declarations, writing hash in full h t d o etcetera whatever whatever whatever. The second advantage is that such answers can also be graded automatically. How can they be graded automatically? What is done is that you actually write as a model answer the complete program, test that program and ensure that that program works correctly. Now, people submit such things online and when they submit such portions online an auto grader will take this portion which is submitted by the students and will replace the correct answers corresponding portion, insert the portion supplied by the student and simply run that program again. If that program compiles properly and if that program gives correct results for two or three different values of n which the auto grader would already be have been given then the student will score full marks on the short question otherwise the student will score 0. While the multiple choice with questions are typically of one mark it is not uncommon to have quizzes also varying in complexity. In fact, there is a sort of continuum from the smallest and the simplest quiz of one mark requiring less than a minute to a medium quiz requiring two to three minutes to answer to a slightly more complex quiz requiring may be five minutes to answer to a quiz which almost becomes like a test question which may require ten to fifteen minutes to answer. So, there is a continuum of the assessment moving from the notion of a quiz which is always assumed by the nature of the meaning of the word quiz to be a quick short response kind of thing versus a test question which is assumed to require some more time. So, please remember that such a continuum can be provided in our question papers also. When you provide such a continuum when you give several questions of one or two marks a few questions of five marks a few questions of ten marks etcetera it adds to your ability to discriminate between the students understanding of the particular subject. Coming back to automatic grading you can appreciate that what I just described need not apply only to small portions of the course. After all in online examinations students can write a single program of hundred lines and they submit it online and the auto tester can actually test that program both whether it combines correctly or not and second whether it executes correctly or not. Why not then at least in the subject of computer programming you conduct only online test it can be done it is being done in fact everywhere and I am not talking only in the context of massive open online courses I am talking in the context of normal examinations. NCST the erstwhile NCST and now CDAC for ages have been conducting such online examinations for the subjects which they taught in their diploma. In fact we are actually seriously considering whether we could use their auto grader their parikshak system I do not remember what exactly they call it whether it would scale up for let us say one lakh students when they take a MOOC course and at least for thousand students in a particular campus and so on. There is one problem however with such auto grader the entire program either works correctly in which case the student gets full mark or the program does not work for some reason or the other. The reason could be as silly as a logical mistake in some assignment statement where rest of the program is correct. Unfortunately the auto grader will say program does not run you are given zero marks. Now this may be useful for very competitive evaluations such as for example entrance evaluation for a job interview where the employer is not particularly concerned with being fair to all people. The employer is concerned to ensure that the boys and girls that the employer employees are the best among the available lot. Say if 100 people appear and in only 20 people are able to get perfectly working program although 20 others could be as good programmers or in fact better but because they make some silly mistake their program does not go through they just get rejected. In an education system we cannot afford to have such kind of assessment. Remember what I said the assessment must provide discriminate that is the reason why we always have partial mark. When we evaluate a written program that is submitted by the student what do we do. Let us say there are multiple segments of the program doing different things. Let me give you an example suppose I am given two arrays as inputs. This array is array A and there is another array called array B. Let us assume that both these arrays contain m and n elements respectively which are all integer values. The problem that I give to the students to write a program for is that use a sorting algorithm on a program. Each of the arrays and then merge these in the given order of course in an array C. Now this is a program which would take some amount of time perhaps 20, 25 minutes etcetera. And as I said earlier if I were to automatically grade it and the automatic grader would say 0 or 10 marks it would be very silly. The student might have made some silly mistake in the logic somewhere and should not suffer. When we manually evaluate this what do we do. We typically identify often pre-identify the segments of such a program independent of who writes it and how one writes it. What are the typical segments in such a scenario where two arrays are given they need to be sorted they need to be merged into a single array and of course the merged array need to be output. I can discern something like five segments here. Segment one will be what I call an input segment which will collect all the values then value m value n m elements of a n elements of b etcetera. I will have another segment which will say sort a. I will have a third segment which will say sort b. I will have a fourth segment which will say merge a and b. And I will have the last segment which will have output. Now if I do not tell the student to follow these steps the student might for example collect input only for a here then sort the array a then collect input for b again and then sort the array b and then proceed. Some other student might collect input for b first sort the array b collect input for a then and then sort the array. In short the sequence in which the student would write program may differ from person to person. Remember we are now talking about automatic grading and not manual grading. In manual grading of course when we evaluate the question and answer book we can easily see what the student is doing. The point I am making is that as a teacher when I set up such a question I would allocate some predefined portion of marks to each of these segments. For example suppose this is a 10 mark question. Now I know that logically the merging is the most critical phase. So I might allocate for example 4 marks for merging. I might allocate 2 marks for sorting each of these arrays and I might allocate 1 mark for the right input statement and 1 mark for the right output statement. Indeed when we evaluate answer books on large scale and you would also be doing the same thing you would have several associates either teacher associates or PhD students or M.Tek students as associates like we have you would invariably be giving such a marking scheme. I understand that most universities require paper setters to provide such a marking scheme. Now imagine that in addition to providing this marking scheme I am now conducting an examination online and in that online examination I have already pre-written some portions of the program as I mentioned last time something here, something here, something here if nothing else they may be just comment line. And I require that the students actually take this template which is available as a text file and type in their values their input their programming statements here, here, here and so on. And then I will submit this entire single file to my auto grading program. How does auto grading program work in such a situation or can work? There are not many such tools available. In fact my colleague and advisor to our NMA ICT project professor Avinash Aute is actually currently busy trying to construct such a tool. What that tool would do is I would supply that tool with the correct program. That correct program is pre-written but that correct program also has exactly the same components identified. Now what the auto grader does is when I submit my program to this auto grader the auto grader works in stages. First it removes its own first module and replaces it with the first module of what the student has written and runs this program again. Since the rest of the program is known to be correct the program will run correctly if what student has written is correct. Next I do the same thing I remove the students module from the first stage put my original correct answer here and now take the second module. Successively in five different runs I will be able to evaluate each of the five portions that the student has written automatically and I would have stipulated to the auto grader that if portion one is correct give one mark portion two is correct give one mark etc. It is for example quite easy for a student to goof up in not writing the correct input statement. You have to write a loop moving taking values from I equal to 1 to m for A and 1 to m for B you make some slight mistake here and you might not get your input correctly. If you do not your entire program will not work properly at all but in this particular case suppose the input statements have some error and the input statements do not work. In the second phase when the grader takes your second path and fits it into the second module here it will be using the correct portions of the remaining program and if your second part is correct the auto grader will say yes sorting of A is a complete. The student will automatically get two marks. I am sorry for elaborating this but such auto grader itself can be a good example of a C C plus plus programming project barring of course a few things where the auto grader will have to intrinsically interact with the operating system and so on. So you can write programs to check programs and you can give partial marks. I will conclude this part of the discussion by saying that computer programming is a very different kind of subject. In no other subject such evaluation of long answers automatically can be done where partial marks can be given. Imagine for example in a humanities course on philosophy or on history where you are required to write long essays. Now in those essays also you will have different portions describing different things but the teacher will have to read that entire essay read every portion and evaluate in his or her mind as to how many partial marks are to be given for the correct answer here and there. Such auto grading of large essay type answers in fact is a matter of great research. These are called AI graders or artificial intelligence graders. There are machine learning activities that are going on in the research field which actually are very close to cracking the problem of automatically evaluating long answers. However such auto grading programs of long answers require what is known as straining. That means you want to evaluate one lakh answers automatically. You first evaluate 100 answers manually. These are online submissions but you evaluate them manually and tell those grades to the auto grader. The auto grader actually analyzes the 100 submissions the same 100 submissions and using the AI algorithms it classifies these 100 submissions as per some different marks. Then when you supply the manually evaluated marks it uses them to learn if the student has written something like this it should get so many marks and so on. Then it evaluates additional 500 papers automatically using your parameters. You are also expected to evaluate some sample of those 500 and this is how the machine learns. That is why such algorithms are called machine learning algorithms. The EDX platform which we hope to modify and use in India as the base platform for running massive online courses is actually working very hard towards perfecting an auto grader of this kind. Sorry for digression but I thought being teachers of programming you would all be interested in these developments. However coming back to the mundane matter of automatically evaluating the papers which are submitted by students in online form and yet we want to give partial marks. Such auto graders are available and they are able to give partial marks to long questions also. This has an additional implication that I would like to give a final statement question which requires a long answer but I would like to examine the student's ability to do small portions correctly as well and I would like to award marks to that student. This amounts to my saying that I am almost conducting five quizzes of one mark each or two marks each except that these five quizzes would necessarily be on some disparate topics or disparate aspects and which will be individually assessed and not collectively whereas here I am able to understand the total knowledge of that student for writing a full program from start to end and yet I am able to evaluate how different portions are tackled by the student. By the way I am aware of the fact that many of my colleagues attending this workshop are already familiar with these things. I wanted to emphasize that setting up of such questions not only results in easy evaluation but it also permits partial marking to be correctly awarded to students. As I have already mentioned I could have difficulty levels of quiz questions and these difficulty levels are typically at least many of us usually classify them as simple, medium and complex needing respectively about one minute two to three minutes and five minutes. Now if we have a large question bank, a question bank is large number of quiz questions on every topic in each of the three categories how could I use them? Well I could actually set multiple examinations of the same general complexity level. So for example I may suppose I want to evaluate several students say 50,000 students on let us say just two concepts of handling arrays and ability to generally write iterative solutions for numerical problems just these two topics. If I have a one set of paper to assess these two topics conducting physically the 50,000 students examination in one shot may be difficult. I may have facilities to conduct examination only for 10,000 students at a time. What I will have to do then I will have to give a different paper to the other 10,000. Another paper to another 10,000. How many papers can I set such that the papers are roughly of the same complexity as far as students are concerned? You will know that many competitive examinations world over do precisely that. Of course what they do is most of their questions are multiple choice quiz questions and they do create a large question bank of this kind. Please note that the team activity that is prescribed as a part of this workshop which the teams which you form yesterday would be doing back home and submitting these assignments in the next four weeks actually are going to contribute to such a large question bank. Designing a question of this kind requires you think about a particular aspect to be tested decide whether you want to give a multiple choice question or a fill in the bank kind of question write down that question write down the possible answers as the choices then you ask a colleague of yours to actually solve that problem as if the colleague is giving the examination and then a certain whether you are estimate of the complexity of that question is correct or not. Then you identify the right answer mention it as a part of the question bank collection and more importantly you write down explanations as to why all other answers are wrong. It is this complete description which makes one quiz question assembled. Now imagine more than 880 teams that I noticed yesterday are going to produce questions of this kind we would have a huge potential question bank as I said we will be releasing this question bank in public domain but when I say public domain what we propose to do is only the questions will be released in public domain it will be available and accessible but the right answers and the explanations for wrong answers will be given selectively only to teacher that is because the teachers when they select these questions either randomly or by some choice should not have to spend time in reworking the solutions of those quiz problems somebody has already worked that out and somebody has created a data bank. This then is the advantage of having a question bank of this type and we propose to get one for the subject computer programming in cc plus plus using the resources in the form of all of you attending this workshop. Here is an example write missing code for printing elements of the merged array look at what this example does forget the standard statements include IO stream and include STD IO data actually is not required in my opinion using namespace STD. So here there are two arrays A and B which are to be merged into an array C enter how many elements of array A input is N enter how many elements of array B input M. So there are N and M elements respectively now please note that the elements of the arrays A and B are simply read and while they are read they are merged into C please note that the program does not require any kind of sorting or any kind of merging in any order here merging merely means collect the elements of array A collect the elements of array B and put them in any order and but put them all in C and print them out this is a simple problem since no sorting is required the way this program is written is that while a loop reads all the elements of A it also transfers these each element into the array C incrementing a particular index here this continues with the array B here. So you enter the elements for array B and as the elements of array B are read by the C in statement here the same elements are also added to the subsequent position of the array C at the end the program outputs merged array C this is where you are required to write the missing code. It is very obvious that you will have to write a loop to answer this question which will output all the elements of C the program ends with written zero and less. Now this quiz could have been said where you have to actually write this code the way this particular quiz has been said is different it gives four possible choices of code that should be written in this missing place here and these choices are named A B C D some of you might have been observing these screens very carefully and would have noted what were the variables used and the array names used try to speculate which of these would be the correct choice and why it would be correct choice. There is a very quick answer now. Yeah, A S K N college what is the right answer over to you. My answer is A, option number A but the condition which is given in the slide is that i is equal to zero and i is less than k it should be there i is less than k should be there not i is greater than k and i plus plus. Okay, let us go back to that code and let us look at the code. Thank you so much your answer is i greater than k for i equal to zero i greater than k i plus plus it should be i less than equal to zero. So, you see a friend has also found an error and also found the plausible answer very clearly N and M cannot be the right answer because these are the number of elements in the array A and B by the way whenever we set a question of a quiz question of this type where we give multiple codes the question becomes quite often very artificial and of course it is important to avoid the errors such as the one which our friend has pointed out but anyway this is one type of quiz questions that can be quiz question can be asked. So, the correct option is A with the syntax error corrected in that particular answer. We now come to discuss some test questions. Incidentally all the sample test questions which I have expanded in the slides have been put together in the form of simple text files and they have been uploaded on the Moodle. So, in the afternoon session I will describe what exactly your activities would be there is a small modification you will of course work on the planning of the activity of your team project but there is something else that you would do namely use all the samples that we have uploaded sit on your board blocks and run on and try to execute them and study the variations possible. So, here is one sample test question this is a rather simple type of test question which requires you to add terms of a series they are commonly asked numerical analysis questions here. So, I will I am just taking this as an example all of you would be familiar with this the problem here is that the series is given as 1 minus x square plus x raise to 4 minus x raise to 6 plus x raise to 8 etcetera. Many of you would recognize this to be a familiar series. However, for a programmer what is important is not to necessarily to understand the mathematical significance of this series but to understand how to write a program to calculate the sum of so many terms of this series. The question stipulates not just adding n terms that is something which is a very common sense problem and is a solved problem in most of the books one kind of the other series is some like that. Here we have added a small squiggle which says another value delta is also given as an additional input. Now, delta defines the lower limit that means at any point in time when I look at successive terms if a term is smaller than delta then I am supposed to neglect all the remaining terms. Even though I might not have added all n terms if the series the values of terms go down so drastically that my delta threshold is reached earlier I will stop submission there. So, note the wording of this question delta defines the lower limit on terms to be added. The summation process the summation process requires add n terms of the following series. The summation process stops earlier if a term being added is less than delta. Now, for us at least the solution is obvious we will set up an iteration to successfully evaluate the terms of this series and add them into a growing sum. However, we will also check whether the particular term is less than delta or not if it is we will quit the iteration. Here is a program which effectively does the same thing I define the value a variable sum term delta x integer n k etcetera etcetera and I read the value of n value of x and value of delta. So, these are the input steps one thing I check right at the beginning and what is that thing is x greater than equal to 1. Let me go back to the problem and read this carefully it is required to add n terms of the following series for a given value of x less than 1. Note that many students will make an assumption that since I have said for a given value of x less than 1 that only a smaller value will be given. One of the important aspects of programming that must be taught to all our students is that they should programmatically verify whether the input data supplied meets the specification or not. If it does not they are expected to terminate the program giving some error message. This is the discipline by the way which should be inculcated amongst our students right from the beginning. It is considered a mandatory discipline of professional programming and there is no harm if all our students understand that mandatory discipline. In fact in IIT we would generally allocate certain marks for checking whether the students have taken care of validating the input or not. So here you just check if x is greater than equal to 1. The command says if x is beyond limit stop execution get off. If it is within limits then you set up the iteration. Notice that because this is an explanatory kind of program that I have written here I have written the entire algorithm as a series of commands in the program. Such commands incidentally also serve as what we call inline documentation. There is a program documentation which is written outside the program but very often programs also need to be inline documented properly. So that if somebody is reading only the program and not the documentation the program still makes sense to the reader. So here you say let us say the first term is 1 that is what 1 plus the series starts in that fashion. Set sum to 0, set a counter variable to 1 and then at the beginning of every iteration check if k is less than or equal to n or alternately whether the absolute value of that term is less than delta. Please remember delta is the threshold, the terms alternate in sign. So just checking whether a value is less than delta in absolute terms or not is not correct. The value may be very large but it may be negative. It may turn out to be less than delta. So absolute value needs to be checked. Now this is what a person would do if a person thinks before starting to write the program. Unfortunately most of us fail to inculcate this discipline in our students particularly during an examination type when the student sees this problem, the student reads this problem and says oh I have to add n terms of this series. I know this and the student simply starts writing without looking at the specific points which are mentioned here, the conditions which are mentioned here which might make this particular summation a bit different to handle as it indeed is. The program is of course very simple. I set sum equal to 0, I set up an iteration. The cracks of the algorithm logically is this condition. That k should be less than equal to n and the absolute value of the term should be greater than delta. If I have already crossed n terms, I must stop. If I have not completed adding n terms but the absolute value of term becomes greater than delta, I must quit. So while this is less than equal to n and while the absolute value is greater than delta, I must continue doing this. Please note the function used is f a b s. The floating point absolute value is typically determined by f a b s and not a b s. Sum is equal to sum plus term, term is equal to term into minus x star x. By the way innovative students who think differently can come up with very exciting possible solutions for this and to evaluate them manually sometimes is both a pain and fun. I remember for example that one student had answered this question by checking whether k is odd or even. He would calculate the term as merely x square and will negate that term if k was odd and keep that term as positive if the k was even etcetera. But whichever way you do this increment the value of k, complete the iteration and output the value. This is a simple program to write. We do indeed must insert such programs because there are a large number of students who have understood the basics of programming but not really the complexity. As I said at the beginning we should be able to ensure that students who have learned basic programming should be able to clear the course with reasonably good mark.