 I am Rinald Patwardhan, faculty from DGC and the College of Engineering. I have teaching experience of around 15 years or so. Being a faculty, I have always asked these questions to myself, are my students learning? Can something be done to improve their learning and what can be done to improve their learning? Obviously, there is always scope for improvement. So, I was sure that there must be some ways of improving the learning that happens with the students. In fact, these are the questions which have motivated me to take this new rule from moving from E.T. Practitioner to E.T. Researcher. Now, what was the problem that I was trying to address? We know that engineering curriculum caters to numerous concepts, process, procedures and principles. Not only that, it also addresses to varying difficulty level of the content and from simple to abstract concepts and from understanding to designing aspects. What I mean to say is, we expect our students to prepare a truth table for a given logic circuit at the same time or eventually, we expect students to design a single board micro-computer system as a part of embedded design system. We want students to understand simple IV characteristics of a PN diode and also we want students to understand the fabrication details related to VLSI or radiation pattern of an antenna. So, in fact, the scope is really very large. Interactive visualizations in engineering education have proven to be very friendship. They help students in understanding the things which they are not able to see sometimes, some unseen phenomena or the concepts which are really abstract in nature. It is a proven fact that interactive visualizations offer better learning in these scenarios, especially when while learning scientific concepts, processes and principles, intensive chemical processes, chemical reactions or maybe molecular nature of the subject. All these things have in these are the areas where interactive visualizations have proven to be beneficial. No wonder this is being adopted as an important instructional strategy for achieving effective learning. However, there is a inconsistency in the reported results. Not that every topic from engineering curriculum or every scenario has resulted into positive learning benefits as far as interactive visualizations are concerned and in fact, this is something has motivated me to take up this research. So, will interactive visualization promote effective learning for all levels of learning? That is the question I am trying to answer here. This problem is very important because we are focusing on all levels of learning that the students we want to undergo. So, here as we can see that we have to, we want our students to go from understand to apply and to create, we want students to create something. So, when we talk about all these levels of learning, can interactive visualizations will be effective in a similar manner? That is something that I am trying to answer as a part of this research. This work is being done in the context of signals and systems. So, what is my idea or what is the solution that I am trying to propose? We have interactive visualizations which we want to make them, we want to have their effective use in the curriculum or in the teaching learning process. So, identify the potential topics, identify develop interactive visualization, develop assessment tool, give this treatment and analyze the learning impact. Now, here are few terms which I will be explaining subsequently in my presentation. So, if this is the idea or this is the solution approach that I am going to take ahead, it was worth answering this question, is my idea novel, is the solution approach that I am proposing, does it have novelty? So, learning happens at different levels. Let me elaborate this, we know that cognitive efforts needed for remembering a formula from students perspective will be different when same student is asked to design a decade counter or maybe the student is engaged into solving a differential equation. If we look at student as a user, we will find that students get involved into various kinds of activity when they are interacting with interactive visualization. What it means? Sometimes they may be taking very passive role, they must be, they might be observing just the animation which is being played in front of the screen. They might be varying some of the parameters which interactive visualization environment offers to them or it could be they might be creating some block diagrams or some sort of design system maybe using sister's softwares like MATLAB or SYLAB or similar. So, interactive visualizations do offer these variations when students are interacting with them. So, we see here that there are different kinds of visualizations, there are different kinds of learning tasks and there are different kinds of knowledge types. For example, when we say learning tasks as I have explained it, it could be just remembering a formula, it could be solving a numerical or it could be creating something. When you say knowledge type, it could be just a concept that they are trying to understand or it could be a principle that they are trying to learn or they are trying to establish from engineering curriculum. So, in a way as a solution approach here, we want to establish connectivity between these three important topics or three important aspects of interactive visualizations which ultimately play their role in deciding how much effective learning is happening from interactive visualizations. So, how do I position my idea now? By positioning my idea, the literature survey that was done that has helped me in understanding and exploring the interactive visualization resource space. So, in order to understand what are the various cognitive levels, Bloom's taxonomy was referred to, also there are different types of knowledge types, there are different kinds of interactivity levels which have been defined in interactive visualizations. So, all this literature has helped me in positioning my work in the context of the research space where interactive visualizations are being explored. Now, as I have said it earlier, all this work I am trying to during all this work, I am trying to focus on the context of signals and systems. So, therefore, review of signals and systems education research has also helped me a lot. Now, at this point of time, I would just like to share with you that in engineering stream, there are many branches, many courses which are being taught. So, in some of these areas, there is a concept called as disciplinary research where faculty involved into typical course, they are trying to come up and they are trying to point out what are the problems that students face while learning typical topics. So, this is what we call as disciplinary research. So, like that in signals and systems, few researchers, few people from academic, they have tried to come up with some of the problems that students have while learning some of the very fundamental concepts from signals and systems. Now, it was the time for me to implement my idea using a very systematic approach. So, to start with I have said identified topic. Now, here when I say identify topic, what I want to make it very clear that when we say that interactive visualizations do offer learning gain, it does not mean that each and every topic in engineering curriculum can be taught by using interactive visualization. There are certain attributes that the topic should have where interactive visualizations will be helpful while using them as a part of teaching learning process. So, therefore, some amount of time and some efforts are definitely required in finding out whether the topic has potential for making use of interactive visualization. Define learning objective was the next task. Learning objective is a something where we try to define what is that exactly we want our students to learn. To give you an example, suppose I am talk I am planning to take a topic of sampling from signals and systems. What is that exactly I would expect my students to learn at the end of this when I am covered this topic in the class or how am I going to assess students understanding by in this in this state topic. So, that is what we try to define by means of learning objectives where we make it very clear for ourselves as a faculty as well as for students that what is that they should be learning, how their performance is going to get measured or when they claim that they know something or they have understood sampling what is that they should be knowing. So, define learning objectives and next was identify or develop interactive visualizations. Now, the last point which has been mentioned here on this slide is very important that is establish suitability of visualization for given learning objectives. It is very essential that the interactive visualization that which we want to use while teaching this topic should be in line or should be aligned with the learning objectives those are being set for the set topic. After we are through with this it was now time to finalize the research design. So, as a part of research design the details of that I would be giving you after this. Assessment test instrument was developed just a lecture before our land station we have understood what is the importance of having a very appropriate or very reliable and valid assessment test instrument. So, development of assessment test instrument was the next task establishing validity of the developed instrument and so that means now we are in a position to actually implement the plan that we have proposed. When we come to actual implementation the very first obviously we identify the students on which we want to conduct this experiment. It is very essential that we establish group equivalence give treatment these are the typical words which are being used in research methodology in education research that now we want students to learn from the different types of visualizations analyze the results and then look back and try to answer has the solution work. Now it is very important to see that the solution that was proposed could find its base or its foundation from the various theories which are existing. So, the first point what you could see here on this slide is when to use interactive visualization. It is very clear that if the topic does not have the potential for making use of interactive visualization one cannot expect effective learning results from the experiment. So, obviously the research literature which is related to interactive visualizations and the attributes of topic that where we can make use of that was studied. Significant systems contained analysis and education research related literature was again referred to. Multimedia based principles are very important for effective learning. So, those were referred to because ultimately we are talking about a computer based interactive learning environment. And also this varying level of interactivity in visualization and what is going to be its impact on cognitive engagement of the students. Will students be using the same amount of cognitive efforts if they just look at the animation or if they are going to create a model by using simulink that is something what we mean by the last point which has been mentioned over here. Now it was time to find out does my idea of whatever solution that has been proposed is it working or not. So, I would say that this entire research experiment process can be broadly divided into three steps plan implement and analyze. By means of plan we expect to come we are expected to come up with research design details. Second part implement is actual we try to conduct the experiment and analyzes whatever results we get from the experiment. From that we try to infer from that we try to find out what are the findings. Now as my through from my experience I would like to suggest here that the more amount of time we spend on the first step that is the planning step and more careful we are in the first step itself that is the planning that will decide what is going to be the overall success or what is going to be the overall effectiveness of the experiment that we are conducting. So, here itself we will try to see what could be we have to anticipate what could be the problems that we may be facing as a researcher what could be the confounding variables which might come in your result analysis we have to keep them in mind and our plan has to be full proof from that angle. So, these are some of the details of the experiment that was conducted. See the part of this experiment I was interested in finding out how much are the results or what is the test score of the results in the assessment test that will be administered when students learn with interactive visualizations that has different level of interactivity. So, as you can understand here visualizations with different level of interactivity will become independent variable of my study and the test scores in the assessment test will become dependent variable of my study. It was a research design that was planned with two group pre-test and post-test design and the sample where the students from third year of engineering affiliated to colleges from Mumbai University. So, the students were divided into two random groups where they would be learning the same content the same topic, but by using different kinds of visualization and the test instrument was obviously developed in order to measure how much will be the learning of students. So, in short this could be the summary of the research design that was planned you could see here, here is group one group two. So, that is what I mean by when students were randomly assigned to group one and group two. So, it is a two group pre-test post-test design pre-test was conducted after that treatment was given. Now, by treatment what we mean here is that students were asked to learn the given topic from a kind of visualization. So, for group A that visualization happened to be in the form of animation whereas for group two that visualization was a visualization where variable manipulation was provided. So, as I have said both the groups were given pre-test initially to know their knowledge level in the given topic. So, for example, from the multiple experiments that I had conducted one of the topic was graphical interpretation of convolution in signals and systems. The groups were given treatment one group learnt from animation whereas other group learnt from visualization that has variable manipulation and the learning impact of the treatment was just by conducting a post-test. Now, this part I think you know is very important in this overall research process that what all did I have to worry about? The very first step that is group formation. So, one has to be very careful when we go about this group formation. Though there was pre-test planned for as a part of this research design still it was seen that or it was considered that the students are unfamiliar with the topic that is they do not have prior knowledge related to that particular topic. Group equivalence was established on the basis of their previous academic scores. So, it could be maybe say semester 3 examination results or semester 4 examination results. The point here is that we need to compare them on the same scale and not only that group matching was done with respect to taking into consideration the gender issue as well. Second question that needs to be answered is, is my visualization selection appropriate keeping in mind the learning objective? If my interactive visualization is not going to offer an opportunity for students to learn the topic for which the learning objectives have been designed the entire exercise will be of no use. So, the visualization that was selected was validated by domain experts to check its appropriateness for the design learning objectives. For example, if you do a very quick Google search you will find that for a topic of convolution there could be at least 10 to 12 different interactive visualizations you may get. But you will have to see that if at all you are using it with certain learning objectives in mind are those visualizations appropriate for the learning objectives that you have planned. Second point is usability study conducted was conducted and so that usability aspect was also studied. So, what it means that if at all a visualization that has been defined, if it is very poorly designed and if you do not get results you cannot be sure that the visualization has not worked it could be just because the visualization was not designed in a proper way. Next question or next area that needs to be explored is, is my assessment tool measuring what I intend to? Now here since as I have said I was working in the domain of signal sense systems and I was trying to address various cognitive levels such as apply or understand and different content types for example concept or process. So, it was essential that I set my assessment tool or my questions are set in such a way that they will be trying to judge, they will be trying to evaluate whether students are able to apply the learned knowledge or are they understanding some concept basically. Hence carefully this assessment tool was designed and so which had questions which targeted at various cognitive levels as well as for different content types. This assessment instrument was validated by not only by domain experts but from by ET experts also to establish its validity. The relevant feedback which was given by the expert was used for modifying the assessment instrument. So, for some amount of time it happened to an iterative process where whatever feedback was given by the experts that was taken into consideration and based on that assessment test instrument was getting revised and modified. So, here maybe you will be able to see clearly or later on you can refer to this. Here are some snapshots of the questions those were set for various cognitive levels and various content type related to topic let us say signal transformation or convolution from signals and systems domain. Here are some snapshots of the interactive visualizations those were used as a part of this research study. This is related to signal transformation where here in this part you could see that this visualization allows variable manipulation. Here is another one in the in the topic related to topic convolution. Now here again variable manipulation is being one of the features of this visualization. How I collected and analyzed my data. So, going back to the research design what I had we could clearly see here is group 1, group 2. Both the groups were administered with pre-test. They both the groups received different treatment and later on again a post-test was conducted. So, here are two questions those need to be answered. First of all has the treatment worked within the group and then it is time to identify or it is time to analyze that which treatment is better as compared to the other. So, obviously in order to answer these questions it was necessary first of all to find out how much has been the learning gain for individual groups. For example the difference between this post-test and this pre-test will give me how much learning has happened because of this treatment. Whereas the same exercise when it is done for group 2 will give me some idea about how much has been the learning gain for second group when I try to find out the difference between this post-test results and this pre-test results. And now so what we will have in our hand will be learning gain when students study with animation and average learning gain when variable manipulation was one of the features of interactive visualization. So, here is the important question that I need to answer that how did I know which group performed better. So, the learning gain average was compared for both the groups separately for assessment questions belonging to different cognitive levels. So, it was not just the final score that was important for me to analyze, but it was individual score that was more important. What I mean to say is for example how much students are learning or how much students are scoring in the questions those are addressed for let us say understand cognitive level or how much has been the score of the students for the questions which are catering to let us say apply cognitive level. Like that there were questions for different content time. So, what was important here not just to look at the final test scores, but more important was to look into these individual scores where they try to give us idea about how much learning is happening for this individual cognitive levels or different cognitive levels and different content types. So, as I have said now I need to answer this question as a part of this research work that did the treatment work. So, statistical tools were used for analyzing these results further. So, there is a statistical test called as paired sample t-taste which was used to compare the statistical significance of the difference between the post-taste and the pre-taste for individual groups. Next question that I need to answer is which treatment worked better. So, now here it will be independent sample t-taste that could be used for comparing the learning gain average for both the groups. This will allow me to answer the question that which treatment has worked better. So, this is a typical table which tries to show us the results of a test called as ANACOVA. Another form of representing results could be in the form of graphical representation. Now here you all could refer to this legend that has been provided. For example, AP here stands for apply process that is those are the scores for the question, those were targeting for apply cognitive level and say process content type. Similarly, for understand process IL-1 here corresponds to interactive visualization which was just in the form of animation and visualization IL-3 corresponds to visualization where variable manipulation was provided. So, here you could see here is the comparison happening between what has been the scores of both the groups in the pre-taste as well as in the post-taste. If we look further into this typical graph, we could see that there is a difference between the pre-taste results. As I have said earlier that when the groups were created, the groups were created on the basis of randomized assignment, their semester 3 results were taken into concentration and both the groups have similar averages or statistically same averages. In spite of that, the pre-taste results has shown something different. So, in a way this was suggesting that both the groups are starting at different level and therefore, one has to take into concentration a concept called as non-equivalent group design. So, researcher has to be very, very careful in analyzing these results because it is it will be very wrong just to comment without taking this into concentration that both the groups are starting at different point. So, whenever we need to analyze such results, we have to be very careful. We will have to find out what is whether these two points or the pre-taste scores are they statistically significant or are they statistically same or they are different and based on that the research design might get modified. So, as a when this experiment was conducted, one I could see that for apply level cognitive task and process related content type, variable the groups or the students those learnt with variable manipulation visualization, they perform better as compared to other group. Similarly, for the understand level cognitive task, I could see that the visualization in the form of variable manipulation and visualization in the form of animation. The scores the students get they were statistically same. So, in a way it suggested that for understand level perhaps interactive in interactive visualization variable manipulation may not be necessary. So, after getting these results now it is time to look into further and try to go back from the point from where we have started that did my idea really work. So, my idea was to use interactive visualization to achieve effective learning while dealing with abstract concepts in the domain of signals and systems. Now, keeping in mind I need to take into consideration what are the cognitive levels that I am addressing and as well as what happened the content types. So, results shown here that different amount of interactivity in visualization will assure effective learning while catering to different kinds of levels. So, for this typical topic, it was obvious that if higher level of interactivity is provided in interactive visualization for that will prove to be effective for higher cognitive levels. So, these results are contributing towards ensuring effective learning from interactive visualization. So, as a faculty when I have to make this decision that which interactive visualization or what kind of visualization I should be using. So, that my students learn in the most effective manner these results are going to help me not only that, but for the people who are developing these visualizations or if faculty is interested in developing these visualization. What should be the learning objectives and how the alignment of interactive visualization should happen with this learning objective that can be understood with the help of these results. Thank you.