 And for a case study in this section, we have Professor Sahana Murthy, who teaches in the interdisciplinary program in education technology at IIT Bombay. Professor Murthy's focus has been on exploring how learning can be enhanced through the use of technology. Towards this, she and her team have been designing, implementing and evaluating different kinds of learning systems in the engineering and science disciplines. Many of her students and colleagues have explored different forms of research to understand how people learn, how they solve problems and how they make sense of complex phenomena. They have also employed ethnographic methods for this. Professor Murthy will share with us one such project in which her student used cognitive ethnography to learn how scientists solve a hypothetical problem. Sounds intriguing, doesn't it? Let us hear more about this from Professor Murthy. How does cognitive ethnography compare with traditional ethnography? How is it similar and how is it different? Cognitive ethnography is rooted in the practices of traditional ethnography. It employs some of the traditional ethnographic methods such as participant observation, interviewing, artifact analysis and so on. The goals, the broad goals at least are to understand a cultural group or a community of practice and to build a knowledge base which is then used to understand certain specific episodes of activity. And all this happens in real world settings. Cognitive ethnography extends and sometimes even deviates from traditional ethnography in a couple of ways. One is in the usage of media to record and interpret and analyze the data. While field observations are used similar to the traditional methods, digital media such as recorders, film, screen captures, these are very common in cognitive ethnography. One is because of their wide availability and usage these days. And secondly, these media afford interpreting a particular episode from multiple perspectives. Another difference is in the purpose of cognitive ethnography which is how people create meanings, how people use various resources in their activities. And this is slightly different from the traditional ethnographic purpose in the sense it's not only about or it's not about what people do and trying to understand why people do it but the process underlying their activities, the how part of it. In order to go a little deeper into the methods and tools for engaging with participants, for recording and representing of data, for interpreting and analyzing data, let's look at a specific case. The specific thinking skill that we wanted to focus on in this project is what's called engineering estimation. Estimation is an ill-structured problem. Not all information is known, yet the engineer or the engineering student has to solve this problem using various principles and using various techniques. Expert engineers routinely solve such problems. They use it to select certain materials for design. Sometimes preliminary design estimates are made using this method. Estimation is used to determine the feasibility of a certain solution or to eliminate certain candidate solution. So it's a wide practice that experts do. So given this context, the goals of our research was to develop an understanding of estimation problem solving processes. And what I mean by understanding here is the cognitive processes as well as the resources people use, the various strategies and so on. The project is based on the PhD thesis of my former student Dr. Aditi Kothyal. So our solution approach had various stages, various iterations like any research projects and cognitive ethnography appeared early on here. After some preliminary literature survey, we got some conjectures about how people do estimation, but there's really not much knowledge about it. So we used this method to study experts to understand their processes of engineering estimation, that is to identify and characterize authentic expert practice. Our guiding research questions were what are the practices that experts use while doing estimation and what cognitive processes lead to good estimates. And by some sort of order of magnitude, good estimates by these experts. So these were our questions. And for this, we chose two experts. These were experienced engineers. They had several years of experience both in industry and academia. And we gave them three problems to solve each, three estimation problems. Now why did we do this? Why did we choose such a setting and so on? So it turns out that if we want to understand how experts do something, you know, they're really good at it. And if we go and ask them, hey, how do you do it? We ask them the question, how do you do it? We get answers which are really not that useful because of what's called expert blind spot sometimes. It's very hard to reflect on one's own practice and also get at what difficulties students have and so on. So what we do in our research is we give problems or we give tasks to experts which are challenging for them at their level. And then we study how they do it. Some of these studies go on for hours, sometimes it's days. It's often in authentic settings. I'll come to that. And then we use it to reconstruct the cognitive processes of experts. So in that sense, cognitive ethnography is a really suitable method here. The problems we gave these experts were of the power estimation type. So one of the interesting problems was that can a human heart power a wine opener, an electric wine opener? We asked these experts where they wanted to solve these problems and we did them one by one individually. And both of them said they wanted to do it in their respective offices. We gave them the problem on a sheet of paper. We gave them many other sheets of paper. We asked them to solve the problem. They could write. They could refer to anything, books, the computer, internet. They could solve it in their natural mode. If they wanted to be silent, that was fine. If they wanted to talk, that was fine. Often there are things on their table which are used very naturally. It might be a little toy that they take and turn in this manner. It may be a little piece of note paper. So in terms of authentic practice or authentic location, the office is in fact a natural setting for academics. So at an abstract level, this is true to the primary goals of ethnography of studying people in their natural setting. In different contexts, different settings are natural. So this was an important point in our method. We recorded multiple forms and multiple types of data. There were video recordings. There were screen capture logs. So when one of our participants went onto her laptop and looked up a lot of things on the World Wide Web, there was something that captured all her, the pages she visited, what she typed and so on. There were researchers' observations, field notes and especially marking what could be a critical event. And I'll come to that especially when I talk about analysis. There were participant-generated artifacts. And in this case, it was mostly what they drew or wrote on the sheets like the pictures you see. So one of the persons tried to model it as a system of police, the wine opener and the heart beating. Sometimes they wrote equations, another person modeled it in terms of a pump. So a lot of the drawings were the artifacts that we captured. And then we did what's called stimulated recall interviews. So these are interviews with participants on why they did what they did, but we didn't interrupt them. These were retrospective in nature. So after they finished solving the problem, it was anywhere between one and a half, two, two and a half hours, we showed them some of the screen captures and the film and we asked them specific questions. What made you use this equation? What made you think of a system of police to try to probe why they did what they did? To analyze the data, we use a method called microgenetic analysis. And I won't get into too many details here. It can get a little technical. We used a software called Elan and there are many equivalent ones. There is Atlas.ti and a few others. Elan worked very well because we had multiple sources of data and we could synchronize the various data sources with the timestamp and many other things. You can see the screenshot of one Elan screen. The steps that the researcher took was after getting familiar with the data, she tried to identify the critical events. So let me give you an example here. She saw in the video that for a long time, the participant was sort of staring off into space. Not doing much, just staring. And at one point, right after that, the participant jumped in and did a search for animations of pumps. So there was a jump between the actions and this was a critical event because we wanted to understand what triggered the pump, the idea of a pump. And we probed more there. There were transcriptions of the data done within Elan. And after that, for each participant, a workflow, it's like a block diagram of the main critical events was created. And this is an example that you can see. The analysis progresses. After that, there is identifying of the key processes. There is abstraction of those processes. Some literature can come in at this point. And then there are some validity checks. So this is a very careful, painstaking and iterative process. So what did we find and why was ethnography so important here? There were one or two key results from the understanding of learning perspective. One of it was that estimation is actually a three-phase model-based reasoning process. And I'm going to skip this because there's, again, a lot of detail here. But what I would like to say is that this was something that wasn't known in the theory so far. An interesting finding that we saw was in trying to understand what sort of cognitive mechanisms participants use. So we saw that they use gestures a lot and they use diagrams a lot. And when we probed them in the retrospective interviews, we found that what they were trying to do in their minds was to form a simplified version, which means they were trying to model the heart. One participant said that the heart is a pump and that's why she had the pump and the equations related to it. Another participant said, look, the heart beats. So it goes back and forth. So he used this gesture or sometimes this gesture. You can see that in the picture. So we zoomed in to those screenshots, we magnified it and we tried to understand why exactly are they moving their hands in such a way. And we found that these gestures were in fact a stand-in for the modeling that they were doing in their minds. This is also called mental simulation. What did we conclude from this and what were its implications? So when we started off, we had only some broad conjectures on how estimation happens because we had some idea of how problem solving happens. But we had to do these several hours of recording and several dozens of hours of analysis to understand the details, the deeper cognitive mechanisms of how experts did estimation. Some of our conjectures were validated and many got new details. We also got some new insights. For example, we found that experts use what are called metacognitive processes a lot, reflection, evaluation. So they would calculate something and immediately before anybody else could ask them anything, they would say, hey, does this make sense? So these are called metacognitive processes. There was a shift in some understanding. So for example, we all know that domain knowledge, disciplinary concepts are important. But why are they important and what do people use it for? What do experts use it for? What we found is that experts are using this knowledge to contextualize the problem into the actual physical system. It was not that the knowledge was just being used as a set of facts of principles, but the contextualization was where the deep disciplinary concepts were playing a role. This set of insights, this shift in understanding was possible only because of the really minute, detailed and long engagement with the participants and with the multimodal data. And cognitive ethnography is really all of this. That was a really absorbing presentation and a new way of applying ethnographic research. Through this course, we have discussed the purpose of ethnography as a means to understand the meanings that participants assign to a phenomena. Professor Murthy's presentation takes us a step further. In the projects that she described, ethnography is used to understand how participants make meaning. Even in this new domain of cognitive ethnography, we can identify some of the fundamental principles that we have discussed. For instance, think of how the researcher observed her participants as they thought through the problem they had been asked to address. She observed their visible behavior. This behavior was quite subtle for the most part. People thinking, people speaking to themselves, making gestures or scribbling. It is through observing this behavior that she tried to understand their process of problem solving. This also reminds me of something we had discussed while talking about ethnographic interviews and observations. Which is the importance of paying attention to people's silences and non-verbal expressions. Recall the moment where the participant moved from staring into space to searching for a pump. It is these moments that became critical points for analysis for the researcher. These were for the most part moments of silence and non-verbal expressions. This project shows us how such expressions can act as a window into a person's innermost processes of thinking. In our next section, Professor Alka Himurani will share with us her experiences of using ethnographic research in a participatory and collaborative way.