 So, I would like to thank Sameer first, he touched upon two things, user research and the importance of a minimal viable feature, kind of a prototype. In this particular project presentation, I am going to talk about my learnings and how I have always tried ethnography and see how it feeds up to the data that we obtained from our different product. I am an HCI researcher and a designer working with Zerox and I have worked on domains in health care, education and I am going to take you through my presentation and learnings using a case study which we recently concluded in which we actually did user research with caregivers and patients with chronic chronic illness for a long time and on the way during the user research we actually build a minimal viable prototype so to say which was purely meant as a technical probe during the research process and it actually led to all the ethnography insights feeding it into design of our data analytics framework. So, looking at the typical human centered design process, our project actually began with going to fields, doing some observations, maybe doing secondary research, reading blogs, reading research papers in the same domain, then collecting stories in the field, trying to derive some themes out of it and what could be converted into some kind of a product implication or opportunity areas that we could actually propose to different product managers. Then prototype some solutions that could be actually presented in a product showcase workshop kind of thing, test them and implement them and during this process actually you start building with a concrete focus during the research somewhere in between it becomes abstract and you try to find out abstract themes around it and once you have identified certain themes it again becomes concrete. So, our ethnography study was close to around 6 to 7 months and somewhere between we actually thought that the insights that we are getting from the field we should actually build an evidential proof for that and that is when we actually build the data ecosystem. Now, I will actually go to the details of it but this is something that I try in other projects as well and I will give you an example around that but during the end when the research was still in progress we started getting this ontological understanding. So, people who are here from information science or computer science would understand ontology it is nothing but a thing in universe and its relationship with other things and the title of my presentation talks about user knowledge generation and that is not user data what I really mean is having an ontological understanding of the user. If you speak about one user how do you identify different things, different other users in the ecosystem, different type of product and how the user actually interacts with them and gets affected by them that is that is user knowledge generation and I am going to talk about how an ontological understanding was built based on the data analytics framework. So, briefly touching about ethnography what we do these are the two different type of ethnography processes followed across globally one is participant observation in which researchers actually go to the field very unstructured approach they end up taking hours and hours of collecting hours of video data, audio data, transcripts they speak with users again very unstructured, build journey map, space diagrams and then bring back to their labs and analyze them. The second is key informant interviews in which you actually identify a purpose sample they may be experts of your user or potential users of your product and you actually do interview conduct interviews with them. This is more of a structured approach and sometime you miss out on some of the context so there are like other terms other researchers do they call it contextual inquiry and then there is third which is focus group study which brings in good elements of both the approaches in which you have a more snowball kind of a sample coming together and you try to build a context which is similar to the context you want to do your participant observation you give them certain task may be have them run through an exercise or a competition and during the process you actually talk with them and collect data. So, ethnography is actually like writing about people and when your researchers go into the field do all of these things they collect a lot of subjective data, subjective in terms of interpretation because every user is different. Whereas when we talk about data analytics now this particular term thick data was actually first mentioned by Tricia Wong an ex frog researcher and she actually told that big data which is really the archive this hard coded objective data about the user they talk they give quantitative numbers they actually rely on machine learning because you have to actually run through them analyze it for a long hour and they talk about distribution like how the 20 percentiles of the users are using 80 percent of the features or maybe what is the dropout rate and what are the features they are not using whereas when we talk about thick data which really comes from field studies or user research this brings in a lot of qualitative data to the table a deliver stories. So, when we have done ethnography and talk with product managers so and we have always felt this unease when we present these stories during the initial phases of research because they cannot connect with that. So, really the thick data talks about these stories it relies on human learning the more you do research the more you become observant you learn the art of looking and not seeing and it actually tells about arch type when you do ethnography you actually learn about your supermans or batmans or black widows who are going to use their product but when you do actually the actually just the big data rely on just the big data you only get going to get like information about the user. With arch types I mean when we do research when we talk with these different users we always keep this thing back in our mind that all the statements that they are telling we need to build an arch type persona so that we can actually think of all the exhaustive features when we are back in the analysis board. So, in this particular work we were doing ethnography of chronic care ecosystem and we started with secondary research we started looking at the research papers around that to the left like all these papers they actually talk come from social science some from user research actual ethnography conference and they talk about different aspects of how it's not just about patients there is three lines of work involved there are three different type of users when I talk about a patient as a thing in the ecosystem all of these actually talk about that kind of work the kind of different partners involved and when we actually like during the middle phase of this research when we are actually reading about this one of our colleague actually told use the term that it's spaghetti bowl of data now this guy was a principal is a principal interaction designer and he was looking at this particular study that how the insights could be converted into some kind of a available design or a physiological sensor that could feed in to the product suits that we have when we looked at product design and computer human interaction literature they talk about a lot of approaches in which designers have gone into field studied build certain element build certain product and like data usability study build that have that pilot have have users use them and they have reported about different kind of finding but we found one clear pattern that they didn't look at all the aspects that they could gather from the field and more it was like a technologically deterministic point of view somewhere in between they were actually trying to oversimplify so oversimplify in terms the kind of spaghetti bowl of data that was coming from the field real field when we actually talk about just one product it's it kind of depends on the technology that we are selecting while designing the product and kind of simplifies the problem so going back again where to actually where to start all these literatures we have helped us to identify two major basis since we were focusing on building a variable or a physiological sensor kind of product we looked at what are the different self-care behaviors these self-care behaviors are important because end of the day patient is going to wear the design and and it talks about a lot of things these actually come from healthcare research healthy eating being physically active and so on and so forth and we also looked at what the social science research brings in brings into the table that is three lines of work because we wanted our product solution to be really effective so these three lines of work actually talk about the three different categories of users involved in the ecosystem not only just the patient and the kind of professional training required for specific kind of work as compared to a lot of work which doesn't require that kind of professional training so this was our sample we were doing chronic disease management studying chronic disease management in India and particularly talking about India and since we were looking at the ecosystem so the social construct itself is comes into picture because we are mostly joint families we interact with neighbors a lot we have close-knit families we looked at 18 families on three different tier two cities majorly these chronic diseases which are also more over in series in nature because when diabetes come it also brings along hypertension and arthritis and similarly brings along renal discoloration problem brings along brings along hypertension so in all our samples all the 18 families we found that these patients have one or multiple of such symptoms and when we were looking at this since we were looking at the families we were also looking at the kind of caregivers involved staying with the patient or staying away from patient situated so I'm going to use further further use this these term situated caregivers in person caregivers or remote caregivers and what are the different kind of activities and responsibilities exchanges that happen among the caregivers so talking about the interviews this was actually informant interview process that we need to and we identified these 18 families we had to interview the patient as well as the key primary caregiver in the family there were 36 interviews the key theme that we used these and each team would have multiple questions we would go them one and after another or maybe based on the conversation we will switch so this was a more structured questionnaire but as in the previous talk so we told that you keep asking wise so we were making sure that to actually get the new stories we'll keep asking these why be it five times to one statement we'll keep asking these why so that we actually the kind of findings that we get are more contextual in particular to that user so lot of data we got we converted all the audio transcripts audio recordings into these sort of spreadsheet which would mention about the maybe first statement would be just the introduction about the user but following statements will be different responses we got and if responses are related we'll keep right next to each other and that way we had actually for 36 users we had these kind of data now I'm actually going to talk about how we derived themes from that because that's the process part of it and pretty important we coded the data as device techniques so for example then a caregiver mentioned that his father had renal disorder and not allowed to have more than two liters of water a day he was actually giving a two-third of ice with water every time so that was a devised technique that caregiver had built on his own so every statement we obtained when it indicated towards some kind of a workaround users have created on their own to solve some problem we tagged that particular statement as device technique similarly breakdowns breakdown indicated about the failure of the system starting from mild breakdowns which could be just the pain points and the pressure points as the users mentioned but breakdowns are like the major big major failures of the system we tagged certain sentences as contextual interpretations these are the interpretations we as designers were gathering while we are talking and certain design ideas that would come during the affinity analysis we came up with bunch of themes I'm going to talk about just a few briefly but we were actually we took all these sentences mixed them up and then during the analysis we were finding sentences based on similarities so what are these user statements indicating towards the similar context relationship what are these what all of these user statements are indicating were indicating towards the similar outcomes and proximity are there one reason leading to another outcome and vice versa that so that way we were actually tagging and identifying these user statement and that helped us build certain themes so I'm only going to touch upon some of the themes and quickly moving to further details so one theme that came from actual field study is the level of trust and assurance we found out levels of levels in trust and assurance is quite different when the caregiver is actually filial as compared to caregiver who is actually conjugal as in marital partner or filial as in son or daughter taking care of the patient so so so these filial caregivers were seeking continuous assurance as compared to as compared to the partners who were aware of the general general general ecosystem or behavior of the patient because of the sense of that growing old together we found out that filials especially those who were in person staying with staying with patients they had their own way of motivating and persuading the caregiver but that was totally absent when uh when when these caregivers were remote and interesting uh an intriguing fire finding that we had was uh there was these relationship friction because of role reversal so so these patients they had been taking care of the family for a very long time and suddenly the roles have reversed so that has again brought down certain reasons for friction certain reasons for not not feeling self-reliant and the kind of data that that came to us and when we actually put went back to all the data and tried to put it based on the kind of emerging teams we found we came up with this quadrant where we were the filial caregivers and the trust and assurance level were pretty high in this case but they were also a lot of deceiving happening and this was actually again going back to that statement of my colleague that it this was actually a speculative whole update and when we when we intend to show this to somebody who comes with hardcore computer science background uh will not be able to make a sense of it and uh we know that we knew that where there is a interplay of trust and relationship that we have studied uh maybe that is correct or maybe more insights that we have to do but there is also a health data ecosystem that we can capture and get evidences from from the data that we have and the data that we got from ethnography that's when we built this prototype it's a classical example of you are building a technical probe uh that's not going to sell this was just built for the study uh we we had uh some six families volunteer so so this this had simple physiological sensors that would identify certain activities such as uh walking such as eating if you have a tilt action for a repeated number of time RFID readers would actually tell tell about medicine intake followed by a tilt action and and real-time clock will will basically tell about the kind of time of rest the total time of rest for the patient and we had this particular armband uh uh had had had the patients to use this for a period of 10 minutes uh 10 days the department during this time the the the the the the variable was actually gathering data and sending notifications to the caregiver and uh as the data that we were receiving was the call logs from the caregivers what are the kind of activities which had happened at patient end and what are the kind of sms's that were sent to the caregiver are the caregivers responding to these kind of sms's uh if let's say there is a intervention requirement do they actually call back and remind their parents uh they were implicit was it explicit actions that would just happen just because of a message being sent or implicit as a general behavior uh and and we were trying to identify are there any corresponding pattern hidden between the kind of interactions caregiver do when kind of data being sent to them so we had this data ecosystem work with all of these families and we did semi-structured interviews post pilot uh what is interesting is there are certain findings but i'm going to talk about these findings when i talk about challenges uh this kind of a data ecosystem that we had built and the understanding from the ethnography that we need to actually identify all of those things in the ecosystem had led us to build uh build an ontological framework now this particular thing is a screenshot of a ontology uh web ontology language in which uh these five information which is essentially the information about the patient i s d a r whenever a doctor communicates with a nurse certified nursing assistant they they actually tell about the patient in these five and five data point which is identify about the patient age sex uh uh situate about the patient what is the current situation background what was the medicine given to the patient assessment what's the current assessment what is the medicine required and uh response so what the like what is the nurse supposed to do and this is way uh uh in these five term terminologies all the healthcare communication happens so we picked up this healthcare web ontological language and we found out that till this date uh the patient thing information had only these things there will be uh owl files for diabetes there will be owl files for other other diseases other medical complications but they they had never had a caregiver as a part of it this is a simple simplistic screenshot there in which there is only one caregiver and and and this is this is just the wrapper of the database but uh i suggest that yeah it's it's difficult to show the what's going on and what are the different kind of data that could go if we have multiple type of caregiver and if it's uh healthcare service provider location where are where there are multiple caregivers taking care of inpatient as well as outpatients uh this is a very complex data so we we were able to have capture these different users ecosystem and that's where the ethnography finding kind of led us to have uh not only just the patient but uh but but actually a 360 degree feedback working between the data ecosystem as well as the trust and relationship the the findings from the ethnography so i'll give you a certain examples for example uh when when these messages were sent to the caregivers the kind of evening discussions the coffee discussion used to happen the the total period of time actually elongated to almost three times uh this came out from the post pilot interview uh at times so so we had taken care of when the patients don't use the device during the pilot so at times patients were not using it and we we actually sent the message to the caregiver so so that led to some kind of a discussion and uh possibly it was an argument actually so so so so this kind of a technical probe that we had sent to the field had also brought certain behavioral changes uh or or determinant shift to the user behavior so to say so so based on this framework that we have designed probably uh in future we can have data coming in from multiple variables we will have data about knowledge of or experience of caregiving about the caregivers different things in the ecosystem and uh possible medical technologies can actually be uh able to identify the best interventions best communication corresponding to different responsible caregivers in the ecosystem and use that kind of a framework uh as a system in action we were able to uh increase caregivers contextual information to the system so for example one of the findings from ethnography that we had the both are passed from doctor to nurse and let's say a conjugal caregiver if the conjugal caregiver is not an expert and most of the cases she was not an expert she had a lot of misunderstanding she felt more comfortable with the nurse as compared to the doctor because she could talk in the same language with the nurse uh it's just the difference in the experience and the knowledge uh whereas this this was never find uh this was never found with failures as a given when always go back and google if they are not confident about certain things we were able to recommend uh personalized intervention reporting uh so this is just that we are not only talking about the patient but if we have three or multiple caregivers involved in the system uh we could easily identify what kind of activity can be done by a cake and and we eventually also enabled a many too many patient caregivers intention intervention now the most important part in all of this journey where we actually did uh ethnography and somewhere in between we implemented a data ecosystem and how things got came in together is uh you need to you need to actually identify the right sample during your entire study uh we were fortunate enough that actually these six families got agreed for the 10 day long pilot uh but many attempts uh that will not be the case so so so so as researchers we need to actually be careful about that and uh keep persuading and have have have these participants become the important part of the study have them feel that they are the contributor during the study so they can actually be along with you and uh continue helping you as as data point the probe can bring in determinant shift in user behavior now this particular example is just in the caregiving ecosystem for example research uh let's say for e-commerce shopping behavior and we we build a beta prototype and uh and and send it across to certain percentage of you to identify what they are doing with the prototype now sell on its quality or based on the kind of features that we have built can bring in a lot of deterministic behavior so that should not bias the findings that we have and there's always there should be always a very incremental uh shift when we are actually trying to implement any such technical throat in the field uh an important thing during such study is identifying what we are trying to assess from the study now in this particular case all the families found the particular device and the communication mechanism happening pretty useful but we could not report effectiveness of such a system so so so so there are definitely these three triangles when we are trying to study and identify effectiveness of a system is are we going to study the perceived usefulness of a product or a service or are we going to study the adoption of a product or a service or just the effectiveness uh that will actually help us design our research in a much better manner in this particular case we were actually just looking at uh the perceived usefulness and if in future this kind of a data ecosystem exists how it's going to help or or play out in the overall caregiving and fourth and the most important thing is uh most of the time participants will not use your technical probe so so you will have to have ways to identify when it is being not used or when it is being abused so actually you can that also feeds in as a form of data in your research yep thank you this was the team who worked with us any questions hi so uh mostly the ethnography more about uh qualitative research right so uh so i i just want to know like uh how do you so the sample size which you take is very small to do a qualitative research so we started with 18 families generally you can start with uh 7 to 9 yeah and you should always look back and do this affinity analysis till the time you get a theoretical saturation uh oftentimes timeline of the project itself uh works as a constraint but uh i think starting with the reasonable sample size of 9 so how do you uh like how do you manage not to fall under so because most of your results are based on the users right so how do you manage not to fall your results to be more biased toward the sample size that you have taken uh so mostly in these kind of study uh i'll just talk about the study we did a purpose if sampling uh there was strict questionnaire in which we were actually trying to screen these families uh only the families which had more like any one member or more than one member had had these chronic conditions uh had caregivers who have been taking care of the patients for more than five years or so uh so so so we yeah and and as i mentioned in the previous uh when when i was explaining about thick data uh a lot of times some of the interviews we will we will be having these supermans and batman's r-style personas arriving but they could also be outliers in the system so so some of the outliers are easily recognizable when you are doing analysis uh yeah so you should actually not consider it as a part of the user statements while analyzing and another question is uh so ethnography is generally like a long term process like like some people do for like years so how how do you you know how do you keep track on your initial objective and like if like in time it might change like how do you decide okay this is the changing point and have to change my track and then again do how does that happen that's a very good question and mostly uh so so i come from a background of a design research and uh i try to find a middle ground by doing contextual inquiry and because ethnography actually comes from anthropological science in which we had these renowned ethnographers staying with these societies for a very long time they used to write about uh attic perspective like an outsider perspective of the society and then amic as an insider perspective of the society and it's a long very long process but uh in the in the first times that we live now we need to identify and maybe focus on so so some people try to uh try to focus on the kind of problems that they are going to focus on so so yeah there is always a middle ground that you'll have to identify but you can always go for a long term ethnography. Did you have did you have any uh experience in changing your objective and what are you your project towards like was there any uh could you repeat the question? Like before starting that a study you might have a you know a problem statement to solve to solve yes did it change eventually when you are doing your study? Yeah most of the time problem statements change when you do these periodic affinity analysis but you kind of report the holistic findings insights that you have uh and and they should actually change the kind of questionnaires you build after these every periodic analysis the question questionnaire should also evolve because you will every time find a new insight or new new new fruit for thought actually. Hello nice talk so uh the research that you talked about so is it only after the probe the MVP has been made or does it apply even before you know what we want to do say uh in startups right so you can't make the whole thing and then show then the whole if it doesn't if the perceived usefulness is not there then the whole work has been wasted so uh are the techniques that you talked about uh apply even before the product conceptualization? Yeah so I think uh when you are like building a variable you will also have to go through building these kind of MVP uh this particular MVP or was only used for a pilot or a technical probe during interviews but even for product design you will have in not only just one but multiple versions and they will improve during the study okay I'll say your question I mean generally we show designs and they say yeah it's it it would help us but later when the product is rolled out uh it turns out that adoption wasn't good so how do you mean the MVP ultimately turns out to be the product itself so how do you get the right balance between uh the pilot project and not becoming an exhaustive this was just a probe we only had one version but uh let me talk like let's just talk about a website or a web product uh we will have multiple versions they will not be as expensive to build or we we can actually easily get users have them use and get feedback from them so so there is also difference in uh trade off that we have to do the kind of product that we are building sometimes the mobile app could be very easy but let's say we have to build a for example a automobile so it will have an incremental we will have incremental mbt is being built for that and not multiple ones