 Hello and welcome to the eighth webinar of the Engineering Rising to the Challenge Initiative from Purdue Engineering. My name is Arvind Raman, I'm the Executive Associate Dean and the College of Engineering. Now this initiative started in May 2020 partly in response to the National Academy of Engineering's call to action for engineers to tackle some of the challenges posed by the COVID-19 crisis. But our initiative also looks at the longer-term future to rethink and re-engineer the very systems that our modern society has come to depend on so that they might be more resilient to such shocks in the future while also serving society better. Now part of the initiative involves webinars where distinguished panelists unpack some of these challenges and provide us a glimpse into what the future might look like. And today's webinar's topic is how will data science and artificial intelligence reshape the post-pandemic future of health care? A really important and timely topic in these times. And I'd like to introduce the moderator for today's panel, Professor Guang Lin. Guang Lin is an Associate Professor in the School of Mechanical Engineering and the Department of Mathematics at Purdue University. Professor Lin is a Director of Data Science Consulting Service that performs cutting-edge research on data science and provides hands-on consulting support for data analysis and business analytics in all areas to overcome data science challenges arising in research, education, business, and organizational management. His research interests include diverse topics in computational and data science both in algorithms and applications. His main current thrust is in machine learning, data-driven modeling, stochastic simulation, and multi-scale modeling of interconnected physical and biological systems. Professor Lin is currently a member of Purdue Engineering Initiative on Data and Engineering Applications. Professor Lin has been widely recognized for his scholarship. He has received various awards such as the National Science Foundation Career Award, the Mid-Career Sigma Psi Award, the University Faculty Scholar, Mathematical Biosciences Institute Early Career Award, and the Ronald Bradzinski Award for Early Career Exceptional Achievement. Over to you, Guang. Thank you. Thanks, Irene, for the introduction. So, let me share the screen. First of all, click the share. So, we're going to have the share screen. So, okay, let me do the full screen mode. Okay. So, hello, everyone. So, it's a great honor. So, we moderate this webinar on how weather science and AI shape the post-pandemic future of healthcare. It's a great honor. We have five distinguished panelists from various backgrounds. So, let me introduce the first panelist. It's Yuhano Kuluchano. Yuhano received the PhD degree in Electrical Computer Engineering in 2004 from Johns Hopkins University. Since July 2019, Yuhano works as a fellow at Micro. He was an associate professor of the School of Electrical and Computer Engineering, and the Welldown School of Biomedical Engineering, and the School of Mechanical Engineering, and of Psychology Science in the College of Health, Human Science at the Purdue University. Where he directed the ELAP, his research focus is in artificial vision system, deep learning, hardware acceleration of vision algorithms. Yuhano is the recipient of the Presidential Early Career Award of Science and Engineering, we call it PKASE, and the distinguished lecturer of the IEEE. In 2013, Yuhano found the Teradip, a company focused on the design of deep neural network processors. In 2016, Yuhano founded the Forward NEXT to deliver the next generation synthetic brains for artificial intelligence acquired by the Micro. So, the second panelist is Mario Orantresca. Mario is current and associate professor of industry engineering at Purdue University. His broad interests generally fall in the area of computational science and engineering, and across multi-application domains. Prior to joining Purdue, he had a post-star positions at the University of Toronto and Cambridge, which followed the completion of his PhD in system design engineering from University of Waterloo. His undergrad and master degrees are in computer science from Brock and the Gulf University, respectively. So, the third panelist is Mona Flores. Mona is the global head of medical AI at NVIDIA. She brings a unique perspective with her variety of expertise in clinical medicine, medical application, and business. She is a broad certified cardiac surgeon and the player's chief medical officer of a digital health company. She holds an MBA in management information systems and has worked on Wall Street. Her ultimate goal is the betterment of medicine through AI. My other panelist is Tanya Pujo. Tanya is an assistant professor in the School of Industry Engineering at Purdue University. Her PhD is in industry engineering from the Georgia Tech with a concentration in statistics and a minor in biomedical informatics. Her doctoral research focused on analytics and machine learning applied to health data. She leverages methods from data science, statistics, and network science with application to rest populations, including pregnant women, infants, and chronic optic users. She is an awardee of the National Institute of Health Training Grant and the Alfred Sloan scholar fellow and has received various other awards, including from post-sessions and informs. During her PhD, she also had the opportunity to serve as a visiting scholar at Harvard Medical School in the health care policy department, where she completed research in causal inference. So my last panelist is Yao Shen Chen, as the director of vaccine modeling team at economic data service of the Center for Observational and the Real World Evidence within Merck Research Laboratory. Dr. Chen constructs, calibrates, and validates sophisticated infectious disease models that integrate both clinical trial and the real world data. Dr. Chen's major technical responsibilities include applying modeling techniques to evaluate the health economic impact of various vaccine strategies as well as to inform indication, site selection for vaccine clinical trial planning. Since join Merck, nearly three years ago, Dr. Chen has led development of analytical modeling activities for vaccines. So let's include COVID-19 as well. So let's welcome the four panelists. So today, the topic will be the Harvard Data Science and AI shape the post-pandemic future of health. So the 2020 pandemic has challenged our health care system, but it has also opened the door to a greater acceptance of the virtual clinical visit and AI-enabled automation. So what were the future of health care looks like in the AI era and how data science and AI will shape the post-pandemic future of health care? So today's topic is also related to two of the initiatives that are going on at the College of Engineering. One is the initiative in data engineering applications that myself and Mary are part of this initiative. And the other initiative is the initiative in the engineering medicine. So today, the following, we're going to talk about some addressing some important topics related to this future, how the AI and the data science can affect the future post-pandemic health care. So the topic I want to address is how AI and the data science can help with a vaccine discovery, clinical trial, and the vaccination implementation planning. So Yaxuan, can you help address this topic? Sure. I just want to test. Can I, do I sound okay? Yes. Yes, okay. See, Tong-Yuan is nodding her head. Thank you. So before attempting to us answer this quite rich and complex questions, let me try to briefly give an overview of my work related specifically to these questions. So my work in the vaccine modeling team at the center of observational and real-world evidence, we think the Merck research laboratories is mainly we construct and validate very sophisticated models. Some will call its AI model, some will call infectious disease models. I think those terms can be understood interchangeably in a relaxed manner. So we construct this kind of models for vaccines that integrate both clinical trial data as well as the real-world patient's behavior data into the models. And we think my company, the models we constructed are later usually applied to evaluate economy and health impact of various vaccines and their vaccination strategy implementations in the countries that we identify as a key market. And sometimes this kind of models will also be used to inform the vaccine trial indications as well as the site selection for the purpose of planning large-scale trials in the midst of a lot of uncertainty. For example, in the COVID-19 outbreak where one can really know which outbreak will be or how large it will be. And before joining Merck where I work right now, I actually work at the U.S. Centers for Disease Control and Prevention. Part of my job there at the time was to apply these relevant and similar modeling techniques to assist the agency's public health emergency outbreak response. And I have what was involved in a various of, I will say, disease outbreak that including Ebola, pandemic influenza planning, as well as HIV outbreak that happened back in Indiana in 2016, I believe. So actually give a very short answer to the questions that Juan just proposed. It's like how AI and data science can help with vaccine discovery, clinical trials, and vaccination implementation planning. My short answer is that it is actually a very challenging process from the discovery proof. It's, and also attempting to prove its efficacy while designing a large scale. When we say large scale, it's like 20,000 to 60,000 subjects experiment to show its efficacy, operation because people go to country in different world. This whole process is full of risk and uncertainties. I think how we can, AI and data science, we can work with the researchers, clinicians, researchers closely to help them filter through a lot of their concerns, problems, and listen to them carefully and apply our techniques to help them identify the key problems to focus on, and then apply our techniques to help them build an understandable tools for them to make sure that they can be used in the real world settings to find the right solutions to the problems that concern them the most. So that's... Okay, yeah. So Yaxuan, thanks a lot for your insightful addressing this topic one. So let's addressing the topic two. The topic two is how AI and the data science can help on the healthcare policy decision making. So Tanya, can you help on this topic? Yeah, thank you. So going with my research, a lot of it focuses on using claims data, medical claims data for this implication, for how can we improve policy or understand what's going on at a population health level. So I often mention in a lot of my talks into people that data science is becoming more and more important within healthcare, and one of the reasons really is that healthcare data is expected to grow faster than any other sector, and that includes all types of data, right? When we're talking about healthcare data, we're working with EHR data, we're working with these medical claims data, we're working with image data, we're working with biometrics data from wearables, right? So this brings us a lot of information where we can start to understand a patient and understand what the implications and impact of policy can be on these health outcomes from this data, and also how equitable the implementation of some policy can be. And that includes looking at policy affecting different subpopulations such as vulnerable populations and understanding any heterogeneous effects even within those subpopulations, and having really large data allows us to do that because we're able to, we have enough data on all these subpopulations to really be able to assess these things, right? So in addition on this is there's all of this new health IT legislation legislation that's coming out that's really pushing for this interoperability, so that we can really look at data from different providers and bring it together, or from when a patient moves across to different insurance companies. And a lot of times we're limited and we can't get a full picture of a patient because we are limited by, okay, we have data from a certain set of providers. So if this person goes to a different provider, we can't see what's happening there, or we only have it from a certain insurance company. So when they go to a different insurance company or their insurance changes, we also lose that data. So these also initiatives that are happening in policy will allow us to get more longitudinal studies, get a fuller picture of what's going on with the patient, as well as for these population studies. And then as well as this interoperability will really help other groups, ideally it will help other groups really assess this type of data. So more people can really look at this and more people can do analysis and look at these impacts given right now. Normally it is only larger corporations or really well-resourced universities that have access to this data to be able to do these really, this really good policy work. On more of the technology side, the state also what we can for again looking at policy, this can help with when we're looking at AI or data science, it can really help with technology solutions that can improve healthcare for time constrained clinicians. I've heard of solutions where you can create computer programs that can really assist with care, such as explaining different contraception options to women or explaining maternal risk factors. And this helps low income populations as well as not even that, but just really supplement that 10 minutes you may get with the physician given how time constrained they are. And this can really help with implementation by allowing for screening or allowing for faster implementation of healthcare policy recommendations and making them more feasible. And it also helps with easier information dissemination of these new policies. So I think these are the forefront of things that are coming up and how we can continue to really use this, use data science and AI to really help with policy, not just decision making, which we get from the data, but also better implementation for policy. Okay. Yeah, thanks a lot. So let's move to from the high level of policy, let's move to the topic three. It's about how medical AI and the digital health will reach healthcare systems after the pandemic. So we would like to invite Mona to address this question. So Mona is going to share the screen. So I'm going to quit sharing. So Mona, you are muted. Can you unmute yourself? Yes. Okay. Let me share my screen for a second. And you should be able to see Dr. Lin now. Are you able to see Dr. Lin? I think it's maybe another screen, right? Yeah, yeah, no, that's actually, I think, what are you seeing now? I'm seeing everybody. Yeah, that's what you should be seeing. Okay, great, perfect. So let's just, and now you should see the Nvidia screen and we can go to the next one. Perfect. So thanks again, Gong, for this question. I think it's a very important question. How is AI and digital health shaping our healthcare, especially after this pandemic? So the pandemic has caused a fundamental shift in healthcare delivery from new ways of doing all things to creating new things altogether. As I see it, there are a few different ways in which medical AI and digital health is going to change healthcare. Bear with me as I go through them. And we're going to go through each panel of the slide to illustrate each one of these ways. So let's start with virtual visits. They are the new way of conducting the old-fashioned doctor's visit, the old true and tried way. Virtual visits have increased 10 to 15 times from pre-pandemic levels. And I think they're going to stay. Why? One, because patients, for the most part, like visiting with their doctors from the comfort of their own homes, it's easier. They don't have to accept their schedules. And because CMS, the Center for Medicare and Medicaid Services, is now even reimbursing for these visits, making them on par in terms of reimbursement with physical visits. So the incentives are there. But besides video consoles, we now also have an increase in telework and not just for office workers, but for the pathologists. They can now read studies from home. For sure, again, we have more incentives for televisits and for telework. But that is not the whole story. The reason this is going to stay is because it is now easier. And you can do this in a much more straightforward fashion. AI played a part here by enhancing the connectivity platforms that allow this telework and televisits by providing transcription and translation services, video compression, noise cancellation, even the wearables that allow this doctor to visit where the doctor has access to all of the vitals because of the wearables that the patient has. So let's now look at the second panel here and look at NLP, Natural Language Processing, and how that made it possible for all kinds of chatbots and AI virtual assistants, like Misty here, the blue NVIDIA avatar that we use for conversational AI. Today, there are chatbots for checking symptoms, chatbots for answering medical questions, for scheduling visits. We have bots for mental health and ones for billing questions. All of these are made possible by deep learning and new language models. And then digital health and AI has also made it possible to have remote patient monitoring. This is the last panel on the top, whether it's in the hospital room or at home. Now you have technology that can check on patients, then submit their vitals, remind them to take their meds, and send alerts when bad things are about to happen. Fourth, there are new ways to develop medical algorithms. In the rush to develop models from diagnostics to risk factor modeling to triage algorithms, it became apparent that we need more data and we need more diverse data to make our models more robust and more generalizable. Affiliated learning is a distributed way to train algorithms without sharing data. Now data scientists and clinicians in Wuhan and Indiana can collaborate on training an AI model without ever having to share the data. And the resulting model is actually a better model and will work at both places. Federated learning adoption is increasing and very recently the FDA even announced that it is evaluating the use of federated learning in its regulatory approval process. How about medical devices? Newer medical devices are now emerging to fill the need for smarter, smaller, and more mobile devices. Whether it is a mobile MRI scanner and autonomous ultrasound or a faster PET scanner, all of these devices now made possible and made better by software. By software that's right software and essentially by AI, they are becoming software defined. And at last, let's look at drug discovery. It used to take 10 to 12 years to develop a drug or vaccine and Dr. Chen here is probably working for Merck is very aware of this. Well, no more. Again, the COVID vaccine or vaccines, I should say, took less than a year to develop. AI created a time machine for drug discovery from accelerated sequencing and molecular dynamic applications to better simulation. Pharma essentially had a wake up call and this is just the beginning. Precision medicine is no longer a pipeline. It will be in reality in the not too distant future. So these are the ways that I see AI and digital health changing health care. Again, I just wish this did not take a pandemic to get us here. Yeah, Mona, thanks a lot for this overview. I think it's very helpful for us to better understand how AI can help. Okay, so let me share the screen again. So let me click. So, okay, let me see. Let me stop sharing. Okay. Okay. Let me see. Okay. All right. So let's continue to the topic for the topic for will be how AI enabled medical device and machine learning can change the health care after a pandemic. So, you know, can you help address this topic? Oh, yes. Hi, everyone. It's nice to see you here despite this COVID here. It's a pleasure also to be in this panel with such great panelists. And I have to thank you, Dr. Flores, because she basically painted a really good picture of a lot of what I think AI and devices are going to be in the next few years. So, and I want to iterate a few of this topic because I think they're quite dear to me. And I think probably we'll discuss more in this panel. First of all, I think one of the major challenges I think that we have is really how to share medical data, even anonymize medical data across the board. I think really this is the biggest problem here to create successful algorithms and devices as well that can really make a difference. And yeah, I'm sure we'll talk more about the topic. But yeah, when you think about medical enabled AI or medical devices, I think many of you might be thinking of wearable devices that can monitor you at home or at the comfort of your home. Or even during this pandemic, right? At the end of the day, before the vaccine or other help, I would think that maybe people were even a little bit scared of going to an health center. And so they definitely welcome remote monitoring or remote diagnosis. And even patients that were affected by the disease, I would think that there are many long term effects that can be studied. But it's really hard to start with the current system that we have. So I think really what we need is something that can monitor a patient at home over a long time. So, you know, we're all been dreaming for years of these fancy medical devices that would, you know, like a wearable bracelet or something that would track all your vitals. But unfortunately, they're not really here yet. And probably a lot of it has to do with the fact that it's difficult to interface, still difficult to interface medical devices with the human body for an over extended period of times. And so I think, you know, because I worked a lot on cameras and monitoring, I always bring this to some project that we, you know, even a research project that we were planning almost 20 years ago, which is monitoring people at home with standard camera networks, for example. I know some people are really concerned about privacy, but I think that can be dealt quite easily. But even if you think of a long term monitoring at home, this is a non invasive system. Well, let's say privacy without considering privacy, not invasive, but it's a system that makes it easy to look at the behavior of people even living alone. And imagine a camera can really notice some of the changes or movements or routines or changes that affect your behavior or your mobility. And a lot of people are concerned because they think of camera just like the one you're seeing now, right? I'm streaming over the internet to who knows how many people, right? And so if there's something behind me, okay, I'm in trouble. But a lot of these things can be, I think it can be mitigated because with the use of AI and especially at the edge servers at home, you can really conceal you can really conceal all all the video streams and just extract information purely. So I'm really in favor of a system like this for a variety of things, and especially also for general care in general. So I think that's one of my biggest bet. And I wish you all a great year and let's continue the discussion. Thank you. Thank you. Thank you. So let's continue moving on to the topic of five. So we can look into how our data and the complex system science change what we demand our modeling tools and the role in policy decisions and the house care infrastructures. So Mario. Thanks, Gwon. So first, as the previous panelists have definitely brought up the data science side of things has become much more rich lately. Exponential growth of data and many more different kinds of sensors and COVID has definitely made a lot of the need for these to be more obvious. And we'll definitely be seeing an increased growth in that aside from what we've already seen. The complex system side I'll introduce a little bit. And if you look at the world around you, you actually see that everything is really what we call a complex system on some level. And this deals roughly with systems that have many different pieces. And these pieces interact with each other according to some rules, which we may or may not be aware of, but we can maybe model in a probabilistic sense. And as these different components of the system interact, they give rise to the overall system behavior. And sometimes that system has statistical properties that are relatively consistent, sometimes not, but usually there's an unpredictable nature to the overall system behavior, even if we know with a pretty good resolution what the individual components are doing. And if you look, this kind of a system exists on very small levels like proteins, as well as very large levels like the global social network that we see either online or in reality as we interact with different people. And so the work I've been doing is in this area with respect to disease mitigation and so on is to construct models of social networks. These are real world social networks, not online social networks for the purpose of understanding how people interact because a lot of those interactions could lead to disease spread. And if we can understand that overall network, we could potentially integrate data about those people to understand policies that are able to mitigate the disease more effectively than some of the broad heuristics that we typically use. And this is where I think there's a lot of potential still going forward. Traditional models for disease spread tend to be focused on very high level understanding of things. So they may model the entire countries global or spread or even globally as a differential equation. And so that's a very, very rough approximation in terms of what actually happens. But it can still be useful, again, if you want to answer very high level questions. As we get more information of individuals and how they interact with each other, and as we have more computational power to be able to process it, we can create more rich models of those systems, which allow us then to create more granular policies to understand things at a more local level, which can help us be more effective and efficient in detecting potential outbreaks or to make policies that are more tuned to specific kinds of communities geographically spread or otherwise, though we can more effectively use our resources to stop the disease spread and so on. And I think we're just at an early place in these kinds of roles, in these kinds of technologies. As we get more data, and I said we can get much more granular in terms of how we see the world. And with resources, especially monetary resources becoming much more scarce, I think there's a huge opportunity to be using such tools, again, keeping even the same amount of resources, but being much more effective going forward. Yeah, exactly. I think right now during this pandemic, we collect all kinds of data, such as the social distancing and the contact tracing, all this kind of data was making the public available. So I think Google also generated a lot of data for the contact tracing of the stuff. So how we can utilize this data to make a better model, I think that will be a very interesting. Also bring up a big data challenge, how do we utilize AI to do that? Yeah, definitely just to jump in and pick up on that is you can mix, for example, the machine learning itself that all the other panelists have been talking about with complex systems to even automatically generate these policies or at least potential policies for the policy experts to consider in a way that may highlight non-trivial solutions. But when they do see them, they're like, oh, wow, this is a really good way to do this, but it would be very difficult to otherwise have thought about just because of the data explosion and the fact that our brains can only handle so much information, especially when it gets in a very large scale in a very dynamic environment. So it's a good point. Yeah, exactly. I think also you think about like a New York City, right? They have a lot of street cameras, right? To monitor in the traffic, right? So in the meantime, you also know, looking at the passengers, right? You can check whether how many people wearing masks, right? So all this information you can post-process and you can get a sense in the whole city, how many people wearing masks, right? So whether you need to mandate some policy to enforce that, right? So I think that will give you some real-time feedback. I think it will be a very interesting. Okay. So I think now it's the time. I think we can move on to the real-time question. The audience can post your questions if you would like to ask, right? So you can post it through the chat. So we'll address your questions. So, okay. So right now you hand them post, so Jacob, I think that's someone asked, the Jacob Stevens, right? Ask a question to Johanna about its current HW capability capable of providing useful AI-enabled medical device that are either internal to the body or affixed to the body. Do we still need further improvements to enable their level to monitoring? Johanna, can you address that? Yes. Thank you for the question, Jacob. Yes. So I think we already have some of these devices, I think probably Apple has been spearheading them. We have the Apple Watch that can do some monitoring for the masses. And also a lot of us are carrying cell phones all the time. And so even those can really take lots of measurements, I think, of our daily routine and behavior much more than probably we're willing to give away as easily. And I think in the future there'll be more of these devices. The question it has to do with how you interface, for example, even blood glucose levels would be really useful if they could be monitored all the time. But again, there might be issues there with chronicle monitoring, right? That it's hard to do. But if somebody has a pacemaker, definitely I'm sure that that device can also be monitored externally when relying through some cell phone or something else. Again, here we have a few issues that I guess probably the panel is going to talk about even more is that one of them is security, all these devices are secure to the point that definitely we don't have interference with anyone external, right? And the second thing is also is the data protected even when it's extracted from this device and eventually probably makes its way to a cloud. Is this data protected? And at the very least it could be the anonymized so that in some cases, even if you have the data, you don't necessarily are able to trace it back to a particular individual. But these are tricky things because like, for example, location data that some a lot of our cell phones capture can actually be traced back to a person even if it's anonymized. So we got to be careful about things like that, I would say. Yeah, thank you for the question. Yeah, it's a very good question. Yeah, thanks for your hand on addressing this. So one follow up question by Shirusi. She asked how AI and the data science affected the way clinical studies are connected post pandemic. So, Mario, who feel comfortable? I think this may be more up to the clinical people. So maybe Mona or Yeah, I can. I think that's probably better for answering this, but I'll just take a jab at it. I think one of the things that we're going to be able to with seeing today is using AI using an LP to for cohort creation for clinical trials. So you can imagine now we can come through the records, unstructured records, progress notes from the physicians and figure out who we might be able to recruit for a clinical trial. And the other aspect of it that with the new wearables and new miniaturized lab tests that you can do at home, there's machines today that with a little chip on the cart, you're able to do get your white blood cell count at home. So you can imagine if you have a drug for chemotherapy and you're following white blood cell counts. Now you're able to do that again, the same way we do it for Coumadin. You don't have to go to the lab to do this, especially if you're immunocompromised. You can get measurements of other biomarkers from your own house. I don't know, Dr. Chen, do you have anything to add to this? I like your answer, Mona. And I also want to point out that when this question is posed, my first question to the question is that clinical studies for what? Are they going to be clinical studies for vaccines or for drugs or for medical device? Because these three different kinds of types of medical, I would say, instruments have a very different, it's a very different level regulatory requirement. So the point that Eugene pointed out earlier about the sharing of data has been kind of a key, because AI and data science rely on data to bring out the insights that these studies can reveal, but the regulators around the data itself for either vaccine or drugs or medical device are very different. And so whether post-pandemic period will change the clinical studies conducted with the help of AI and data science, I will say partially yes to some, perhaps, medical device. But for vaccines, I have some reservation there that I feel it's going to be as rigid as it has been, because of a lot of issues we see in the society about vaccine hesitancy. Okay, yeah, that's very helpful. So we have got another question by Angie. I think she asked questions, why is interoperability such a big challenge? What are some steps to make data sharing more effective and easier? So yeah, Tonya, do you think you feel comfortable addressing this? Yeah, I can talk to this, and I think Mona probably has some things to add as well. So one of the big things, it's vendor lock-in. So a specific EHR company will do their best from the company perspective to make their data not operable with other EHRs, which makes sense from a business perspective. But from a researcher perspective, then it makes it difficult to compare that data across and different providers or across different types of vendors. So that's one interoperability aspect. And then also, again, I was kind of mentioned before when you're looking at this claims data versus this EHR data, how are we able to really bring this data together so that we're able to get a fuller picture? Because the claims data is what we're sending to the insurance company and what information we have in terms of what was done, but we're not getting those clinical notes, we're not getting the responses to the results to test or labs, and being able to bring that data together is not done right now. Some of it is due to restrictions, but some of it is just due to it doesn't work between the different types of codes and systems. So that's the type of interoperability issues that come up. There's also the issue of just a lot of data bringing this EHR data, bringing this digitization to healthcare is somewhat new, right? So how are you able to bring these old systems up to speed also in a way that is usable for data science as well as AI? And being able to, again, when you bring it up to speed in a way that is uniform so that all of it can be brought together to see the full picture. So those are the challenges in terms of steps to make it easier. That's kind of what policy is looking at now, right? How can we really approach interoperability in a way that is useful, right? And so that even a patient can understand their data from all different providers as well. Mona, I don't know if you want to add more to this. No, I think that that's exactly right. Maybe one part of the question was, why is it so important? Why is interoperability such a big, big challenge? And well, you know, as Storia said, there's just many different systems, lots of the systems don't talk to each other. You go to one hospital where you had a CT scan, you go to another hospital, they can't access it. You sometimes have to repeat the study. And, you know, there's also different ontologies. We call things differently. Someone might call it a blood test, someone might call it a lab test, you know, just simplifying it. And as you can imagine, if you really want to play with data science and you want to develop models, you need a lot of data. And that data needs, you know, you need to make sure that you're collecting similar data from the same entities. You know, if I'm looking to the creatinine level, I have to make sure that it is the same creatinine level that I'm looking for at every different hospital, right? So if there is no way to assure that, you know, the data is uniform, at least in the way we refer to this data, and in the way we access it, it's hard to train these models. Yes. So, yeah, very good. So let's look at another, I think it related to questions. So I think a question, I think the answer to the question is interoperability. That's the only issue. How about HIPAA? So issues that are mandated by law or by something else? So, Tanya? So, I mean, interoperability is one issue, right? Which I wouldn't say it's the only issue. The data sometimes not great, right? Like in all sorts of data, especially big data, there's cleaning, there's, you know, things that are mis-entered, there's physician bias, there's, you know, some physicians just prefer using some codes versus other codes, right? And so those are the type of problems that you run into when I'm working with this type of data. How do you really take into account these, you know, these differences for these inputs that are given by humans, right? In terms of being HIPAA compliant, this we kind of discussed offline as the panelists looking at privacy, right? And I think what we're learning in other privacy spaces is sometimes the current laws for de-identification probably are no longer strong enough. A study came out a few years ago from this CS professor from Imperial College London, and he really found that just being able to take de-identified credit card information and link it with spatial temporal data, he was able to identify 90% of like this 1.1 million people dataset. And so that's really, I think, the next real stage of challenges when we're looking at HIPAA or this de-identification in privacy is, as we start to get more datasets, one dataset, you may not be able to identify that person, but what happens when we start linking that to all these other datasets? And then, you know, this person is no longer anonymous, you're able to learn a lot about them, right? And how do we protect against that? And but also on the other side, we do want researchers to be able to use this data and we want this information for researchers to be able to move forward and do all these great things with it. So that's really a balance of, you know, supporting research as well as protecting privacy. I think that that's a really interesting question that's going to be coming up for a while. Yeah, there has to be a balance, right? Okay, so Mario, do you want additional comments? I think there's one that's next that I think it would be going for me to kind of mix that together. Okay, let me read the next question. So the next question is about, under the COVID-19 situation, so which kind of data are most in need and collected for policymaking? And the second question is, what is the biggest blind spot of spread modeling that policy makers feel uncomfortable? So Mario, you can start first. Sure, so the work I've done with COVID has been both in Canada and the United States and there's slightly different social dynamics that happen in each one. Primarily in the US, I'd say the contact tracing has been the one that the policy people and health people are most asking for as far as the data because that gives us an insight in terms of what's really happening and how the disease is spreading. And so while we have, you know, so many hundreds of thousands of pieces of data coming in a day on that, it's not enough to be able to really understand that spread, which means the kinds of policies that we can find that could potentially help mitigate the disease aside from wearing masks, which is pretty general, but more granular policies become very difficult to understand because we have this huge gap, this big blind spot in terms of where we should even be looking or potentially even what kinds of, why they're not even reporting and what kinds of policies would be more effective for those kinds of communities. And we need to have the spotlight on the society as much as possible, which does bring in some questions, you know, going to the future in terms of data gathering on privacy and so on, but at least self-reporting contact tracing in the case of COVID is one that I know both policy and health people are really yearning for and would be a huge help. As far as the biggest blind spot on the spread modeling and what makes people uncomfortable, I think, I don't know if uncomfortable is really the way I would look at it, but I think, especially comparing Canada and the US, I think the thing that policy people would like to see improved most is education and compliance. And so whatever we can do in the modeling side of things, the science side of things is great from a modeling and science perspective, but at the end of the day, even if we have the super most hyper accurate models possible, and we come up with these really good policies, if the people that we're telling to follow these policies don't believe what we're saying or just don't comply with them, then it doesn't really matter. And so the thing that I would say makes the policy side people more uncomfortable with the whole situation is that they're coming up with these great policies and they don't know how to tell people or to convince people that they really are the good policies and in a way that the people would actually follow them. Yeah, exactly. We may need to have some incentive to encourage people to follow policy. Yeah, it's whether it's incentives or whether it's just a better way of educating or whether it's reaching out to different media bubbles, whether it's online or whether it's in person or whether it's in print or whatever the case is, how to best communicate with different people in the community in a way that they really understand and start to trust what's being told to them is a big gap right now. And I think if just comparing Canada and the US, which are not so different, there are big enough differences in the kind of natural inclination of the general public to abide by policies, even between neighboring countries. Yeah, exactly. Yeah, so Yosha, do you want to add on some comments? Sure, I can. Let's try to add some comments on the question number two from the perspective of UK implementation. So in terms of the blind spot that I fully agree from the UK perspective, there are various many good modelers trying to bring out the warning ahead of the say the now famous UK variant strain circulating in the society back in early November's time. But the government just put aside the scientist's advice and then keep pushing forward in hope to repost the economy as much as possible. So for me, the biggest blind spot is sometimes not how at events our work is or how accurate our data is, is that as a scientist can we build up long term relationship with those people who are making policy decisions either in the government or in the company that actually could accept the work we do, not take them as a not not consider our work as a black box and therefore have some doubts with the outcome that we generate. Educate education as well as long term correlation collaboration with those decision makers is really important. Not our work is suspicious. It's just they need to understand in their own way. So that to me is the biggest blind spot concur and concurring what Mario say. And in terms of the COVID situation, data needs the most for collection in for the policymaking. I think you need to you need to the needs change with time. In the beginning it is the scale of the pandemic in the middle of the pandemic. It is the potential in the midst of pandemic when we see for example back in December and November's time when COVID was going COVID cases are going down in Europe. At the time, the understanding of how the interconnection between countries, how importation of cases from other area could possibly come back that that kind of data needs is more urgent. Right now the data needs become the implementation of vaccination, the speed of it, the effectiveness of it and how it will help on top of the social distancing policy we see already in place. So even complex situation there are as I mentioned earlier about vaccines. It is always important for us as scientists with modeling techniques to work with decision maker to understand the whole system with the ultimate goal of eliminating the disease. With those in mind, we need to identify the right problems at the right time so that our work can be truly trusted as a solution providers. Yeah. Okay. Yeah, thanks. Yeah, man. So let me ask you since you're an expert for the vaccine, right? So I think everybody here is also, audience is also very interested about the vaccine, especially for COVID vaccine, right? So my question is after the first shot of Pfizer vaccine, how long it takes to take effect? How many days normally do you know? If the first shot then we need, I think it's two weeks, we need to take the second shot. The problem is there is no data to really just to really say anything about what they say in the label. Normally in the normal time, a company will do a lot of rigorous experiments in their clinical trial, which compose of like at least 20 to 60,000 people. They need to design trial in order to understand the question you just raised, Guan, like what is the right, what is the right efficacy? And then when we, efficacy and duration and also intervals between doses. Right now, because of the emergency need, the clinical trial actually is designed in a specific way that they can only say there is 95% efficacy, not in terms of infection but it's actually in terms of decrease of severe cases in a 12-week interval that specifically they say 12-week intervals, they will reach 95%. Anything that the UK government as well as Canada government now trying to push out saying that because it's emergency cases, we should give the second dose in a delay time to a wider population so that everybody has first dose and delay the second dose outside the 12-weeks recommendation. In the industry, they call it off-label use and it's actually a murky zone to go into because we don't know, because there's no clinical study conducted to understand off-label use efficacy. But public health concern and clinical studies approval, there are two different things, so no one can answer the questions right now and people can just do some calculations smartly to come up with their own interpretation of the data but those interpretation will never be accepted at the regulatory agency including FDA or European medical agencies. So vaccine development till the approval process itself is interconnected and it's very complex. Yeah exactly, yeah also there's no evidence in the data for the single dose vaccine right, so how effective it is right? People say 50% or some people say 40% right? It's hard to get this number right? I guess they don't have this data. Yeah just want to jump in quickly and add something to Yashun's really good comment that Eugenio and Mona could probably or will definitely have no more insight than I will and that is as you gather all this other data about people with different sensors and so on during the clinical trial, we have the ability to understand the pre-existing conditions about people that we otherwise don't typically monitor in those trials to the extent that we could if they're just wearing those sensors and so on and so there's I think potential to better understand the interplay between those factors and say vaccine efficacy or whatever it is that we're trying to study but I'll leave it to you, Eugenio and Mona to give their more insightful comments on that specifically to see the potential for improving our understanding of those interactions exists. Maybe I could start. So I think you're absolutely correct. There is definitely a role for wearables and you know Eugenio here it says I am actually wearing an Apple watch and it can it has an FBA cleared algorithm to tell me if I have a fever or not so but you can imagine using such wearables and such new technology to be able to collect adverse data you know after vaccination to collect real world evidence as you know the drug development and Dr. Chen correct me if I'm wrong but that doesn't stop for the time the moment you actually distribute the drug and people use it you know you have to continue studying it afterwards to you know to look at side effects to look at efficacy so no you're right there is a lot a big role for that I just want to go back to one thing that Dr. Chen said you know I love what you said about having to educate the policy makers so that they understand where the science is coming from and not the adversaries but I think sometimes and you know maybe I'm treading here on murky grounds but even if they you know I think there's something beyond just understanding I think sometimes the incentives don't align between you know what what the government or what businesses want and what science says so even if they understand that they might not listen to it I just want to say that yeah yeah they have to balance about the economy and also the the health right I think that's the tradeoff between that too yeah and also I think for the vaccine also people talk about using reduce the dose right they say half dose right also not just the one shot they also talk about the half dose right so so that they can have more people get the vaccine vaccinated right so so Yosha do you have any comments on that? Two comments I think why I want to respond a little bit to the questions about the follow-up studies so once a vaccine is approved yes I think at the time once it's approved the medical device is so helpful AI techniques so helpful to do this what we call post licensure studies that's usually we don't call them clinical studies in the traditional sense we call them real-world effectiveness studies or real-world safety monetary studies so that at that point I think AI and data scientists has been tremendously helpful that I fully agree and one your questions regarding this half dose one dose delay second dose these are public health policy concerns but this in the normal days will never be feasible it's only because it's a pandemic situation where our weighing our weighing different kind of interest and also our weighing different kind of objective policymaker decided to take actions as a result of their own evaluations but this is what we always call off-label use we do not recommend that it's very dangerous because we are taking human being as a as as becoming a testing without rigorous following up the consequence of this so-called off-label use and in the midst of the so-called vaccine hesitancy this will only this sometimes will generate unnecessary pushback something that on vaccines such instrument can really save life in a tremendous way so if you look at what happened in France right now this is an example of why one would discourage so-called off-label use because if there's an adverse outcome from vaccinations it could be through different kind of platform it could generate a lot of bias against a life-saving device so we do not encourage off-label use and it's not because the interest of a you know pharmaceutical industry it is it is just vaccine itself is so complicated than what we normally understand about drugs and the code device it's in a very complicated policy and social context environment if I can just add to that I think that's very well put you know a lot of these these vaccines came to market really fast and and they they have certain efficacy numbers and they've been tested on certain things but and and if you know for off-label use we use that we take medications and use them for something else but these are for medications that have been there for a long time that we've seen you know what the what the side effects profile over years and years this is something new and and you are just taking it and and changing the parameters and changing the way it should be administered and the period between administration you don't have enough information to know that that is safe you know for when when you have limited data to bring something to market I would think that you want to follow exactly the rules that allowed it to be approved what wouldn't you say that I totally agree with Mona but the exact but the the way policymakers policymakers for public health sometimes different from policymakers for approval and if you see the way society work now in the pandemic situation we will get to see it's not lake of the technologies it's actually the lake of coordination of different disciplinary as our panel is here we actually can talk each other reasonably to understand where we come from what is important but somehow in the society level when this important decision making process is made the coordination just miss is missing in my opinion but it's just my opinion yeah yes agree yeah okay so let's look at the audience questions so there's another one so the portion ask how do we address the issue of bias in data this bias in AI and what the level of transparency should be available to the end user or consumer in the either of AI maybe you handle you are the AI guy right yes everybody here but yeah I can yes I can give it a stab so I think yeah we definitely have a lot of problems of bias because a lot of what is AI algorithm at the end of the day they rely on data and then the data itself although some people don't agree on this but the data itself has some some bias it could be just for example that you have minority groups that they're less represented than than other groups right and then eventually you create an algorithm that definitely you don't want this algorithm to be evil or do anything wrong but you know it happens that because of the data or the ways things are structured that you may give the disadvantage to some of these minorities just because maybe the data is different right maybe the data for a group of people is different from the data from another group of people and then you end up recommending to minorities something different from to another group of people and it's just because of the rest of the data that you have on them or the or just affected there they have you know the different kind of the lifestyles or other or any or other things so I would say I have a feeling that there will see a lot of these issues coming up and in part I think it has to I guess you know until we have some pretty good automatic tools that can spot these things I think probably individual data scientists should be assigned to to look at look at bias and I think it's important especially in a lot of companies to to have a good team that is looking at at these issues try to mitigate them as soon as possible and I have a feeling that some of these issues are going to keep coming up because just just because like I said you know the algorithms are imperfect and even though some people say it's just an algorithm it's just math but combined with the data it doesn't work the way you want it for everybody so I think I don't know if I really have a solution for this really it's just I think that unfortunately at the moment I my bed is just a question of trying to assess every case and then whenever you create some policies or whenever this algorithm creates some policies I think I will it's very important to plot them across all different groups minority or majority groups and see you know is there really a difference and sometimes you won't be able to tell and and you'll be too late yes I want to jump in on that oh yeah okay I'll just jump in make a quick comment so I think you could jump in because you're you're more than I would but I just want to agree a lot with what you Daniel was saying that the machine learning tools are meant to learn patterns and data and so they're going to learn how the data how the world is not necessarily how it should be or how we want it to be and that's where that bias comes in and how to gather more data and stuff that's more up to Toya's alley so I'll leave it to her but I just wanted to reiterate that um yeah there's that question that again the what machine learning is doing fundamentally some people are aware and some people are not so I just want to make a second it because I think it's really important point that was brought up so oh mo and I think you wanted to go to sure yeah okay so um the bias normally is coming from two different one of two areas right so um as was mentioned it could be that your sample size or whatever sample you have is not balanced and it's not representative and so you end up with these really biased results right or there's an aspect where there just is bias within um whatever decisions were made right like people often use the aspect of admissions right so if you're using like oh we normally take this specific candidate and use them to um and who's normally admitted right you are incorporating the bias of you know whatever the admissions office is right into that outcome right so so um there are um lots of uh statisticians and people within ML uh researchers are really looking at different methods to uh really figure out one how to make sure where the bias is coming from and two how to check for it right and some of the ways are checking you can just see you know like oh is race really this driver of this result right or is it really these other types of characteristics and then you also have to take into account oh well we have this characteristic but this characteristic is really representative of is you know confounded with something else right so um and how do you start to tease those apart right and so this is where it gets really complicated right and this is where we need a lot of people looking into that work of you know what um because people will suggest other like one of the suggestions has been you know why don't we remove race entirely from you know from one of our covariates or for analysis right but then we'll incorporate something else right like zip code or some other indicator that really is like you know can indicate some is bringing in still that race component right so so it becomes really complicated in ways to figure this out when we talk about um transparency I think for I think it depends on kind of what we're using this algorithm for right I think if we're doing something like ads you know on Twitter or Facebook like maybe we don't care so much right but if we're again um as was mentioned if we're doing things for medical decisions or we're doing things for policy decisions or we're doing things for justice decisions you know the big known um the really popular example right of when they try to use you know ML to predict if they should let someone go home or you know keep them give them bail or not right those type of things then we really should be incredibly careful and I think those things should be transparent and they should be in at least at the level of transparency that we do in academia in which you anyone can replicate to what you've done and um and then test it right so um I mean that's my two cents regarding you know kind of what's happening and bias how can we really um address it and um and of course if that is yes we just need to be you know more careful and continue to develop tools but this is definitely being addressed and there are definitely already methods that are out where you can compare um results with or without race or with or without income or whatever you're trying to um look at within um within this bias to assess bias um within your data yes mm-hmm yep very good time so mona do you want to add on some um no I think these are all great points you know as I said there are two sources of bias there there's the the bias that's already based on society and that machine learning and AI reflects when you develop these models and there is a bias of the lack of diversity in the data you know you're studying a disease on one certain population and trying to um take the learnings and apply them to a completely separate population uh so for the second one it's almost easier to take that bias away it just increase your data samples get more diverse data federated learning you know I'm just gonna put a blog for it again here that is one way to allow you to get data from all over uh the globe and and and take some of this population differences away but for both kind of biases you know what Toya said replicate and test it's like you can just put an algorithm out there and say you know this performed great this is my AUC and we're gonna start using it you don't do that when you develop a drug you go and and and do clinical trials and and prove that it works and so you have to do the same thing with AI algorithms just because I developed a neural network that gives me a certain prediction doesn't mean I have to jump and use it this has to be validated in real life um you have to validate it and then you have to continue learning and add add more data to it as as you learn new things and you also have to implement specific tests to look for bias to test for that so that you're seeing it to flag it when it happens so that you can go back to the drawing table and reach the algorithms yeah exactly so so you're showing from the clinical trial perspective and the for the vaccine study how do you address this bias data issue Ramona says when we don't have data we just get it I was actually thinking that's actually not not working in vaccine because we usually don't get the minority data from in the vaccine trial because they have very high vaccine hesitancy and so in terms of recruiting the patient now we know we kind of learned that COVID-19 for example is affecting the minority especially racial minority the most and yet in the pharmaceutical company and government put so much effort to ensure they are included in a clinical trial design but they are just reluctant to be put they they are reluctant to be on board with any clinical trial design which just couldn't get their data and that actually is to some degree hamper the conclusion of certain clinical trial because we couldn't get their data so this kind of bias I think aware of it may not have a solution of it but aware of it recognize it put it as a limit may be a first safe step to go and it also increased the transparency of knowledge that we acknowledge that's a limit and we did our best to get their data but we somehow we have a long still a long way to go to achieve that goal yeah yeah yeah I just want to add as well if we do have these kind of more elaborate models of the population and so on when potential policies or machine learning imply policies or suggested policies as well come up we again if we have these good simulation models of what the population looks like there's a potential to use them to see if there are potential things that we didn't think of that are impacting people in a way that we wouldn't have otherwise thought either from the data or just we were just we're not aware of it because of our own kind of blinders on because environments that we each kind of are accustomed to and so kind of wraps around the whole system where we have the data gathering the machine learning for policies and so on integrated with individual makers decision makers and then we can potentially if we have the sophisticated enough tools on the simulation side to represent a population we have a potential test bed to evaluate hopefully at least to catch potential policies that would have these effects that we didn't think of as well so I do think there's a lot of opportunity in that domain and of course you can just keep wrapping this around again well let's learn from the simulation model and how to make the policies better and so on we can go forever but I think it's an interesting circle that has a lot of potential in it so what's pocket vigilant whether in AI or in development that's important right yeah yes okay so okay so let's move on the next question so brand ask how do you know and determine what data to collect or what data could be useful it is good question who'd like to address this depends on what problem you're trying to solve here right you could take the google approach i guess for the facebook approach and just gather everything and if you need a use for it later which is maybe in a default scenario if we have the capacity to do so may make a lot of sense even if we then have the problem of how to figure out what data is useful on the computational side but we don't have to worry about gathering it from scratch we just have to figure out about realizing that we have it already I guess yeah but data gathering there's a cost of socialized that right so it might be we can first collect a small number of data first then we'll do some feature engineering to identify which feature was critical right we'll really provide you a lot of information so then we can prove that it's useful then we use them then we collect more data right I think that if you I mean all this will also relate to the bias right if you didn't find the enough data or didn't find the enough important features then you're going to create the bias right so all this is going to relate to the previous question as well yeah so I think this is an interesting question because I come really from the other side of really doing these retrospective studies right so um yes ideally that's what I would do mirrors approach if you have the option collect everything that you can because you never know what you'll need but um at least in my space and I think this is true for a lot of people a lot of the large data sets right we're not we don't really get to choose what we collect right we are given some data set that exists be it in a EHR claims data or your biometrics data someone else has kind of decided that and then you go in and say what is the how can I get the information that I want from what is actually available um in this data set how can I identify my study population or whatever else and um so I don't really think it's flipped right it's more like how can we figure out what we want to give in the data that we have um yeah it's a challenge yeah I would say um yeah I say okay let's say um okay so I think uh uh yeah so I think obviously you guys can post more questions uh in the meantime I think I have prepared some questions uh maybe uh so we can talk about it um I think the first question it is what this is the my obstacle in virtual clinical visits and how AI and the data science can help so maybe uh Tanya or Mona you guys uh who want to address first let me start with this one sure give me some time for you to think about this okay so you know there's obviously they're very low hanging to very easy stuff that that's its obstacles you know connectivity they have an internet you have a cellular connection how old are you and how familiar are you with using technology you know are you able to join a zoom call or the Webex or the Microsoft teams or all of the above you know all of that plays a big part in making that successful and then you know depending on what kind of visit you know is a physician able to get the info or the nurse able to get the information that they need from from the patient is there a way to send them wearables in advance to be able to you know to to measure the oxygen uh oxygenation or or to you know even today we have things where uh you know physicians actually they just got uh FDA emergency use authorization to be able to do ultrasound over a televisit so you send the probe to the patient and you tell the patient oh press over here and move it over there so that you can see the images so so that you know technology is making that stuff easier uh the ability to you you know I don't know a year ago you probably couldn't have a zoom visit where with a hundred people with all the videos are all on but but today you are because there's certain ways of making it you know compressing the the video stream uh so there are ways to to make this the platform easier to use for for the patient and and physician I think that's the first obstacles that I see uh the other obstacle is uh you know depending how many how many patients you get to see and and you you're having you know in a clinic visit you you have certain time where you see patients and then you have times to collect your thoughts and and you know write what happened and write your progress note and uh I could imagine a physician seeing patients one after the other and not having time to do that the technology can help here you can have it can then scribe and summarize the visit uh it can also you know provide a summary to the patient you know the patients might forget to ask all these questions and now they have a summary of everything that's unscribed uh that that happened and uh transpired I should say and they can go back and look at it and and they can show it to their daughter and say oh well this is what the doctor said but I didn't understand you know can you tell me uh so I think technology is is helping in in all of that and helping remove these obstacles yes okay yeah there's a cool thing that jumped in my mind too when you were talking Mona and that's the ability to be automatic translation so you can talk with a doctor with different language as well who may be a really expert in something and just increase overall accessibility and I think that's a really cool area as well that's exactly that translations are happening real time captioning is happening real time we actually have we just we're releasing a feature where you know how when we're looking now at each other in zoom and depending where you're looking your eyes look like you're looking somewhere else you know there's now a AI that can redirect uh you know so that every time you whatever you're looking it looks like you're looking at a camera I'm looking at all of you now but but that's AI doing it uh now I don't have that feature on so it's not working but I'm just saying that there's a lot that that can make this experience more like the real experience yeah it'd be cool if those automatic captioning tools as well could distill the technical jargon to the more kind of common person jargon that they actually understand some of those things exactly level of those tools as well yeah for sure for sure yes Tanya do you want to add down some comments um so one of the biggest um limitations for um for virtual clinic uh is actual reimbursement right one of the reasons um that we've been able to see the spike during COVID is that you know CMS and insurance companies decided to start covering it right so so um that's one thing to also kind of like start with right where if people aren't going to get paid no one's gonna be able to do it right so um that's a major obstacle for sure um you know what's covered and how can it be covered um a lot of the things I was going to mention in terms of um obstacles that come up such as you know um how do you physically what do you do if you can't physically see a patient if you need something physical like in order to touch the patient um and um but on the flip side there's a lot of there's a lot of opportunity for uh for these virtual clinic visits to help a lot with health disparities right like it will help with people who have um trouble with transportation right it helps with um low a lot of low um income or people in rural areas it can really help them be able to get regular access to uh virtual care right through through virtual care but yes we run into issues of course with band with issues and connectivity especially in rural areas we run into the issues as we talked about like you know technology uh help the level of comfort of technology with um older patients um even just enough um even just enough like minutes on your phone and or you only have one cell phone and your husband takes it out like there are lots of things that as I've been speaking with clinicians they run into with this virtual health space um and it's been mostly used in mental health and so that's where I've seen um I have the best understanding of cognitive limitations and um and even just on the mental health side they've said oh it's really helpful for you know our anxiety patients or you know for let's do self-development patients but for trauma patients or really young patients or patients we do play therapy with like all of this becomes very difficult right um so um substance abuse patients it often becomes really difficult so it's um so I think the obstacles come up in terms of what needs to be treated and how important is that in-person component um becomes uh as well as reimbursement right I think those are the really too big um obstacles for sure it's a privacy also issue right so let me talk about the second one right so what are the challenges for AI in health care and how to address the privacy issue during the data acquisition process for AI in health care so yeah so Mona do you want to talk a little bit of this uh what are the challenges for AI in health care to address privacy issues well I think we've just been touching yes upon this all along it's you know we have we have HIPAA regulations that make sharing data uh less accessible uh I I'm trying to think you know one thing that is happening these like I know we keep talking about HIPAA and we keep talking about gtpr and about other things that that make it difficult to share data but there was I think I let this somewhere the other day someone just you know put a they had a store front this is pre-covid and they told people if you come in and give us your you know a couple of items like your date of birth and and how long you've been married or whatever and just if you like information about you will pay you I don't know how you know some amount of money and the amount of people that were willing to go in and just give their information just for for little money uh you can imagine that there's actually a lot of populations that have bigger incentives than being paid to share their data you know there are people that belong to a group or social support group of of very rare diseases and these people are dying to have medications that are you know that that can help them so they are willing to put their data out there for research and technology now is enabling people to actually obtain to create you know to be able to share their data for research so maybe Dr Chen that's where you get some of your data at some point uh but you know so privacy is a barrier but I I think depending off on on the situation and the specific problem um that there's ways to to get around it I don't know if I really got to the maybe Toya you can answer this more to the spirit of the question well a lot of my points with privacy I already kind of mentioned earlier but um on to kind of add to uh Mona's point on people being paid I think um like I've definitely read some stuff where that um it um privacy is going to end up becoming a luxury service that lower income people just may not be able to afford and I think that is that um I think that's a major challenge right like how do we protect you know um how do we not use how does privacy not become this new thing we're like you know poor people are just selling their data or their privacy in order for whatever either free medical care or whatever else and it's just another layer of exploitation um and I think that's something we need to be careful of and that's a potential challenge that's a great point actually I haven't thought about that's an amazing point Toya yeah just to jump in as well and maybe Eugenio can talk about the feasibility of this but um is it a loophole so Toya and Mona they know this um to the HIPAA regulations to create uh an avatar for everybody essentially that learns their personal data that in a way that is not specifically saying their birth date or something but can represent them medically that can then be put in the kind of a cyber world um and be used in that sense kind of like a proxy for you um again in a way that hopefully doesn't have specific de-anonymizing information but is medically representative but then digital twin of source yeah exactly again I'm just just a random idea not by any means I guess you run into the same problem right they were saying before right like at some point right like if someone was like oh I know a professor at Purdue in industrial engineering who like I don't know some random other characteristic about me it's going to be very easy to identify who I am right so you run into those issues like even when you work with uh claims data you often need to aggregate it up right because if it's one or two people someone can probably figure out who you are so so that's where it becomes difficult where um you can pull out this information but eventually you have enough information where you pretty much know who the person is um I see what you're saying I was again I'm not by any means I actually oh yeah no no yeah I was just no they were good points I was just kind of curious maybe some you don't need all of that data like you were saying like maybe you don't need to know somebody's a professor at Purdue or any other location specifically to be medically representative and so I was kind of thinking some subset of data does it exist that's medically representative that it is not de-anonymizing as a kind of fun you know yeah maybe you genuinely can uh remark more on this but but there's you know this new new models that are being used to generate synthetic data right to generate synthetic medical records and synthetic claims data there's data sets out there that are completely synthetic that were trained on real data initially maybe that's where we go with you know with GANs and other uh ways of creating data that that is not it cannot be to respect a certain person I don't know Eugenio if you have anything else to add on that no yeah I um I think yeah you're right I think it could something could be done like that I'm I just I'm not I'm not sure how we would deviate from from real data so I would have some concern maybe but yeah it's a you just have a great good idea I haven't I hadn't thought about it before for sure yeah well I know that there's some companies working on that some some institutions that already is starting to do that so I don't know yeah just like you can generate like fake faces that are really super real and um but yeah I mean I would be yeah honestly probably would be worried I I don't know yeah I guess you would have to see if this data actually you know you can prove that it has the same quality as the original one but I think you can yeah you can still collect data so you know I think we're all afraid because sometimes we see that you can actually identify people from the data but I think if you take some some precaution for for some things it might it might be hard right I mean it's one thing is to have a location data that really puts you somewhere and then you know you can really find your house really easily or your where you work but I think medical some medical data maybe people are just too too worried for for you know not for the right reason I guess it's I think some of these data can be anonymized especially if you like separate it maybe because not you don't need maybe like a complete digital you know digital twin of your for all your medical records sometimes sometimes it's good to have but for maybe there's only parts of it that you can collect and regardless I would think if you know if I if I had like a medical record of you know the land I don't know five or six major disease I had over the last 20 years let's say your major operation I would think it would be hard to trace it back to me unless there's some yeah there's some location data or there's something else but yeah I know I think sometimes we sometimes we blow it out of proportion like at some point if someone if you know I have a CT scan of my brain somewhere and and and my name is removed and my date of birth is removed and and someone is able to go back and reconstruct that CT scan of my brain does it matter to me personally it doesn't like is there someone out there that's gonna go and say we're gonna go look at every patient and go and figure out if they had a tumor that they didn't tell us about because we're gonna go into this one model that uses data I think it's a little bit far-fetched in like in conspiracy theories just like this do they say back of the data but yeah oh sorry go no no no that's it oh no so yeah so I think there is I think in general as a society we're going to have to accept some level of risk right I think the benefits do outweigh the cons right of some you know loose-ish like privacy rules right that being that being said right like I in reality most people as you bring up in your example aren't aren't necessarily care about your privacy specifically what I mean by that is like nobody really cares I don't care if you have my social security number I care that you are you what you use it for right I care that you can use that for a negative purpose right and so that's really maybe where we should focus right except that there's going to be some level of sharing right but how do we protect people in in that way right like how do we make you know researchers have access but make these really you know restricted use or something like that so that we know that you know there isn't going to be this it's not going to be sold to like your insurance company and used you know to calculate your premium or something which is what people always use is the most fearful thing around around you know like EHRs and stuff being shared I think that's pretty much how we're going to have to address it in the future. Yeah I think there's also a point in Yosha mentioned this earlier in the pandemic context about educating people in terms of what what data is gathered so some transparency as well and how it could potentially be used and and that more towards that intention of what could be used in the future what's currently being used now and to help them understand really what the world is like versus the conspiracy theories or the far far fetched things that even if they are created is probably so far into the future that it wouldn't matter because they're probably not going to be around anymore because it's a hundred years or something down the line and and so I think there's an opportunity you know being back since we're an educational institution to infuse that into how we at least from the student perspective how we educate the ones that are here to exist in that kind of a world and be aware of those things. Exactly yeah I think yeah I think yeah okay so the last question okay for due to the time limit so the last question will be how to take it under HPC to get AI at scale for health care so you Heno can you help on this? Yeah definitely and it's possible we can take even another question because I think you know in terms of what we can yeah what we can do in terms of what we need to analyze data I think it's the same that we need or for a lot of all I don't really see it much of a difference it's probably just the same kind of clusters or GPU or you know DLA the learning accelerator clusters that that we have available I don't really see any difference yeah this application or any other one Yeah I think more important is the next question is how to build the trustworths right how do you build the trust I think that might be a more important question in particular for health care right so do you like like to trust a robot right to make a decision for to to say right whether you like how healthy you are right how do you trust that right so I think that might be a a challenge question Yeah even you know go ahead if Mauna No I was just gonna say that goes back to the same issue of transparency and you know showing people what this model is doing and how it was trained and why and and really assessing it all the time and validating it and doing trials on it you know validation trials and getting someone to say yeah yes the specific piece works you know having a model that this generalized concept of an AI or a robot that's gonna treat us from A to Z I think that's a little bit far fetched you know with the way I think of AI now is for specific you know very narrow AI tasks and for those hopefully we should you know we are able to tell the world how they're trained and and show them that they work so that they can trust them if I understood that question correctly yes yeah I think part of the trust as well is to not just show that it's say compared to a person how much more accurate or whatever is it but also to not be afraid to show that the machine learning can go wrong and this is the cases where it has gone wrong and the way people can understand it's not a silver bullet and it does make mistakes and maybe at a lower rate than people but it can make mistakes and they can maybe work together in certain ways to overcome individual's limitations yeah but I think there's an over emphasis when people think of machine learning or AI that it's a silver bullet and it's always right and I think I'm highlighting that it's not just as people are not may help in that as well because I think people are afraid of what the tool may tell them under the assumption it's always right as well and I think to highlight that in in better ways may be useful because that's actually you you bring up an amazing point Maria you know the other thing that we always we're holding AI to a level that we don't hold ourselves to it's like the AI has to be 100 percent correct over the time I don't know one driver that's 100 correct percent correct in all of the decisions I think you have to take these AI models and what they're doing on the aggregate is on the aggregate are they doing better decisions than you know then if we didn't have them are they giving us better results and if we didn't have them as opposed to saying every time it's going to be right and I I think that's the standard that we don't hold humans do they exactly yeah oh yeah no this goes yeah to just piggyback on this that this goes back to the initial point we're talking about bias and samples right like I'm sure everyone heard of the study where in general right ML does better at predicting or diagnosing what's wrong with the patient for a very for common conditions but as soon as you get rare physicians are better right so so those are so also again like as you're saying like taking into account when it works well and when it doesn't and you know using those together right like I think will will be best when we use ML as a tool to support a decision by a human instead of it being this one or the other yes exactly all right okay so let's uh here's the final I would like to thank all the panelists and the audience for joining and participating these webinars thank you bye bye say thank you so much thank you this is great okay bye bye