 Alright, so why don't we get started here. I take great pleasure in choosing Professor Gadak. John asked me to do this bio shot, so I'll just say a couple of sentences. John is the Duke-L.C. Jackson Professor of Electrical Engineering and Computer Science at MIT, and also former head of the ECS department. Among other things, he's also my former PhD advisor. So you'll say some dirts down about it, right? Well, I think that's why you want me to keep it short, right? Yeah, he knows way too much about me. But one of the things that you probably have noticed is that John has worked in many different areas from hardware verification, software abstraction, software radios, and now telemedicine and medical decision making. So he's been talking about a couple of projects, but broad reaching interest over the years has ultimately made it to this. One way to describe it, the other way to describe it is adult attention deficit disorder. My attention span is too short to work, and I need one thing for very long. Alright, so thank you, Zeishan. And as I've told several people today, one of the biggest mistakes I ever made as a faculty member was letting Zeishan graduate. He was way too good to let him leave. But you guys went for that. Alright, so we'll get on with it. I was always taught that there's a high risk that people will leave your talk before it's finished, and therefore the things you really want to say, you should say at the very beginning if you're not getting what you said. So here's the real message of today's talk, which is not what I'm going to actually be talking about, and I address this mostly to graduate students and undergraduates who might be in the room, but even to faculty who are not too old to learn new tricks. I think it's time for computer scientists to really focus a lot of their attention on research that impacts health. It's intellectually tremendously interesting. It lets us do interesting computer science. And it's psychically tremendously rewarding to know that you're actually doing something that might someday help people. And in case there are any doctors in the room, they should learn some computer science because it will make them better physicians. And when you're talking to undergraduates, what I've done over the last several years is spend a lot of time convinced in undergraduates who want to become physicians that they should major in computer science, not in chemical engineering or some such thing. They will get into medical school and they will be better physicians for it. All right. Quickly, here's my research group at a glance, what we do. This is a dated picture. And I brought it out only because I wanted to show you what Zeeshan looked like before he adopted all the burdens of being a faculty member. You were the tallest member of the group? Yes. So the good news is that since Zeeshan left, I'm the tallest member of the group, which is not easy at 5' 9". So that was one disadvantage of having him around. We work at the intersection of computer science and medicine, mostly in algorithm design, signal processing, software systems, and more and more attention on machine learning, clustering, data mining, things of that nature. OK. Here's to the real body of the talk. I think it's productive to view management of health. And so I'm looking at health care very broadly construed from the perspective of someone who studied a little control theory. And any of you who've taken a control theory course are familiar with this kind of diagram. You've got a controller. You've got some actuators. You've got what a control theorist calls the plant, more humanely might be called the patient. There's a hidden control system which unfortunately we can never see, the homeostatic control system that all animals come equipped with. There's an environment which we usually can't control. Is it hot out? Is it cold out? What is the patient doing? There are some control variables. We don't want to see everything about the patient, or we can't. And so we might look at the course of the disease or measuring some side effects. There's a huge error element. And it's one of the things I'll talk about a little bit is that, to my mind, way too much of medical decision making these days is based upon evidence that's subjective rather than objective. Psychiatrists give somebody a bunch of pills and a month later say, do you think you're less depressed than you used to be? And then they change the dosage based on the answer. Who knows how reliable that is? I work in epilepsy a lot. And there have been some control studies which show that patients' reports on incidents and severity of seizures are incredibly inaccurate. So there's a lot of that. And then back to the controller. Well, that's a complicated picture. Here's a much simpler picture of the same thing. And this is what I think of the medical controller, get information, evaluate the patient. And that's different from getting the information, drawing conclusions from it. Choose some intervention, perform the intervention, go back to the top and do it again. Unfortunately, for a lot of interventions, for example, typically drugs that are taken on, say, a daily basis, the intervention is done over and over and over again. Every time you take a pill, you're making an intervention. And very little checking is done as to what the effect of that intervention is. As anyone who's ever looked at any control theory knows, the length of the loop should be governed by the rate of change of the plant, in this case, the patient. If the plant is changing quickly, you've got to interrogate quickly. Otherwise, you do it more slowly. Unfortunately, in medicine, it's much more likely to be controlled by the availability of the physician than by the actual need. So people get high blood pressure medication, and they come back six weeks later when, in fact, there is something measurable much faster than that if someone would take the time to measure it. So when we control complex systems, we would like them to stay with intolerance. It's much easier to keep a plane from going into a tail spin than to get a plane out of a tail spin. And you'd like to minimize the severity of the interventions, because almost every intervention has some unpleasant side effects. How do you do that? You measure often. You analyze the data quickly and thoroughly. You would like to be able to predict future states, which is very important, and then intervene before it becomes unstable. So for those of you who don't know about control theory, here's a very simple control system. You all use it most days. Your morning shower, you get in and you turn the faucet. You notice that the water is too cold. So you turn the faucet a little bit. Water gets warmer, but not warm enough. You turn the faucet a little bit more. Water is now a bit too warm. You turn it the other way. You enjoy your shower. Well, what would a shower look like if the doctors were in charge, or the medical system were in charge? You'd get in, and instead of the faucet, you'd just push a button that would turn the shower on. You'd notice that the water is too cold. So you telephone the plumber, because you can't control it yourself. And unfortunately, you're not allowed to get out of the shower. You have to wait until the plumber arrives. The plumber arrives, and instead of going to the shower, goes into the basement somewhere and does something to the hot water heater. The plumber leaves. Minutes later, the water becomes scalding hot. You telephone the plumber again. And he tells you to call 911 and go to the emergency department. This is kind of what the control system looks like in the world of medicine. And the shower is kind of like that. Well, obviously, we should be able to do better. Here's an example, a congestive heart failure. Probably the leading cause of disability these days in the United States, typically managed using some combination of a half dozen drugs. The appropriate combination dosages should vary a lot from person to person. And in fact, for every person, they should vary from day to day, depending upon exercise, diet, things like that. In most cases, the adjustments are clearly non-optimal. They're not done very often. They're done with very poor information, using things like weight as a proxy, resulting in millions of unnecessary days in the hospital. And who knows how many unnecessary deaths? So all of this is just by a way of saying there's a problem. There's a logical way to think about it. And we really can do better. So we can reduce the length of the loop. We can get better information, more representative. And by that, I mean too much of our information about people is gathered in the clinic. Well, fortunately, most of us do not live in clinical environments. And maybe our blood pressure in the doctor's office is not the same as our blood pressure during the day. Certainly, if you're exercising, things are different than when you're sitting in a chair or lying on an exam table. So you'd like to get information in vivo that's representative of what the patient's life is really like. You'd like to get it in a timely fashion. And mostly, you'd like to get a lot more of it. And then you want to make better use of the information. And people hear a lot about personalized medicine these days. But mostly, when you read about it, they're talking about genetics, talking about the genotype. Well, most of our health is not about our genetics. It's about things that happen to us. What viruses have we been exposed to or bacteria? What do we eat? How well do we sleep? Many, many things that are not part of our genes. And yet, when people think about personalized medicine, they don't usually think about that. And I think that's something we could fix. And time series data. As all of us in this room know, first and second derivatives can carry a lot of information in them. And yet, very often in medicine, people focus on the current state and don't pay enough time attention to the derivatives. Well, we'd like to improve the quantity and quality of the controllers. It's not obvious we can improve the quality of clinicians, at least not dramatically, because clinicians are people. And evolution is a very slow process. People do not get smarter very quickly, or better eyes, or anything else. So really, if we want to improve the controllers quickly, we have to rely less on people doing the control and more on machines and software, which we can evolve very quickly. Furthermore, if we think about scaling health care, today in the US, we have the same number of caregivers per 1,000 patients as we had in 1970. Yet, we have an aging population that's in far more need of care than we had in 1970. We're not going to fix that. Certainly, anyone who reads the paper and looks at the economics of health care, most of the money goes to paying salaries. Almost all the money in health care goes to salaries. We can't afford to add more caregivers, at least professional caregivers. So the only thing that's going to scale is if you have patients and family members providing more of their own health care. And so as a computer scientist, if we think about scalable systems, this is where we have to focus on the scale. All right, I want to give two examples of my current work. Now, before you laugh, I can actually count to two, even though I do computer science. I can even count to three. I know there are three things here. I'm not going to talk about Zeshan's work because I would get it wrong and he would laugh at me and that wouldn't be fun. But I've recommended it to all of you. I think it's incredibly exciting work in which he's doing work on how do you anticipate the need for an intervention, figure out who might need an intervention and what should it be. And particularly in the area, many areas, but when he was working with me, it was in cardiovascular risk gratification. But I'm not going to talk about that today. He's right over there. You saw his picture earlier. You should talk to him about it. I do want to talk about the work that my students have done in, I'm a professor, right? Professors, professors. The actual work is done by the students. So I'm going to talk about the work that my students have done in the area of medical telepresence, mostly the work of Espanja Hershey and Eugene Shea, and real-time detection of events, mostly the work of Ali Shoah. So medical telepresence as a way to shorten the loop. So here the idea is to change the point of care from the clinic to wherever the patient happens to be. And to the extent we need to involve clinicians, and I do think we need to involve them, is to bring data, not people, to the clinic. I don't know how many of you, I'm sure all of you had the experience, you go to the doctor, you wait in the office seemingly forever, and in fact, instead of looking at you at all, he or she ends up looking at data about you. So maybe we could just send the data. Bring appropriate and accurate information to the patients and empower the patients and the family members to participate in decision-making and the application of the therapies. There are many different kinds of telemedicine. You can talk about the levels of mobility, is it stationary? So for example, there's a large telestroke project from Mass General Hospital because somebody has a stroke out in the middle of nowhere, they go to a center where there's a dedicated line which connects them to a stroke specialist who decides how to treat the patient. It can be nomadic, you can imagine visiting nurses, carrying systems in a briefcase, and or it could be mobile, ambulatory monitoring as people wander around or in vehicles with some communications. There is latency, it can be non-real time, and there's a lot of non-real time telemedicine today. Most of, much of radiology is done that way. Radiologists, when they get a call in the middle of the night and no longer rushing to the hospital, they open their computer screen and look at it, at least in Massachusetts. And it's very productive. Pathology is typically, it's almost always in some sense telemedicine, but it's not real time. Or you can do real time. And there's a amount of information. There's a lot of telemedicine today goes on by text. Almost all of my interactions with my primary care physician or by email, very convenient for me. Phone is a little more, you can imagine shipping physiological data around images and video. And so these are the most demanding spaces, right? Mobile, real time with high bandwidth data. That's the area that we've mostly been working in. Our current project is designed to help very sick babies. This is not a Photoshop job. This is the real size of that baby in hands, and that's not Shaquille O'Neal's hands. Those are hands sort of the size of my hands. A baby that size is very sick by definition. Mostly they're premature. And as you can see from this chart, the number of premature births in the U.S. is rising. It's slowing down the pace of increase, but it's still up, 12.7% in 2005. This is because women are now having babies at later ages, which increases the chance of a difficult birth. It's because of the increased use of fertility drugs of one sort or another, many of which will say lead to multiple births, which are often premature. And these premature babies, particularly the ones down here at the bottom, less than 32 weeks are typically very sick when they're born. Globally, somewhere between three and a half and four million deaths per year. And typically the difference between expert care and proper facilities is a difference maker. Not only the difference between life and death, but between life and lifelong, between lifelong disabilities and not having them. We're increasingly good at helping these premature babies survive, but many of them will go through life with very serious disabilities, typically related to some sort of brain damage occurring early on. Newborn medicine is highly specialized. Obstetrics is not. So every community hospital is willing to deliver babies. But if a sick baby is born in one of these hospitals, they are typically not well-qualified to treat them. There's a hospital I've done some work with in our area, I won't name it, where they have a big obstetrics process and care for the nursery at night is provided by local on-call pediatricians who spend their days prescribing amoxicillin for ear aches and then are expected to come in at night and know how to deal with the premature infant and respiratory distress. Not a good situation. So increasingly we see a lot of inter-hospital transport. The current mode of these community hospitals is to take the baby and if it's sick, move it to a tertiary care center. They're few and far between. I know you have one in Ann Arbor, but I'll bet there are no more than three or four in the whole state of Michigan. Air transport is usually not feasible and so it's done by ground and so the transport can take a long time. Here again is a premature baby. It's actually with the hands of one of my graduate students that is his child. So there's pattern of usage we're interested in is the baby is born in the hospital, put in an incubator with communication capabilities, evaluated by specialists at the remote site, decide whether transport is necessary. If that transport is necessary and this is the way it's done today, a truck and I use the word advisedly, these are big vehicles. It's sent from the receiving hospital. The baby is transferred to the truck, transported back. The baby is at high risk during the transport and so we've been focusing then on how do you provide monitoring at the receiving hospital during transport. Legally, as soon as the baby is loaded into the vehicle, it becomes the responsibility of the receiving hospital, even though the physicians there have never seen the baby. So there are three parts of the project. Picture here of one of the more scenic hospitals that were in our program, an instrumented incubator, a physician friendly application, or at least I hope so, and most interestingly, technically the communication subsystem, which is really what I wanna talk about the next few minutes because it is the technically fun part. The goal is to provide sustained high throughput, low latency in a way that's economically viable to deploy, operate, and maintain. This is very important. There's been a long history of telemedicine projects which get built, they do a really slick demo, and if you go back three years later and say how's it doing, they say oh, it doesn't exist anymore. In fact, it never existed post demo because it was not viable to actually deploy, operate, and maintain. Something changed and the whole system stopped working end of story. So, if you think about this, what are your network options? There are people who've done this by creating their own networking infrastructure. It's just not sustainable for a hospital or any medical system to do that. So, systems that have done really nice demos with satellite networks and things like that don't ever get used. Well, in the hospital, you can probably use something like Wi-Fi. Today, almost certainly most of them. That doesn't work in the vehicles. Anyone who's tried to do Wi-Fi from a moving vehicle knows how kind of hopeless it is and people want afterwards, I'd be happy to talk about the technical reasons why that's a problem. So really the only option is to use cellular networks, the wide area wireless networks, at least during the actual transport. Well, the problems with this, there are multiple problems. When we started the bandwidth was very low, it's now higher, but it's still low and in particular, we're interested in the upstream bandwidth from the vehicle. Most of these networks are optimized for the downstream bandwidth. Therefore, the iPhone user to download YouTube videos, not to upload the YouTube videos. So that's an issue. There's lots of packet loss. I'll talk about that in a minute. And of course, anyone who's used a cell phone knows you do lose the connection from time to time. So what we've done is we've multiplexed the streams, the video streams in particular, over multiple channels, up to nine. We've used it a single time. This of course gives you greater average aggregate bandwidth because you get to aggregate the channels. Network striping for those of you who work in that networking area. You get a dramatic reduction in the variation in bandwidth. There's sometimes 18, sometimes not often, but sometimes AT&T is working well. Sometimes horizon is good, sometimes sprint is good. So having variety gives you, reduces the variation in the bandwidth and the round-trip times and the losses and get increased reliability when we run our system loss of ways. So the trick was to build it around middleware that made all of this work over a sustained period of time. So the middleware manages the multiple connections, takes continuous measurements of round-trip time and bandwidth, other things of the network and adapts accordingly. So decides which channels to put the data over based upon measuring how the channels are behaving. Hides most of the details of this from the application layers, but not all because some of the adaptation, as I'll point out, can only be done intelligently at the application level. So if you abstract all of this away, then you won't be in such a good shape either. So here's the architecture. There's the patient node and then there's a whole group of specialist nodes. This is a slightly outdated picture of what the user interface looks like. That's not a real baby. That's not a real doctor either, so, it's okay. So we've got video, we've got audio, we've got telemetry. They go into our middleware. It gets shipped over multiple channels, gets picked up by the backbone network and then we assume that the specialist nodes, for the most part, there's a broadband connection. Everything gets reassembled and gets put into the user interface. So we want to send the highest quality real-time video possible. The horde and slash tribe levels provide sufficient aggregate bandwidth on average in most areas. When we go through a long tunnel, bandwidth is not adequate. But the real issues are that the channel throughput is highly variable and the packet loss rate is highly variable. So to mitigate the effects of the bandwidth variation is where we use dynamic feedback with the application. Horde tells the application not anything about individual channels but the recently available aggregate transmission rate. The application then adapts to this as appropriate. So it could adjust the quantization parameter. It could adjust the frame rate. Could adjust the latency. Why do we care? Why would the application need to know? Well, if, for example, the physician at the other end is trying to determine whether or not the baby has a seizure, is having a seizure, they don't need high resolution video, but they need smooth video. They need to be able to understand the movements. On the other hand, if at the other end the physician is trying to examine a lesion to see what it looks like, they may not need video at all but they may need as a really high resolution still. And so there's no way for the networking layer to make those decisions as to what can be traded for what. So we have the physician indicate in an abstract way of what they care about. Frame rate, latency maybe is important. Resolution, they speak in that language and then the application takes the information from the networking layer and makes the appropriate adjustment. The other problem I mentioned is packet loss. We've measured it and these numbers are in the Boston area. When our notice station area, we get up to 2% packet loss. When the node is moving, let's say 60 miles an hour, we get up to around 10% packet loss. That's a lot of packet loss. Why do you get them? Well, when you're moving, because you have base station handoffs, you have signal occlusions, all the usual kinds of things. Because this is real time, we can't just use TCP or something and use retransmissions. So in fact, we don't retransmit anything. If a packet is lost, it's lost. Because we're often throughput limited, we can only do a limited amount of forward error correction. We do some, but less than one might and as throughput gets better, we'll do more. But effectively we have to somehow contain the effects of a lost packet. And we do that with the way we encode the video. Each frame is segmented into a grid of subframes. And then each subframe is encoded and decoded independently. The issue here, I don't know if any of you or I'm sure some of you, some of you probably haven't done video encoding, you've typically got iframes and pframes. And the iframes include a lot of information and the pframes are small and include deltas from the iframes. The problem with this from a networking point of view is every time you send an iframe, you're slamming the network. You're putting 20 packets on the kind of thing, all in a burst. So what we do is we split the image into small frames. Typically eight seems to be empirically a good number. Each of which is encoded independently, out of phase with each other. So sometimes you're sending an iframe for one piece of the grid, sometimes an iframe for another piece of the grid. And what this does is it takes the burstiness out of the transmission and is now sending a much smoother, less bursty stream of packets. If furthermore means that if a packet was lost, we're losing only a piece of the image rather than the whole image. And the good news is in our empirical studies, the pieces we typically lose are not the baby. It's usually some, there's a little sliver of the window where something is moving quickly, where things are changing fast. That's what our node looks like. It's quite an interesting thing, the mobile node. This is a really, it's not just an incubator. It's got a respirator, this has got air tanks in it. Image is a little hard to see. A lot of it is not our stuff, it's what comes with it. There are a bunch of cameras on top of the incubator looking down on the baby. There's a camera here looking at the vital signs monitor. Believe it or not, there is no data output on this device. And so this very clue-gee thing of pointing a camera at the screen, capturing the image, and then trying to reconstruct what the data really is. I thought this was really stupid until they changed the equipment on me. And then I realized it was wonderful because I didn't have to change anything. Whereas if I had known what their proprietary encoding of the data was, I'd have had to recode something. See, everything still worked. So it's a pretty, it's a silly, but it's a pretty robust mechanism. I guess it's why fax machines still live. And then there's a really fun part, which is the laptop computer, and all the ports for the modems, et cetera. So I'm just gonna show you a less than one minute video of not the system in actual action, but just to give you an idea of what it really looks like. Oops, I'm back. Maybe I'll show you a one minute video. So here we are loading it into the truck. The woman there is actually a new natal intensivist, and you can see why we call it a truck. It's really big. It's got room for up to two babies. This is me pretending to be a doctor and Monica Kleinman, who really is a doctor, operating the system and checking out the health of this doll. Here's a picture of the Vital Signs Monitor. The doll did not have a very good heart rate. Here we're zooming in to look at a lesion on the doll's face. Interestingly found, they do a lot of instant messaging back and forth between the hospital and the doctors. And just to show you what it really looks like, here's a real baby, not a sick baby. And this is a real image taken from the system in operation. So you can see it's pretty good quality video to be sending over the real time over the cellular networks. Here's one more one, and this one I'm just showing you so you can see my enormous linguistic skills. Now I can go back in again. Good thing I'm a computer scientist, right? Nope. Well, maybe you won't get to see my skills. You don't believe that's my voice? We get a lot of the funding from companies in Taiwan, so I produced a version of it in Mandarin, which, anyway. All right, just to show you what we get from the aggregation. These are, this is old data, but this was from an experiment that I did. So I'm going to show you what it looks like. So I'm going to show you what it looks like. But this was from an experiment where we had two Verizon lines and a Sprint line. And what you can see is that the mean for each of these is about the same. And when we aggregate them, we don't quite get three times the bandwidth, but close to it. So in fact, the aggregation works very well. I haven't shown you, I won't show you the data, but there's also a lot of data showing that the packet loss is typically, in many ways, uncorrelated on the different channels, which is also a good thing. This is just to show you, we spent a lot of time worrying about robustness. So here we were running an experiment with, I think, five cards. And, whoops, we removed the cards one at a time. These are the modem cards. And as you can see, it degraded nicely. And the nice thing is, this was all done automatically. This was to simulate dropouts and things. And it all worked quite smoothly. And during that period, this is the same period down at the bottom, the signal actually was reasonably static that we were receiving. We had enough bandwidth at 200, we were okay. Where is the project? We expect to actually deploy it for real use. This calendar year, let me emphasize the word expect. The IRB has been in progress for six months now. Why am I showing you this picture? Well, one of the first things that the IRB came back is I got an email saying, you don't talk about what you're gonna do about crashes. And I responded, my software isn't gonna crash. And they said, no, no, no, no, no. That's not what we mean. We mean this kind of crash. How are you gonna ensure that you're, for example, computer doesn't fly up in the air and hit somebody in the head? And we actually had to spend a fair amount of time doing a very different kind of crash proofing of the system. Apparently these vehicles have accidents fairly frequently. There is, of course, room for improvement. We can improve the UI. In particular, the collaboration tools. Our original model was wrong. We thought of it as point to point, but it's actually point to many points. The physicians often wanna call in specialists to come in and help them do things. The specialists might be sitting off and looking at things on a phone, for example. So we are working in collaboration tools. And we're looking at adding better and more kinds of sensors. Wireless cameras, pressure pads, ultrasounds. And as I mentioned to some of you, also, analysis and enhancement of the data. We think there's a lot of information that can be extracted from the video and analyzed by machines. All right, so I'm finished with part one of the talk. I wanna move on to part two. Totally different way of closing the loop. So one there, we sort of close the loop by getting humans in sooner. Now I wanna talk about closing the loop by getting humans out of the loop altogether. So it's about epilepsy. It's a disease of the central nervous system. Results in recurrent seizures. A seizure is a transient disturbance in the electrical activity of the brain. And it's important to say, for it to be epilepsy, the origin of the seizures has to be the central nervous system. There are other ways you get seizures. For example, cardiac induced seizures. Those are not what you think of, typically, as epilepsy. It's astonishingly common. Prevalence is somewhere, maybe, in the neighborhood of 1% of the population. Increasingly, it's becoming a problem in elderly people. About two-thirds of the people with epilepsy, the disease is well-controlled by pharmaceuticals, drugs. But about one-third have uncontrolled seizures despite daily treatment by multiple drugs. And that's the third that we're really focused on in our work. So here's what a seizure looks like. Pretty horrific. And even more horrific, I'll tell you, when you see these things in person. Yeah? I have a really big question on the point you made in the previous slide. So that epilepsy is more common in people over the age of 65 now? Is it that... No, I didn't. Well, it will be. Increasingly more common in people over the age of 65. So is it because you are seeing better survival rates in younger people, so that people with epilepsy continue to live as a society? Yeah, I don't have a good answer for why. Epilepsy is, in many ways, a description of symptoms rather than a disease process. There are lots of different causes. Some of it can be inherited. For example, ion channel deficiencies and things. But a lot of epilepsy is caused by injury, some insult to the brain. Most professional football players will at some point probably. We're gonna see a lot of epilepsy from these people from the wars who've been exposed to blasts, I think, where there's been an insult to the brain. So, but the answer, I don't know why. So this is, not all seizures are this bad. Some are very mild. They're so-called absence seizures, which are just loss of attention. The key thing that makes seizures bad is that, well, obviously having this is bad, but they are to the patient unpredictable. And so you can imagine if this young girl had been riding a bicycle when that seizure struck. The seizures are usually self-limiting, not always. And they'll just stop. If not, some patients go into status and then you have to intervene, but most of the time they'll stop. But what terrifies most people with epilepsy is not the seizure itself, but the collateral damage. That they'll fall down a flight of stairs and crack their skull. Everybody who's had epilepsy for very long has the battle scars of multiple injuries. About one in every 100 patient years, there's a something called SUDEF, sudden death in epilepsy patients. Nobody knows what causes it. And it's carefully phrased that sudden death in epilepsy patients, because they don't even know it's caused by epilepsy, but they'll die. That's pretty frequent one in every 100 patient years. And typically it's not from the seizure, it's from something related, right? One of the patients we've dealt with was a young mother had a baby, she would not pick up her baby. She had about less than one seizure a month and it kept her from picking up her baby, because she says, well, suppose I have a seizure when I'm holding the baby. It was very sad, right? So if you could warn her and say, you're about to have a seizure, and even if you could only give five seconds warning, it would be life changing for these people. So that was sort of a thing that really motivated us and got us excited, that we could actually, if we could do very early detection, we could really ease the burden on the patients and on the family members. There are also therapies, which there is reason to believe, and I'll talk a little bit more about this, if it's delivered in a timely fashion, can actually maybe abort or attenuate the seizures. Typically electrical stimulation today, but there are some pharmaceuticals under development. And as I mentioned, the alerts could make a big difference. The main question is, can we detect them? So you probably couldn't see it so quickly, but next to the gurus, having a seizure was a EEG recording, scalp EEG. So this is a technology for recording the electrical activity on the surface of the scalp. Not to be confused with the electrical activity in the brain itself, but indicative of that, yeah. Okay, so they're so unpredictable. So how did they get the scoral on, they had some outbreaks, she's gonna have a seizure at some point, right? Well, so we have videos we've made of these, and it's because typically we'll have the patient on the video for days at a time. Okay, so it didn't happen to be, that was the only time she was doing this. Now, typically what often happens for a lot of the things you see is patients go off their medications, they have them more frequently. And so if there are reasons I won't get into, why for periods of time, we take the patients off their medications, the doctors take the patients off their medication, and then they will have them more frequently, but we just happen to be counterpointed at her at this moment. Most of the time, there's no seizure. They are not that frequent, goodness, at least usually. Yeah, that's true. All right, so that's a record recording. You might think this is the seizure over here, but it's not. This is all muscle artifact that you're seeing, not brain activity. But we'll look in the little more detail at one later. The thing to think about is there are two kinds of seizure onset, clinical, where there's some visible symptom, and electro-graphic, where there's some change in the electrical activity of the brain. By definition, electro-graphic precedes clinical, since it's causing the clinical. That doesn't mean that we can see it necessarily in the scalp electro-graphically, because if it's in some deep part of the brain, it may not be visible promptly on the scalp. But for the most serious kinds of seizures, you can see it usually pretty promptly on the scalp. The EEG signal varies greatly across patients, and for this reason, the generic detectors have not worked well. Every EEG machine that gets bought gets shipped with a detector. All the hospitals I know turn them off, because they get so many false alarms that they're not very useful. Seizure-specific detectors work better. There's been some work mostly out of Germany in this area where if you say, this patient has seizures of type L or B or A, whatever, and you essentially hand code for that kind of seizure into the detector, you can do a better job. It requires considerable medical expertise to tune these to the patient, and it typically works best on typical kinds of seizures, the kinds you see all along. The good news is that while the EEG signal associated with seizures varies greatly across patients, it's pretty consistent for an individual. So a person with epilepsy will have a very small number of stereotypical waveforms associated with their seizures, most frequently one, but sometimes a other small number. That's not to say everyone is identical, in fact, no two are identical, but they share certain common properties. And this led us to the notion of dealing with patient-specific detectors, saying it doesn't have to work for more than one person at a time. If we could use machine learning to automatically construct a detector for each patient who needed one, we would just use that. And the question was, could we do much better? So the paradigm is pretty simple. You get signals from the brain, they go into some sensors, and we've been looking, today I'm gonna talk about scalp EEG, we've also looked at intracranial and deep electrodes, then extract features, and then build a patient-specific classifier. So for all of you familiar with machine learning, this is not a surprising paradigm. I won't have a chance to talk about it today, but we've looked at other sensors as well as the brain. There's often tachycardia associated with seizure and onset, that's to say, unusually rapid acceleration of the heartbeat. And so we found we can actually have many fewer false alarms if we look at both the heart and the brain simultaneously. The features are spectral, spatial, and temporal. The classifier we've been using is an SVM-based classifier using a radial basis kernel. If you don't know what a radial basis kernel is, don't worry about it. The key in the whole work was nothing to do with the choice of kernel or the SVM. Though the SVM is important because we have a huge class imbalance. We'll get 24 hours of recording during which we'll have maybe 20 seconds of seizure onset. The rest of the data points are not seizure onset, pretty big class imbalance for those of you used to using machine learning. SVMs handle it better than most. But the real magic sauce that Ali invented was the features. So if one looks at textbooks, they typically will talk about the spectral characteristics. Interestingly enough, most people who don't study this think that the activity associated with seizure is chaotic brain activity. In fact, it's just a reverse. Most seizures, what you see is excessive synchronization. That the things are too well organized. And so, but you see things happening in different bands at different time scales. So you have to extract the features and you can do this doesn't, we happen to use filters, we experiments with wavelets as well. Filter banks, wavelets, this seems to make not much difference which you use get more or less the same performance. We do that from each channel. In the scalpy EEG there are typically 18 channels if you have a full montage. You don't need a full montage. Eugene, she has done a lot of work showing that in fact, for a lot of patients we only need one channel, which is really convenient. But we start with those and then we look at them over time and get this vector in each frequency band in each channel. So eight frequency bands in channels. We've worked for a long time with this which is what most people work with, something similar to this. And then we discovered that it worked a lot better if you have temporal features as well. Because what we wanted to detect, what we discovered was useful to detect was the evolution of change. So for example, remember whether I ultimately left it in? No, I didn't. I had an EEG, I was gonna show you of a patient where that patient's seizures were marked by first a spike of sort of broad spectrum noise almost it looks like and then an electro decrement where the brain slowed a little bit and then some theta waves in the theta band. If the patient had only the theta waves it was not a seizure. If the patient only had the decrement it was not a seizure and the patient had spikes all the time. So what was interesting was to learn this sequence. Now we didn't want to hand coat it because there are lots of other sequences and the goal was to do no hand tuning of this. So we ended up aggregating the initial features into a different vector where we recorded them over six windows. And that was ended up to being the magic insight that Ali had that made things work. I wanna get to the results. This has been tested extensively. I'll report today on some results from a children's hospital MIT database which is, Ali did it at great personal cost but it's now available freely to anyone who wants to use it on a fissionette. Nine hundred and 16 hours of continuous EEG all from, well 23 pediatric and we had one adult patient. Story I won't go into. Typically 33 hours of recordings, five seizures per patient divided into one hour records. We did the testing, leave one record out and I wanna take just a second to emphasize the importance of this. As I got into reading the medical literature, I became pretty disgusted with the design of a lot of these tests. Not so much in the big trials but in the little tests that people would do for things like seizure detectors. In that they would, people especially who did learning would ignore the fact that these signals are correlated in time. And if you train on data that's temporally proximate to your test data, you can get really good results. Almost all of the things you wanna read in the literature are done that way. So I just caution you as you read this literature to be cognizant of that form of cheating. We tried not to do that, which is why we wanted to leave out a whole hour long record. So all the training data is pretty far removed from the test data. All right, so here's the sensitivity on this group. These are the children's groups, the children, the pediatric patients. We detected 96% of the seizures. Remember we only had on average five seizures per patient. For some patients we had only two so we had only one training seizure. What you can see is that the missed seizures occurred in a very small subset of patients. And so in fact, the good news is if you can do some testing, it's all right, we're just not gonna give these patients or a detector cause it doesn't look like it works for them. Most of the patients we have perfect sensitivity. Unfortunately sensitivity is not the hard problem in many ways, it's the positive predictivity. Here the number of false alarms per day on the median is two per day. Now, and as you can see again, it's highly variable across the patients. You could say, well, what good is it if you have two false alarms a day? Well, I could say compared to the conventional detectors which have order of magnitude more, it's better. But maybe if twice a day this mother were told this would be a bad time to pick up your baby, it wouldn't be so bad for her. If she were confident that it had perfect sensitivity, two false alarms, each of which lasted a minute would probably be perfectly acceptable to her and transformational to her life. So one of the things you think about these things you always think about how it's gonna be used and what could be the impact. How far ahead of the clinical seizure did you predict? That's a really good question. Again, it's highly variable for some patients as long as 30 seconds and for some patients after the clinical event. So again, but we can sort of tell. It's pretty consistent. It will be the same gap from seizure. So not every patient will be able to do this. You'll have to qualify the patients. Yeah, here's the latency, by the way. I'll get to you in a second, just a minute. So this is the latency measured from not the clinical event but from what we had is we had an epileptologist would detect the clinical, we find the clinical seizure on a video, typically. They would then go backwards on the EEG to find the earliest moment in which they saw what a change that they thought was related to the seizure and we declared that to be the electrographic onset. And this is our latency from that electrographic onset. The notion being since if only we're looking only at that information, that's sort of the best you can do. How much time is there generally between the electro onset and the highly variable? There's no way of knowing. And again, remember, we're measuring only what we can see in the scalp. So there are people who if the origin of the seizure is on some deep structure, may have some clinical event and we might never see anything on the scalp that looks like a seizure, at least not for a long time. Other people, it will, if the origin is close to the surface of the brain, you'll see it typically very promptly. I can't actually give you a real, a good answer to you. It's the right question, but I don't have the right answer. The question is better than the answer. All right, there's the latency. Here's the rate of learning. And here you'll see the latency in seconds and the miss rate. And you can see we clearly train very quickly and starts to flatten out. We've consistently seen that after three seizures we're typically not getting much. And the good news is we've had patients come back a year later and tested them on the data trained and it works just as well. So it seems to be pretty stable across time for patients. This is a comparison to what I think is probably the best commercial product. This was also built using Malik machine learning, using neural networks, something called Reveal. They trained on thousands and thousands of patients and then built a non-patient specific detector. There are parameters you can adjust. I should say this is not designed to be an onset detector, but a seizure detector. The parameters of L and T, L is the latency, how long it waits before it declares a seizure. So it needs to see 20 seconds or something it thinks it will see, think there's a seizure. And T is the probability of it being a seizure is 0.9. So you can set these to anything you want. You get the trade-offs as you would expect. If you make L bigger, sensitivity goes down. If you make L smaller, specificity goes down. Lots of different things. We picked this and you'll notice with even a very long latency, well it's a long latency here. Detected 93 out of 152 in the same population. We got 147 out of 152. Big difference there and a lot of difference for patients. But the more interesting piece is the specificity, same parameters. So we win here on sensitivity. You say, well, you cheated. Maybe you should have given them parameters that would improve their sensitivity. But if we do that, their specificity becomes impossible. As you can see, even here with those parameters, we have two, their median was 14. Our average was four, their average was 40. So there's a huge difference and it gets to be unusable for most of these patients. So the notion, the point I'm making is not that we have the world's best patient-specific detector. I think we do, but the state doesn't prove that. But the patient-specific detectors are gonna way outperform generic detectors. Do the patients follow your clusters in terms of the patterns they exhibit? We, with 23 patients, I don't know. I would expect they would, but we couldn't tell you that on the basis of the number of patients we had. Just sample size was too small. Very quickly, I wanna talk about one application. This is a vagus nerve stimulator. This is a device that gets implanted under the clavicle attached to the left vagus nerve. Put the longest nerve in the human body, by the way. Puts a stimulus on it. It's kinda like a diode in the nerve. The current flows up into the brain, causes chemicals to get secreted. It's been shown that repeated stimulation of this nerve can, in many patients, produce the average number or severity of the seizures. It's been shown in rats, in case you care about the life of rats, that if you turn the stimulator on at the onset of the seizure, it has a high, very good chance of actually aborting the seizure, stopping it in its tracks, or at least reducing the intensity and duration. Very exciting if you happen to be a rat. Nevertheless, there have been tens of thousands of these implanted, mostly for the repeated effect. But everybody who gets one is given a magnet and said, when you're having a seizure, hold the magnet over it, it will turn it on and maybe it will help. So about 70% of the patients who get these report using the magnet. Either the patient or sometimes a parent. One of the mothers we talked with wears her magnet as a necklace. And whenever her child is having a seizure, runs and holds the magnet to turn it on. 20% of these people who do it report that when they do it, it stops the seizure. 30% report that attenuates it and 50% say, I keep doing it, it doesn't help. Don't get fooled into thinking that this means that it helps 50% of the people. How do they know it attenuated the seizure? They don't. There's no way of knowing how long it would have lasted since the durations are different. And in fact, even the termination, people say, well, I think I was about to have a seizure and I used it and I didn't have a seizure, it worked. Well, you know, if you carry an elephant gun around Ann Arbor, you probably won't get stampeded by an elephant. So who knows? So we built a system to do it with a closed group control, stuff in the boxes of stuff we built. We acquire the signal, we use our detector, we energize an electromagnet, turns on the stimulator, voila, the patient gets the kind of klugey-looking hardware. Is this an ambulatory system? No. But it's good enough that we can do a lot of in-clinic tests, which we're now doing. And it reliably works. It reliably detects the seizures, reliably turns on the stimulator. Here's an example. Here's a patient, that's the electrographic onset. This is where we detected the seizure and over here the spike is the VNS getting turned on. And this, you know, we see this over and over again. As you can see, by the way, it's not obvious to the eye that this is where the seizure starts. So it can be kind of subtle. Does it help? I don't know. Worse, I don't even have an opinion. You know. But I will in about a year. We have a study in progress. We've got patients lined up. We're running studies where we're sometimes turning it on, sometimes not turning it on. We'll actually have real data. And maybe in a year from now, I'll be able to tell you whether it works or not. Okay, wrapping up very quickly. I can skip most of this, but it is getting late. And I'll just get really to the bottom two points, which is that personalization is critical. You really need to find some way to use learning to build a model of the patient, to know the difference between what's normal and what's worrisome. And it's just, I think, a great, almost virgin field of research opportunities for CS and EE researchers. And with that, I'll thank you for staying beyond five o'clock. So I'm happy to take questions. I'm also happy to turn my back so people can go out of the room without me. Yeah. So in some cases, your epilepsy patients have like auras, or your sensory hallucinations. Can you use partial seizure? Definitely seizure, but it's kind of a step towards it. That would give you more training information. Can you use that on those patients? That they know they've had an aura? So auras are an interesting thing. So there are patients who, for example, will smell something in advance of having a seizure or hear something, or various kinds of things which are called auras. And for the lucky few patients that those occur reliably before seizures, they're not so unpredictable. Most patients do not enjoy that privilege. Now, what's going on with an aura? Again, this is speculation, but the speculation is that in fact, it does mark the onset of the seizure. And what's happened is, for example, there's some seizure activity in the neighborhood of the part of the brain that related to smell that has not yet reached the parts of the brain that deal with motors, motor cortex. And so, yes, there's a seizure, but there's no, say, effect on muscles yet. We could certainly use that for training. Now, the problem is most of these patients with epilepsy have abnormal brain activity all the time. Their EEG, their baseline EEG looks nothing like the baseline EEG of most people. And they frequently have a leptiform discharges that do not evolve into seizures. And frequently have therefore things like auras that do not eventually evolve into seizures. And the difficulty, a big difficulty in our work has been not generating false alarms on these discharges that are not actually going to turn into seizures. Now, it's arguable in the literature whether these are, quote, seizures or not. Right, and if you look at a outpatient epileptologist, if you talk to them, they will say, if there's no clinical event, it's not a seizure, I don't care what the electrical activity looks like. If you talk to someone who works in an ICU, they will tell you, no, the electrical activity is what determines whether there's a seizure. So there's, the end point is not at all clear here. And what you choose to train on is going to depend upon your definition of what it is you want to learn. Do you want to, for example, fire the stimulator in every discharge or not? So I think that's a long answer to a short question. Yeah. I want to say something about the experience of computer scientists working with medical doctors. How does that kind of feel? I like it a lot. Now I'm very lucky that not all of them, but most of my clinical collaborators, medical collaborators have been really a pleasure to work with. I've chosen to work with people who all do some clinical activities, so these are not bench scientists but care providers. And they care deeply about finding ways to improve the health of their patients. And it's kind of inspiring sometimes. It sounds trite to say that, to actually talk to people who are on the front lines. And it's been wonderful, you know, it's wonderful to be able to meet patients. I think, geez, I could maybe someday help this person. Now I choose my collaborators carefully. For the most part, I like to find collaborators who have a background in engineering or mathematics or physics. And there are a lot of doctors out there today who had the wisdom to do that as an undergraduate. Because really what I want is collaborators. I don't want them to throw clinical problems over the wall to me and say, come back when they're solved. Nor, you know, I really wanna be able to sit there and say, we can't, you know, sit there and say, we're getting all of these problems. What can we do? And the doctor says, well, if you thought about the fact that there are these patterns that maybe we could look at. And it really is very helpful. And, you know, to be able to sit and find a doctor, you can say, well, do you think the data is gonna be linearly separable? Well, none of them can answer that off the top of their head. But finding a doctor who is willing to take an hour to listen to you explain what that means. And I've been able, for the most part, to find collaborators who really wanna do that kind of thing. So the downside I've found is it's easy to find people in the world of medical research, more than clinicians, who are so obsessed about the number of publications they have that they treat data as a proprietary asset, rather than something that could be useful to the community. And so when you run into people saying, I don't wanna do any work with you, I'll give you a little data and I'll meter it out a little bit at a time. And you have to promise to put me and my dog and everyone I've ever met as co-authors of whatever paper you write. So to a computer scientist is kind of a foreign culture. But on the whole, it's, for me, been a great experience. And I really do value my clinical collaborators a lot. You mentioned that you get funding from Taiwan, is NIH not so willing to fund experimental kind of clinical work? Well, I'll tell you in a month or so. So initially, the problem with NIH, right, is I had no track records. And typically, NIH likes to fund people with track records. And so we had to actually get some publications, some data. An NIH proposal, unlike an NSF proposal, typically has to have preliminary data in it. Pretty convincing preliminary data. And it was only recently that we've been able to write a credible proposal with enough data. I have two proposals now pending with NIH. So is it worth a hassle compared to getting a guided source like that? Yeah. You know, I was able to hassle. There was a tremendous amount of boilerplate that goes into these proposals. I'm sure you're all familiar with that. But on the other hand, unlike if it comes through, it's for five years, which is a wonderful thing. And typically renewable. Narrow one. Yeah, Narrow one. So I've got one of those pending and got another one pending. So I'm optimistic. It's an issue. It's been an issue with publications. You know, the stuff that I find really the most interesting in our work is hard to get. The medical literature doesn't so much care about it, right? They don't want to know about this wonderful algorithm that you've invented. But we'll see. So I've been able to... Privates funding, I guess DOD funding. The DOD is pretty interested in brain injury right now for obvious reasons. And that's mostly where it is. Yeah. Yeah, so when it comes to publishing, so you've only been working with commissions who don't care about publications. Oh no, they care. So when it comes, do you publish an article in a computer science conference and then also in their journal? Right, so we've been following a dual publication strategy where we'll typically write an engineering paper that has all the beautiful technical material in it. And the medical papers are often very formulaic where you give a sketch of the actual technique and then you give results. And so we've been following the strategy of dual publication in those two venues. We've been, it's been a while. I mean, for me, I was a computer networks guy before I did this stuff. So I didn't, you know, not only did I have no credibility in the medical world, I had no credibility in the machine learning world. So, you know, it's been a little ramp up, but this year we've got papers in ICML and NIPS. So I think we've sort of gotten over the hump. For what it's worth, medical decision making is a nice, can be a nice little room between the exclusively technical paper and the exclusively self-indicated paper. So, because they will care to a certain extent about the technical aspect. I guess more if you have data, you know, more of a well-defined data analysis and methodology like they would call it that you could nail down it. And that gives you some sort of middle ground if you're looking for something that, an audience that would appreciate it a little bit more. And we've done some of those things. And if they're Amy, you know, Jamie is kind of the middle ground and stuff like that. Journal of the American Medical Informatics Association will take some technical, but not overly technical. Yeah. Yeah. The two sort of medical areas that your project's talked about were very dramatically different, right? Yes. In terms of what part of care they address. Are there any portions of medicine that you view as like especially right for computer science intervention where you just pick up the problems or you find them? So a combination of picking up the problems where I find them and where I think my students have some ability to contribute, right? We're not, I'm not a biologist. My students are not biologists. So I have studiously avoided those problems that require deep biological insights or understanding, right? Not because they're unimportant, but because they're beyond me. So that, that's one thing that's just things, a lot of important things we're just not going to do because we couldn't. There are things that are right. So I, you know, so the telemedicine is sort of an offshoot of my networking life. But if I look at the cardiac work and the epilepsy work and other work, we focused on areas where there are either signals because we know some signal processing and electrical signals especially were good or large amounts of data. And you can really do data-driven kinds of things. One of the things I didn't mention though when I talked about my clinical collaborators is I don't want to get carried away with the statistical aspects of the work. And we do spend a lot of time with the clinicians trying to understand the physiology and seeing if we can put together physiological explanations of what we're seeing, right? So that there's some underlying reason to believe that it's not just some statistical aberration but something that makes sense where there's some physiological model that at least plausibly fits the data that we're seeing. Areas that I think are tremendously right, it is especially for the brain work, it is everything related to psychiatry. That there's so little engineering, if you will, that goes on today in those kinds of diseases which are very important. And I think if you could find objective measures of things like schizophrenia, is there a difference between when people hallucinate that they're hearing something and they hear something? Different parts of the brain light up. What happens when you give people these various drugs? How does it change the electrical activity? So I think because this stuff has just not been studied and say to the same extent that cardiology has been studied in a scientific way, I think there's a lot of interesting activity to do in various diseases of that sort. I'm interested in moving into various neurodegenerative diseases where again I think there's a lot of interesting things that we can measure and look at particularly what's going on. We do early detection. So I've been focused on things where early detection is valuable. But there are lots of things, right? I probably would avoid orthopedics, which I think will belong more to mechanical engineers than to computer scientists. But the other thing to look at today is a proliferation of devices of one sort or another. The neurostimulation devices are just exploding. And they're exploding in ways where the control systems for them are very unsophisticated. People implant ICDs. And they really don't have a good understanding of when they should be energized. Yeah, sure, if the heart stops or go into VTAC, you should turn it on. But are those the only times you should turn it on? Does it always have to get turned on full? Could it get turned on partially? None of these are questions that people know the answer to. And I think there's a lot of stuff that we can do. And in general, I think a lot of it will be for us is stuff where personalization is important. Because that's the stuff where you really need computation. Because it's just not possible to have an expert tune these devices to each individual. So I think anything where there is personalization is a great opportunity for us to look at. And I like brains and hearts. They're important going against each other. So do you want to have some cash plates? Well, I think I'll take one more question. Does any of you have any? I have to teach tomorrow morning. All right, Phil. Thank you. I get to talk. Thank you.