 Welcome everybody to the open ventilator remote monitoring project. This was a project that we did. The open source community came together to create software to remotely monitor, do it yourself, rapidly manufactured ventilators during the COVID-19 pandemic. So by way of introduction, my name is Sam Cervantes. I'm the Chief Technology Officer at Maker Gear 3D Printer is based at Cleveland, Ohio. And also leading the project with me is Dave Guthrie. He's the medical device cybersecurity program manager at Mass General Brigham Health Care. So we're going to give you a little description of our project, what we did, and how we went about it. So over to you, Dave. OK, so this has been the year of COVID. We've all been impacted by this global pandemic. And early on in this pandemic, there was fears of massive shortages of ventilators. For example, Governor Cuomo of New York predicted there would be 26,000 people who could die from ventilator shortages. In Italy, they stopped intubating patients over the age of 60. And the New England Journal of Medicine predicted a need of anywhere from 160,000 to over a million. So there's a global pandemic. Early on in this Massachusetts hit pretty hard by this as far as the United States is concerned. Mass General Brigham is the largest health care provider in Massachusetts and has a significant need for ventilators. So Mass General Brigham reaches out to the tech community to develop new technologies, new processes, and has to be rapidly deployable in order to meet this predicted growth and rate of infections. And you need to do this in a massively condensed timeline. So typical medical device development takes two to three years. This needed to be done in two to three weeks. Crazy. So Mass General Brigham reaches out to the open source community to develop rapid prototype ventilators. Open source community responds. This is an example of just some of the resources that the open source community has provided. And Sam is going to go into this in more detail later in this presentation. So now you have the open source community at the ready. Working groups are starting to form to develop hardware systems, control software, and remote monitoring software. All of these are being done in parallel under this condensed timeline. So what's the initial need and scope of this project? So you have this predicted ventilator shortage. You have a need for these rapid prototype ventilators that the community is now building. However, most of these ventilators lack built-in monitoring. And clinical representatives need to have central monitoring expressed this need to have central monitoring similar to central nursing station networks. So in a hospital, you typically would have like a GE network or a Phillips network or a neon code and network or some other monitoring network where you have bedside monitor, bedside physiological monitors going out. And you have a network where you can go back to a central nursing station and oversee the data and the alarms of all of these patient rooms. And you have ventilator systems that can then plug into these bedside monitors to jump onto this network. And then you can monitor these ventilator systems as well. Well, you can't do that with these rapid prototype ventilators. So you have these open source teams actively developing these ventilators and now you need to have remote monitoring and central monitoring of these systems. That's where this team comes into play and it initiated in April 2nd of 2020. So what's the clinical request and the need for this remote monitoring software? So you have this rapid prototype ventilator groups. They're not integrated into central monitoring traditionally with hospitals. So you need a method of monitoring of all your ventilators on a given floor or a given unit. You have to have a central dashboard with visual learning that's easily interpretable. You have to be able to ingest device data from all these ventilators. So not just your patient device data but also device status data and alarm states and alarm data from ventilators if available. And again, you have to do this in this wicked condensed timeline of two to three weeks. And you shouldn't cause patient harm or an eruption in patient care because if you do that, you're defeating the purpose of helping clinical staff and instead you're getting in the way and potentially causing patient harm. So this is where Sam's company Maker Gear comes into play. They have a 3D printing company and they have software that has the ability to integrate with physical hardware as well as providing a centralized dashboard. So this was the initial code base that was hacked and adapted by the remote monitoring team to develop this ventilator monitoring system. So now what happens is we're actively developing all of these rapid prototype ventilators and the remote monitoring team is working on integrating with these ventilator teams in order to build a central monitoring system. However, simultaneously hospitals are procuring ventilators left and right. You're trying to build up your ventilator inventory from traditional commercial vendors. And what happens is there was not a need to then deploy these rapid prototype ventilators because you're shearing up your inventory of already off the shelf commercial ventilators. And the problem though is a lot of these new ventilators may not have been the same model that you already built out integrations for. And so the amount of time to design these integrations was extremely costly, resource intensive and you just can't do that in emergency response situations. So you have new commercial ventilators that can't integrate into your central monitoring system. On top of that, you have infection control restrictions because this is a global pandemic. So you have closed doors for infection control reasons. You have ventilators inside of these rooms that can't be centrally monitored. And now you have alarms going off with these ventilators. Now clinical staff are having trouble hearing alarms through these closed doors. So this is where our team now pivots and we are requested to create a method for monitoring these commercial ventilators in a central means that don't typically integrate with the other traditional central monitoring systems. So what's our plan of attack? So MGH Biomed, for example, researched off the shelf sound monitoring solutions. So they've acted in on one product, the sound ear. So the sound ear can deploy a one to one system per patient room, per ventilator. It's a self-contained consumer off the shelf system that can monitor sound pressure. So you can hook a sound ear with a microphone up to the speaker on the ventilator that auditorily alerts the alarm. And then when the sound pressure is exceeded and then alarm goes off, the sound ear can then trip. So we were then requested to integrate with this sound ear and provide a central means for monitoring the sound ear and be able to create a central dashboard for when it trips. And now I'm gonna pass it over to Sam to go into the architecture. Thanks, Dave. So I'm gonna talk a little bit about the technical aspects of the project and how we went about pulling this off. And then also later on, some of the kind of the future goals. So this right here, this is the user interface for the clinician. This is the whole goal of the entire project right here. So this is what the nurse is gonna see at the nursing station. We've got one little box here for each ventilator. So we've got, in this case, 48 ventilators monitored from our single dashboard. So this is every ventilator on the entire floor of the hospital monitored from one central dashboard. So we've got three states of the ventilator right here that we can display. The normal state, the happy state is green. That means we're able to ping the ventilator. We're getting good status report back and there's no alarm, all is good. The second state is this gray strike through box right here. That means we're not able to even connect with the ventilator. So a technician needs to go check on the ventilator. So there's a fault in the system somewhere. The third state is this red box here and that will actually flash. You'll see in the live demo. And that means an alarm is currently being triggered on the ventilator. So the nurse needs to go check on it. So at a glance, the nurse can see right here, okay, I need to go check on two different ventilators right here. Room seven and room four, I need to go check on those. Okay, so how do we go about doing this? Typical deployment process for software like this is two to three years. We had two to three weeks, COVID-19 emergency. So how do we go about pulling this off? Basically we had to pull all the rapid prototyping of software development strategies. We had to move fast and break things. Well, I heavily on the open source community and the maker communities distributed development. We had the student software developers all over the country pushing to our GitHub repo. We had to hack and fork existing products to get something out in just a few weeks. We used things like 3D printing and quick turn PCBs, off the shelf hardware to get something that we could barely quickly snap together, bolt together and ship a solution to our customers in just a very short amount of time. We had to rely on remote lab testing. And as much as we could source locally, component sourcing was a key thing. We had to design our system around the components that we could actually source that were in stock and locally available. We might have wanted to use a PLC or a custom circuit board, but we didn't have time. So we had to use things like off the shelf, Raspberry Pi's and Arduino's that we could get in quantities of thousands that were actually in stock. So basically we had to move heaven and earth to ship something in a very short amount of time. Like Dave said, the initial code base was hacked from the Maker Gear software. So Maker Gear manufacturers, 3D printers been doing so for about 10 years out of Cleveland, Ohio. And it just so happened that Maker Gear had some software for remotely monitoring 3D printers. And as it turns out, 3D printer is an awful lot like a do-it-yourself ventilator. 3D printer has a Arduino and a Raspberry Pi and a ventilator has an Arduino and a Raspberry Pi. So we were able to adapt and import the software to quickly meet the needs of the pandemic. So this is the overall architecture here. So you'll see the dashboard is what we saw before. The dashboard is the kind of central brains, if you will. All information flows towards the dashboard. So this is the clinician interface here. So the dashboard is gonna get its status data in real time from the ventilators. You'll see three ventilators displayed down here. Each ventilator is in a different configuration. And you'll see the cloud up here. So the cloud is there only to serve up the application to the browser. And the cloud doesn't store patient data. The cloud simply stores configuration data for the app, a very limited amount of data. We don't send patient data to the cloud. So how does the dashboard get its information from each of the ventilators? It was kind of a tricky problem for us because each ventilator, there are a number of different types of these DIY ventilators out there. We wanted to make a universal system to pull data from every ventilator. Some ventilators are controlled by an Arduino. One of the MIT designs was controlled by an Arduino. Some ventilators were controlled by a Raspberry Pi. I think the Jambeth design used that. And then some ventilators are a little different. So we wanted to have a soundier, a separate appliance to listen to the sound the ventilator's making and then trigger an alarm based on that. In order to interface with all these different types of ventilators, we decided to put a Raspberry Pi in between the each ventilator and the dashboard. So the Raspberry Pi could serve as a kind of universal network interface. And on the Raspberry Pi, we have a library. And so we can write modules for this library to basically ingest data from any system. So the system we set up, the dashboard we set up can basically take data from pretty much any device with any output capability. If it's a stereo out, if it's a spy or a UART out or whatever, analog out, we can take that data from the ventilator and relay it to our dashboard here. So this is the overall architecture. And the specific instance that we tested in the lab is this particular architecture here. So the ventilator is obviously gonna be in the patient's room. And then also in the patient's room is the sound ear device which listens for an audio alarm from the ventilator. The ventilator is gonna emit a beep. And you'll hear this in the demo. It's gonna emit a beep. The sound ear is gonna detect that audio beep, that sound pressure. If the sound pressure is above a certain level, it's gonna send a signal to the Raspberry Pi, the network adapter. And then the Raspberry Pi is gonna then send the alarm to the dashboard that we saw earlier. The nice, we should note here that we used the zero tier software defined network, which provides a very secure network in the hospital and even outside of the hospital if we need to to provide connectivity between the Raspberry Pi and the interface here. And of course, you've got the cloud here. And again, we're not storing patient data on the cloud. It's just there to serve up the static assets, the application, if you will, to the dashboard and also very limited configuration information. And again, this is what the nurse is gonna see at his or her station right here, the dashboard. Software stack, we're getting a little bit into the weeds here. We try to use as much open source software as we could here Ruby on Rails, reliable, secure, open source, very quick to deploy. Raspberry Pi's and Arduino's, again, open source, off the shelf, quick to deploy. And then everything we did is open source and we're a GPL v3 license. You can pretty much use it for whatever purpose you'd like. GPL 3 is a very permissive license. You can use it for commercial and or non-commercial. We wanted hospitals to be able to use this very quickly and use here GitHub link down below. Everything we have is up on GitHub. Did a lot of testing at home testing, workbench testing. We did lab testing. We did not get to the point where we actually tested it in a hospital, but we did a lot of lab testing in a clinical lab and it actually worked pretty well. And we got a very solid concept alpha. We didn't quite make it to the beta phase, but we got a pretty good alpha prototype. So one of the lessons we learned was the validation and the regulatory process was actually very involved, as you might imagine, from medical products. So a lot of us come from a typical software development background, but a medical device has a much more involved regulatory and approval documentation process. We just wanted to give you a flavor of what that is right here. So the main three documents are the requirements which have to be formatted in a certain way. The architecture, which again, needs to be formatted in a certain way, and then the hazard analysis, which is pretty involved. Basically for the hazard analysis, we had to brainstorm every possible way that this product might fail. You either fail to communicate an alarm to the central station or fail in some other way. So every method we predicted that it might fail, we had to put a contingency plan in place for. So in very detail, again, just wanted to give you a brief overview of what this, for those of you who haven't done a medical product before, just a brief overview of what that involves. It's a very rigorous process to prepare for FDA approval and hospital approval. So just a little taste of that right here. And again, this is the detailed software architecture. If you want to take a look at that, all these documents are up on our GitHub Wiki, so you can check those out there for details. Another important thing here to do is to do a secure code review to just kind of get a good confidence level since this is potentially a critical application that we don't want to fail. The staff at MITRE did a very nice job and let their support, so we were very grateful of them. They used an automated code service called Veracode and also a manual code review. Still ongoing, they were a big help with that. Just to give you a taste of the automated code review, this was a code review on, I think a Ruby on Rails portion of the application. Basically, you see here, we don't have any severe alerts, but we had a couple of medium alerts. This was generated from some, we had some automated testing code that spit out automated test data. So we need to remove that before deployment. So one little minor thing here. So a few things to fix, manual code review. Again, just a few things, nothing critical, a few things to fix. So we just want to let, just kind of give a little insight to how we went about testing and validating our code. We learned a lot in this project. We did have a few medical experts on our staff, both from the software development side and a lot of medical experts from the regulatory and healthcare field. So a lot of us were software developers. So these are some of the lessons we learned. The most inspiring thing I think was, we had a lot of world-class experts to come together very quickly to donate their time, their nights of weekends, and to really come together and work for this good cause. So that was really inspiring. The biggest hurdles we found were not necessarily technology, it was as technologists, we found it, I don't want to say it was easy, but the technology was the easiest part in a lot of ways. The validation and approval were much more rigorous and time-involved than we anticipated. All that being said, FDA approval is daunting, but it's not impossible. And there are experts out there that are willing to help, and that's what we found. A lot of really good people with a lot of deep FDA experience out there that are willing to help. So it's not undoable. We found it's very important to be aware of HIPAA and personal identifiable information. So that's one reason we chose not to send patient data to the cloud. It's very important to be cognizant of HIPAA. Another important lesson we learned was that hospitals are elected to use open source software without the strong backing of an organization to own the technical support and the liability aspect of it. So in the future, we're looking for potentially a sponsor for the project to kind of own that deployment in technical support aspect. And ultimately, we learned we didn't have a ventilator shortage in the US, which was great news. So ultimately our software was not needed. We're very happy that we could be there and kind of rise to the nation's call as an insurance policy, but ultimately there were plenty of ventilators from what the experts tell us in the US to go around for people needed them. Okay, so FDA approval. We just wanted to give you a little flavor of kind of the difficulties we found in the FDA approval process. Big disclaimer, I'm not a lawyer or a medical expert. So please don't take us as medical advice or legal advice, seek expert advice for your own project. But it's just a high level overview of some of the things I learned. In a normal timeframe, non-COVID, non-emergency timeframe, say last year, a typical approval process for a system like ours would be about two to three years. It's a lengthy submission process. Once you get that approval process, you can use your product for a long period of time. During the emergency, for example, the COVID emergency, a couple of FDA allows a couple of types of new approvals. One is what's called EAU, Emergency Use Authorization. Typically it can be fast-tracked into two to three months, it's much quicker, but EAU is only valid for the duration of the emergency. So if we were to get EAU for our software system, we would not be able to use it after the COVID emergency. And then Compassionate Use is potentially even quicker. And that's a, I'm not an expert on that, but it's a slightly quicker process, even more limited in scope, even more limited in approval, but potentially quicker. So again, not a legal expert, not a medical expert, these are just some of the things that we learned in the process. There was even a little bit of discussion as to how to classify our software system. Is this a medical device? Is it a modification to an existing medical device, or is it not a medical device? Is it a baby monitor, or a pager, or like a cell phone type device? So we did have some ongoing debate about that. So current status of the project, we drafted up our validation documents for approval. We have a working demo proof of concept. I called it Alpha, it works. We tested in the lab, we had a code review. We do not currently have a customer for the project. Our project is currently parked. So if you have a potential use for the software, do let us know. And we would love to see our software be used for a good purpose. Future plans, one thing we discussed is additional audio alerting at the central nursing station that may require, in addition to the visual alerting that we already have, that may require separate hardware appliances and potential AI machine learning to detect different types of audio alarms from different types of medical devices. We're gonna search continually for new customers. Potential customers may be hospitals with limited resources, smaller hospitals in remote areas or developing countries. And then potentially healthcare organizations with needs to monitor other devices like pulse oximeters or things like that. And we're continuing our search for a corporate sponsor, an entity to own the maintenance, installation, support, and liability. So we're just a group of distributed open source software developers that we don't have an organization behind us. And then potentially a future FDA application, again, if we find a customer for this. So Dave's gonna talk a little bit about power and the possibilities. Yeah, so when you think about this system, the real power is in the possibilities of all the integrations that this system can provide. You have rapid prototype ventilator monitoring systems that we've discussed, such as bringing in device and patient data. You have commercial ventilators that are not traditionally integrated with central monitoring systems that you can monitor through sound. So you can get ventilator alarms from commercial ventilators. And now with the sound monitoring, you can monitor any medical device alarm that is based in audio. So think about all the infusion pumps or other devices that you have all throughout your hospital or remote care sites that potentially do not have integrations into your central monitoring system. This could be huge. So this project has great potential to have a massive impact in the healthcare ecosystem. Now for the fun part, we get a live hardware demo. So we're gonna show you the system in operation. So Dave in Massachusetts has what would be in the patient room. What you see on this picture is what's on the bench here in Massachusetts. So the way you're gonna see in the video is gonna be from that camera that you see in the forefront of the picture. So you have the speaker in the background next to the cyber lizard who has a microphone next to the speaker that microphone is plugged into the sound ear. The sound ear is then going to calculate the sound pressure and it has what's called a sound buster port. That sound buster port changes voltage if the audio alarm pressure is tripped. So then the sound buster port comes out of the sound ear into this Raspberry Pi and then the Raspberry Pi itself will then go out through the network cable over to a switch that's on my bench here out over the open internet through the encrypted zero-tier tunnel over to Sam where he's running the dashboard. And what I'm going to be doing also is running a ventilator recording of the audio alarm that we used to train this system. So Dave, you've got the ventilator system. You've got the patient side system in Massachusetts. I'm in Ohio. I've got the clinician dashboard. On my clinician dashboard here I can see a green box which means I'm able to get a good signal from Dave's device in Massachusetts. Again, all this is communicating over the zero-tier secured network or across the country. We might as well be across four in the same hospital but I'm getting a good signal back here from Dave's ventilator. And Dave, if you want to go ahead and trigger your alarm there, we should see an alarm indicated here. There we go. The flashing red means the nurse at their dashboard needs to go in and check the patient room. And as soon as we shut it off, the alarm should go off as well. Okay, so that's our demo. So like you see, it's a basic working system. It's fairly simple at this point but we do have the capability to monitor up to 40 ventilators from one dashboard and it's extensible. So we have libraries that can easily incorporate any number of devices. Pretty much any device that's got some kind of hardware, digital or analog output, we can tie this into our system. Very, very adaptable system. So just some acknowledgments for all of our developers. I was really honored to be a part of a team of so many people, much smarter than myself. Really fun and really great to work with this team. Dave and I led the project but we have a bunch of other open source contributors here. Everybody volunteered their time and effort and sacrificed their nights and weekends to really just contribute to the COVID cause. So words can't express my appreciation for everyone's time and effort here. Really inspired to see so many people come together such for a short time. Thanks to everybody. Organizations here, just wanna thank all the organizations, GHS and FDA for their infrastructure and their funding. Make a gear for contributing development time from the number of engineers and also open sourcing the code base and contributing hardware resources. Massachusetts General Hospital and their affiliates for everything they've done and expertise they've provided. MITRE for its code review and consulting and just expertise and late night phone calls, protocol labs for special thanks for contributing some funding. Soundear, so we contacted Soundear to the manufacturers of our sound interface device and we asked them for help on a very short timeframe and they were incredibly supportive. Not only did they give us technical support, engineering consultation, they open source the portion of their code for us and you'll see that on GitHub. So I cannot have, I cannot imagine a better response from a manufacturer in a time of crisis. They were incredibly forthcoming and I cannot thank them enough for helping us to integrate their device in our system. And then last but not least, their code for their automated security review. So if you would like to become a contributor or a customer or a sponsor, please reach out to us. You can find me or Dave on LinkedIn and I just wanna give an extra shout out to everybody who worked on this project and an extra thanks to everyone. It was really inspiring to see so many people involved in this and in such a selfless way. So it really gives me a hope for the future that we can lean up this thing by itself. Thanks, everyone.