 Please welcome Masrin and Samia that will talk about automated emergency paramedical response system and I guess there will come some impact from artificial intelligence also here. Go ahead. Hi, thank you so much. I know it's pretty late and I appreciate all of you staying back and so this was a prototype that we developed some time back for the title is self-explanatory. My name is Samia and this is my colleague Masrin and so one of our main needs is that we know that there's been a lot of advancement especially in the medical sector in terms of tech but there are some systems that are when we look at it we feel like it's really primitive like you we look at some things and say oh this could be done so much better this could be so much more efficient if you know it had small little tweaks in it and of course we already know that a few minutes can make a big difference between life and death. So that what we wanted to do with our prototype of an automated emergency paramedical response system was to fill in some of these gaps that we thought was there in the current medical system. One of the driving motivations that we had was to provide a feasible solution that spans the entire socio-economic strata as we know that the problems that people in urban areas have in terms of medical care is not the same as people in rural areas or semi-rural areas would have in terms of medical care. In urban areas especially in countries like India where I'm from we do not have a dedicated lane for ambulances so when we need emergency medical response there's a lot of time it's too late we it's just not fast enough whereas in semi-urban and rural areas reachability is the problem for medical care we need to travel kilometers and kilometers there are no specialized doctors that can be that can be present in semi-urban and rural areas what we thought was the best way to address these both of these issues was a drone based paramedical response system. So our prototype that we developed has five different main modules that we'll be touching upon in a little bit and how they interact with each other and make the system whole. One is blockchain for medical records disease detection chatbot a DD disease detection model which is basically an umbrella of specialized healthcare services that you could provide to people in semi-urban and rural areas and a drone for medical delivery and facial recognition during delivery. Blockchain for medical record there are a lot of people who have a divided opinions about what blockchain means and why would you want blockchain for medical record but what we decided to do was implement a blockchain for our automated let's just call it AAPRS the name is too long so this is in the interest of both patients and doctors let me tell you what. So there are a lot of times where doctors face problems if patients do not declare their medical history in cases of communicable diseases like tuberculosis or diseases like AIDS we know that doctors need to take extra precaution if a patient has communicable diseases like TB or diseases that can be transferred through the blood like AIDS. The many patients do not want to tell their doctors that oh I suffer from TB or AIDS or any of the other diseases because they feel that you know they would be discriminated again. Also another thing that we found was that availability of patients complete medical history enables the doctor to find out is something that you're suffering today because of something that already happened to you some time ago or is it unrelated. At the same time patients need complete confidentiality about their records and patients don't have to carry 1000 extra years and so many forms of different hospitals different consultation papers they want that only my current doctor should be able to see what medical data I have so the benefits is that patients can choose who can access their confidential confidential information they're assured about their privacy and doctors can take the right decisions. Disease protection chatbot is the disease protection chatbot so what we found is on an individual level even in urban areas you want immediate you know chatbot detection based on the chatbot thing so for example if you're suffering from something and you want an immediate consultation so this not only in semi urban or rural even in urban areas what you can do is we have a chatbot where so you first fill in your details your name age and whatever metadata is required then you start you know you start interacting with the with the Informatica API what basically it does is based on your given information it tries to detect the symptoms and so basically what happens is so first when there's no the detection has not been done every API call there's a follow-up question so if there's a pattern that you know this can result to something else there will be a follow-up question you know given to you and once the entire diagnostics is done it is sent to the admin who can then look at your chat history see what your symptoms are and you know then look at things like whether it is right to you know deliver the medicines to you or not so there are now a lot of a lot of companies which does online delivery of medicines the regulation says that there should be a prescription but in a lot of cases you don't actually have a consultation already done but people still get it so what this does is act as a layer in between that you interact with the API have the chat history and then the admin decides whether it's the you know it's for the rightful reason or not so the third module of this is the disease detection model based on deep learning or computer vision this is basically targeted for the community hospitals so a lot of these hospitals have these modalities the city scanner and the MRI scanner all these things are there but the you know the the specialist has passed there's like one doctor to ten you know one billion in a lot of cases there are cases like you know neurology where there's one doctor to ten you know perhaps millions of patient in countries like India so what this does is so you already have the infrastructure to take the scans with different modalities so if you are a community hospital what it does is there's a new patient you decide which model you want to test it you have your dichom you know taken from the the modalities you send it you get the you know inference done and you have the result so you have the you know initial diagnostics ready for you at least the initial screening is done so for our prototype we did it with the stroke detection model where we you know did it just detected strokes based on deep learning models another module is again delivery by drone so once it is detected so in case of individuals once you interact with the chatbot you have your like medicines approved or not in case of community hospitals after having a look at the you know inferred results from the model you can you know you can request for some specific medical aid to be delivered in both these cases you can use the drones to use that geolocation and can get delivered based on you know which from your pharmacy partners or your blood or blood or organ or whichever partner is there based on everyone is having access level and it can be delivered yes so one very important thing especially in case of you know individuals is you should make you need to make sure that the delivery is being done to the right person so for that matter maybe I have an account and you know I interacted with my chatbot but then at the time of delivery you never know which person is it at the same time we'll have to do it on the fly and like literally on the fly your drone should detect it on the edge it's very difficult for it to for you to you know keep it in continuous interaction with the cloud to keep sending the HD frames and then getting the inference back so what we have done is we've used neural compute stick basically it's like fanless very tiny device but after I'll talk about it in the lightning talk on 17th in much more details basically what it does is you can just you know offload your models on that on a device and it will run the model on you know on the edge in a very low cost and it requires very less power so even with something like a raspi it gives me the power to at least infer or deploy my deep learning models so that you know I can perform the inference on the edge another thing that is very important is when we are having models like these it's not possible to have all the faces in the world and then train it for example if one of you is my new customer I can't train my model again to add you as a class there so for that what we have done is we have used face net basically what it does is uh only only the vector embeddings for your face the new face is saved and we can save it as an umpire array and while time at the time of inferencing I'm just comparing your vector and you know embeddings with the live camera frames and if it matches I know that person is you and it's this requires me you know not to train my model for every new class that is being added and once I find that you know it's the right person and it's the authorized person I can allow the more the drone to either you know deliver you or give you the medicines or in case of community hospitals again the authorized persons can only get the medical aids delivered yeah so um just these were some of the technologies that we have used to what we thought was the main areas that require some little bit of technological intervention so I just wanted to you know tie everything together please forgive my bad use case diagram I did not pay attention during my software engineering lecture so yeah there are mainly uh three different people who can communicate with the complete integrated system one is an individual and when we say individual we mean a person in an urban area or a rural area if there is enough internet connectivity uh which when you register for the service as an individual what you can do is you can access the dashboard you can register as a new user and then as an individual what you can do is you can um access the chatbot the disease detection chatbot that you say okay today maybe I have a little bit of fever or sorry or you know I'm not feeling too good whatever your symptoms are and uh your symptom your final you know so if it says oh you have a fever have you had a fever for the past five days or did you break a bone recently uh so it gives that sort of follow-up questions that finally gives that you know maybe this person has ABC disease and when you get that maybe you have ABC disease your admin gets a notification that okay this person needs medical attention they might have ABC disease do you want to approve the medicines for this particular person and the admin can approve or you know reject your medicines then we have a module for community hospital and the community hospital can do a lot of things the community hospital you can say okay we are in a semi rural or rural area and we have the need for maybe 500 tablets of paracetamol and can you get this to us without us having to travel some 50 hundred kilometers and then the admin gets that request with your particular geolocation and the admin can say okay yeah we can give you that again uh yeah so the community hospital also has uh under an umbrella of the you know of the I mean specialization or disease detection where it can upload CT scans or DICOM images and it can you can find out which disease a person suffers from using the CT scans so currently we have only one module that we deployed which was for stroke detection is chemist stroke detection and a person can save the CT scan results what the community hospital can also do is approach patient information to the blockchain and as an admin you can have a complete log logging information about what individuals have ordered which type of medicines what community hospitals have asked for and you know help in any case um this was our system architecture if anybody is interested there are a lot of specifications that are that we use that is not necessarily needed to be used we try to keep this presentation as impartial to the technologies that we've used as possible because we just wanted to say that okay this is what we can do this is how we can look at uh as paramedical responses what we thought were the gaps that were there how we thought we could fill it and um yeah I think we have some time for questions and if you want more information you can always refer to this link it has the paper that was published in springer um if you need any more information that probably dwell delves a little deeper we just wanted to touch upon some things yeah thank you very much first maybe an applause for this nice presentation and indeed we have time for questions somebody yeah we have developed a prototype of this and we actually had a video but in the interest of that it's a bit long so if you would like to see it later probably we can connect I'm just interested on the tail end the last mile where for pharmaceutical delivery yeah did you all really test the drone delivery it's on a prototype level we have not deployed it yet the regulations again you know won't let us you know tested that much but prototype wise in a restricted environment definitely yes so yeah thank you do you have any source code public source code available so we are planning to so perhaps in a week's time the deployment models and all those things we're just cleaning it up it was like it's quite messy we were just trying out a lot of things but yeah we'll publish a lot of these modules on get up so informatica is open source the models that we're using tensorflow that's open source so all of this is open source we just wanted to give a little bit of an overview how you can tie all of these together but yeah probably in a week we'll have a little bit more where can be yeah I have we have the get up account with our same name so like for me the name unfortunately so unique it will be very easy to find for her it's a full name so yeah so it's mushroom for me and so I'm not so enough for her on get up so we'll have a perhaps posted on both maybe uh step to step by step of you know which open source tools and libraries that we used how we you know put it together and even whatever components we developed most of it you know that will will be you know pushing it you can find also the contact information on the schedule actually from false asia there's the github and everything available thank you so much thanks for the time yeah thank you again