 So this talk is going to be about telehealth platform. It's powered by Python and Django. It's a social network for doctors. Yeah, that's pretty cool. Yeah, so I can see the slides now. Remember, for everyone that is watching this talk, there is a Q&A button. You can click there and ask questions. And then at the end of the talk, I'm going to ask those questions to the speaker. So, okay, I think you're ready to go. Take over. Good luck. Thank you very much. Hi, today I'm going to talk about telehealth solution. And it's based on a polyglot microservice architecture. We use Python and Django. So it's a Python and Django platform. So what we have is a social network for doctors in the community. It's almost like a Facebook for doctors, where doctors can talk to each other. And also doctors can extend telehealth solution, right, as a medical practice for their clinics or polyclinics through different patients. And especially in COVID scenarios, right, and this telehealth solution has become very popular, as you all know, and it's catching up in the telehealth solution is being used in terms of not just for review consultation, but also for end-to-end, right? After the patient gets the prescription from the doctor, the patient takes the prescription, buys the medicines, right? Then the lab test needs to be done. It's going to go there, get the lab test done. And also the lab test gets integrated into the system back. So when I say end-to-end, all the healthcare ecosystem players are being integrated in a particular solution. So that's the beauty of telehealth. It's become really popular. And we also see why it's playing a big role in terms of voice assistants, digital assistants helping out. And also in terms of helping out the doctors, you are having a clinical assistant. So today we're going to look at what is a typical patient journey? And what do patients want, especially in this kind of scenario? And how does telehealth solution really help? What's the telehealth platform? How does it look like? And how does semantic analysis help? We talk about different patterns, voice assistants, they can speech processing, speaker recognition, speech types, speaker model, variations, speech engine, speech analytics, digital assistant. How does it help in taking modes? And a little bit on the technical architecture from looking at the solution. And what are the challenges that you are faced in a small demo I have? And what's next? How do you go about enhancing the telehealth platform? And what's coming up in the technology and healthcare? So let's look at the patient journey. The patient typically is having a fever, cold, cough, a specific problem. He wants to book an appointment with the doctor. He books the appointment and there's an appointment reminder which is created. And we can have schedule being tracked with both the patients and doctors. And if there are any delays coming in from the doctor friend, you have the delays being intimated. If there are any changes, you go about managing the appointment, rescheduling the appointment. And also after the appointment, there are any follow-ups required. If you have close appointment. So a very simple patient journey. I think everybody is familiar with this. Many of the times people would have gone to the hospital and seen at the help desk, receptionist and you would talk to them regarding appointment. So you envision yourself going into a hospital. The same journey you want to put that in an intelligent solution on the cloud, on your mobile application. And instead of receptionist, you take the same process and have a digital assistant, a voice assistant help out in this particular process. Now what do patients want? Let's take a step back from the technology. Typically patients want a doctor to know him or her. And they want the doctor to be competent. They want doctors to be caring about them and improving and curing and preventing their particular problems. It can be your daughter, it can be your wife or your son. This is what a patient typically wants from the doctor. So how does the telehealth solution help? So from a technology point of view, we have a WebRPC based solution. And there's a deep learning based voice and also a little bit on historical data, patient's data and predicting identifying patterns. And we use neural network algorithms. And if you look at the solution, a lot of things which are happening, but there is a social community for doctors where we do take the feeds from internet data source and do the fact check, right? And have a CMS like WordPress and then publish blogs, publish feeds through mobile apps and web applications. And you have recommendation engine cases and knowledge engine insights and also a wallet and a payment gateway. Just in case you want to have what you call payments to process for the appointments, doctor appointments. And also you have transaction engine, loyalty management, curator. We're also enhancing for supporting IoT devices that's on the radio which is evolving. The platform has support for voice, video, SMS, also calls, service, and also insights. So if you look, look at the previous slide and current slide and basically on the left hand, right here, right? You have the historical data, the patient's data, the prediction, and different patterns which really helps for doctor to know the patients. And especially the voice assistance conversing with the user, right? Showing care, showing empathy, showing emotions, showing the symptoms, right? That really helps in terms of substituting a digital assistant to a nurse or a doctor, right? So how does the telehealth platform look like, right? In terms of different components of different persons getting involved? You have users, you have groups, you have content, right? And we look at audio and video chats which has an important component for telehealth platform. But you have notifications, you have discussions between the doctors, you have calls, you have social interactions, you have service, you have wallets, and also you have courses, events, news, career, written for the doctors and medical students. And you bring in employees, nurses and operational personnel, partners, vendors, and also customers who are the users. And there are groups created within doctors where you can have moderation with them, or you can have group discussions, or you can have case triage where two or three doctors discuss about the case. And you also can publish products, have product reviews, take it from doctors and healthcare personnel. And there is content related to feeds, news, recommendations and there are blogs and also services which are published through the content channel. The semantic analysis, like you were talking about where NLP plays a very important role, was one of the reasons why I had this session. Before we get into the NLP, the idea is to have understanding for the critical relationships between the words and phrases. So natural language processing which fits upon this. You have blogs, news, feeds. So there's a lot of text mining and text processing which happens behind, especially in the voice assistant or digital assistant or digital notes or prescriptions. And if you look at them, just giving a very simple example. John hit the ball. You have subject, verb, modifier in an object and you are trying to identify what are the entities here? What are the verbs here? What are the nouns here? Simple noun verb analysis helps. Or you have, you do entity extraction or parts of speech extraction and you try to identify the nouns and also sentiments. Is he talking about the ball being hit really hard or let's say John got hurt or the injury bad. So if you look at these particular objectives, you get the positive or negative sentiments. Typically you look at words like good, great, fantastic, excellent, friendly, awesome, spectacular, sometimes negative. You got hit badly. It was the worst shot. There was an issue here. There was something lost and awful, problem, bogus. These are negative sentiments. Now, like semantic analysis, the other key section is patterns. We identify patterns in historical data. So what has happened before in a patient's life? What have seen before? We also look at demographic. We also look at appearance, sounds, physical signs, smell, touch, feel, images, fluids, tissues. All these are different areas where patterns do the analysis and they're also mining, data mining which happens in the nodes or in the prescription or in the speech. When the patient is talking over to the doctor, there is a lot of discussion which happens and the patient tries to explain his physical condition or this condition and that's where all the patterns analysis really helps. Now we jump into voice systems and we have understood the semantic analysis and the patterns. There is a simple voice assistant like let's say it's on the Google Dialogflow and you want to look at the doctor looking at the patient's report and he wants to look at the patient. He sends a request and Dialogflow processes that particular simple command, voice command and sends the response back and shows the generator report saying that these are the list of patients. Similarly, a patient talking to a voice assistant is slightly complex and wants to know about COVID-19. So the Google Dialogflow provides capabilities like keywords like keyword search for example COVID or how does COVID affect the pets is having pets a bad situation during COVID or do you have vaccines for COVID? Do you have any medicines already identified? How do you go about preventing COVID? All these are simple voice commands like a keyword being sent as a request and Google Dialogflow provides to process them and sends the response back. Now how does the whole speech processing work? Let's take a step back from AI and look at the speech processing you look at speech coding so when the speech coding involves synthesis and recognition you also have a speaker model the speaker model consists of speaker identity and speech transcription the speech transcription basically is converting speech to text speaker identity is basically the speaker profile who is the speaker, where is he from which language he is talking what is his accent these are all demographics and these are all play a very important role in terms of creating a speaker model similarly speech type speech patterns you have derivation is also very important speech patterns there are many ways of saying the same thing in a hurry we say we show emotions when you are presenting formally the speech pattern might be very different when you are informal it might be completely different from the formal then given these challenges where there is speaker identity accent language and speech patterns how do you go about recognizing the speech there are various approaches you have template based word recognition continuous speech recognition neural network based recognition typically I have a template saying that it is going to be very structured where I have agenda I talk about what when how and last I talk about and then present it is very structured and you know what is going to come then recognition becomes much simpler let's say it is continuous speech like what we are discussing there will be a lot of questions and answers it is continuous the neural network based recognition is basically the next level it is not just template of continuous the neural network has capability of learning so you have feature instruction speech direction vocabulary and knowledge based acoustic model and you have analysis and feature instruction modeling and testing which is done to improve the neural network model what are the speech types in a speech you have isolated words you have connected words you have spontaneous speech let's take a simple sentence John is going to fall so when I say John is going to fall you see there are words which are broken John is going to fall let's take a complex sentence although John was wealthy it was very modest now you see in this particular speech especially after although you see a pause then John was wealthy then you say he was modest so there are connected words there are isolated words broken words and there is a spontaneous speech there are multiple types of speech which needs to be handled so what are the different speaker models we talked about speaker identity now let's look at speaker model where there is independent speaker model especially on zoom we are seeing at the bottom it is able to create subtitles independent of who the speaker is it doesn't know where I am from where I am from and in Euro-Python there will be speakers coming from various countries and it can detect so this model is basically independent it is able to detect the speech and then turn it into subtitles whereas you look at dependent model it is very specific to a particular speaker and that particular speaker notes my identity my background, demographics and there is a neural network engine which keeps on working in enhancing the cookbooks now what is derivation the derivation is if you look at speech being represented on a graphical node you see most of the time you record your speech there will be almost like a waveform where you see a cluster of waves with higher wavelength versus others which are white with a lower wavelength and break you break those down into different segments where there are broken vertical waves or there are pauses take those each segment translate the speech to text you get the transcription done so that is derivation now if you look at speech now you have looked at the various components of the speech if you look at speech engine you have audio and you have what you call the grammars and acoustic model written as test and you have an environment which is around and the channel speaker and you have style gender, age and speech speed and depicting on the demographics speech speed varies some people talk very fast some people talk very slow speech engine needs to have capability to handle both of them how about speech analytics so while doing the speech processing you prefer to look at emotions you prefer you have you look at periods of silence and also non-speech for example I think I am in the middle of the meeting I start thinking I need to start looking at the time also or let me take a step back and think about it and I pause do you see the pause there so even though it is a very simple sentence basically you look at the time and then I am in the middle of this but essentially it looks very fast but in this particular segment you break it down into different segments you see non-speech areas the speech analytics processes what are the periods of silence what are the non-speech areas and what are the emotions detected now let's look at digital assistants having looked at the speech recognition speech processing how does the speech processing really help in digital assistants or voice assistants now let's start looking at digital assistants this is where NLP and semantic analysis plays an important role helps in taking clinical notes retrieving cases processing commands sometimes speech voice commands updating prescriptions creating consultation appointments now let's look at the technical architecture where you have voice assistants digital assistants that's where speech processing speech engine, speech time really help this is a role of the processing Google Dialogflow they are using IBM Watson which is in terms of the speech processing capabilities and then supports them on IBM blooms so from the technical architecture point of view the telecoms platform which we build has Android, iOS mobile clients it also has services layer built on Python, Zango PHP, Java and we have MySQL and MongoDB as the database MySQL is for relational and MongoDB is for NoSQL as a NoSQL database now what are the challenges in building this particular platform especially when you are building voice assistants digital assistants or a video platform you need to look at the browser's capability you need to look at the mobile OS dependency you have desktop OS issues you have speech hardware issues I don't know how many attendees had problems particularly when you get into any conferencing system most of the time you have challenges with the speech hardware it depends on whether you are using Windows 10, Mac OS it's just the incompatibility or the drivers which are challenging sometimes desktop OS let's say connected to this conference through mobile OS mobile OS has challenges in terms of providing audio and video capabilities so now let's look at the demo so I have so here I am signing up so I have the MySQL started I have started the Python Xambo server now I am signing up this is the doctor I am signing up I just put the doctor's name as Dr. John so now I log in after registration so this has feeds now let's look into the voice assistant so I say hi I try to start the recognition process I said hello we started detecting hello now I send the command hello so it comes back with the response now I started looking at corona I detected corona now I am sending that message I got the response so that was about the voice assistant now let's look at the clinical assistant now you are a doctor and you talk to a patient you want to so I have recommended and I am taking the notes now I want to have a video appointment with the doctor so I started the video now let's look at the other side there is a patient who wants to connect with the video system now here I am registering as a patient I go to the register so now I am a patient now patient.gmail.com so I am just simulating both the doctor and the patient for the demo now I log in as patient at gmail.com I go to video over here you see the person logging in and you see the person trying to call the connection gets established after accepting so when I go back over here I go to the doctor I completed the video session and I log out on both sides so that was the demo we looked at a demo which was powered by Python's A platform as my screen in HTML5 and I used the web RTC for the daily help and Google Dialogflow for voice assistants now many people ask in the conferences we talked about voice assistant, digital assistant daily health solution what is next what is next in the technology especially in healthcare or in radio mining with respect to AI so one of the things which are evolving is the biometrics and the other area is the language in another dimension we look at English and we have joining Europe from Dublin, Amsterdam France and many countries can I talk in French can we have doctor and patient relationship doctor being friends and patient being in India can we have multilingual speech and similarly can we have knowledge assistant which is learning has capability of self-learning which is updated itself and also can we have a connected system like we talked about pharmacy and labs diagnostics and pathology and clinics being connected hospitals being connected can we have all systems being connected integration is possible that's the next level now we go to the next level where we want to have dynamic program creation can we have a program being created in AI can we have AI coders can we create a program before normally we have code we used to have code generator now AI coder is intelligent in terms of having a neural network where you have neural sketch for different technology and neural sketch has capability of understanding the technology breaking down into various layers what is required for each and every day what are the domain what is the domain model also what is work created how does the process look like now next level from AI coder is what if I don't want the AI coder I'm a business user I'm aware of the process I'm aware of the domain I'm aware of the process steps in the workflow and you have various what you call users logging in and interacting at the process activity level can I have a no code automation platform where you can drag and drop and create a process and it deploys it on the cloud without any code being created yes and that's the no code automation platform which is up current so that was the brief presentation which is about telehealth colleague or microservices platform so any questions ideas one question regarding mobile app available on play store yes we have the mobile app available it's called white codes we have both what you call network app and also telehealth app for many of the donors we also have a clinic plus mobile app which is available also have a white light white label mobile app for your clinic so we have doctors community around 400k and we have nurses and medical professionals who use our system any other questions regarding the licensing model the doctors app is the network app where we have a social community is there is no licensing it's what you call you're a doctor you validate yourself with your medical certificate finding the payment model for the practice plus friend there are multiple packages which are available if you go for web model or if you go for mobile app model or if you go for white labelled app for your clinic there are various packages which are available online and it's called practice plus suit and you have multiple packages available can you as a patient just log on and see any doctor that needs the platform specific doctor it's a good question so in our platform we have our doctors who refer our patients and the patients will be able to go to a specific app on the play store install the app specific to that particular doctor okay so any other questions I think so people are asking about what about data privacy yeah that's a very good question we look at data privacy very carefully there are a lot of guidelines which are coming from telehealth and we also follow EPOS standards and also in terms of the data exchange we follow HL7 and here the fire FHIR support we have so we take data privacy very seriously especially in terms of the patient's data and doctors information okay so we still have a few minutes I mean how many countries is it used currently we are supporting within India so we have plans to go take it to the other countries so regarding the other countries the challenge is as you know right one is the telehealth guidelines that countries supports so we are looking at various countries within Asia okay perfect so I think there are some more questions so I'm going to say thank you very much for presenting on EuroPython we have to play some so we have some questions for you okay thank you very much that was very good there is a channel in this core so the name of the channel is talk telehealth so if someone wants to continue the discussion you can join that channel there it's in this core you have to go to the Brian breakout and then you have the discord channel for each talk and you can continue the discussion there or if anyone has more questions or they want to understand more about the project okay so I think that's all thank you thanks sir for giving an opportunity to talk about telehealth thank you for presenting speakers are the most important part of a conference so thank you very much have a good day and see you next time thank you