 Yeah, that's that's great. That's great. So now we will have our next speaker. So we have Andrews So I hope I pronounced correctly Andrews, is that how I pronounce your name or did I put your name? Sorry about that Andreas, yes. Okay. So Yeah, I see that you're going to talk about hot beach So it's it's gonna be something different, right? It's like it'll be very interesting like, you know Python usually people won't release that to a human body. So I think that'll be a very interesting talk. So Yes, that's the slice there and I'll let you take us away then Okay My name is Andrea slow salmon and today I'm going to present to you a topic that is very close to my heart and probably yours I graduated from veterinary school in 2018 so I am a licensed veterinarian in Germany, but I have mostly worked as a full-stack developer currently I am employed at the company doing full-stack work with mostly react and dot-net and My talk is going to be about hot beads and how to detect them Your heart is the organ that is pumping blood through your body and it does so at a certain rate and There are quite a few ways to observe what the heart is doing how well it is doing it and even to detect potential problems this talk is going to cover some basics about the physiology different kinds of heart beat sensors and especially electrocardiography or ECG Hey Andrea, sorry to interrupt, but I think that your slice is not really working for the stream. So maybe we have to Do you mind re-sharing again? So maybe hopefully we will fix that. I think it's I Can go I didn't oh now. Oh, okay. Now. It's okay. Sorry about that. I thought it's not working. That's fine. Okay, so Yeah and Yeah, this talk is going to be about different kinds of heart sensors about physiology and I am going to show you how to build a relatively simple Arduino based ECG device And then I'm going to introduce you a bit to ECG analysis with a bit of data science and machine on it So observing the heart with computers There are several basic modalities to observe and we may an art with a computer One would be acoustic as phonocardiography Which is how the smart people call it and This is basically like a stethoscope or you put your user Microphone and you're mostly interested in lower frequencies There are even digital status copes out there right now, but they aren't very Common because they are probably expensive and if you carry them around a lot then they break Anyway Then there are mechanical sensors, which is called seismocardiography and That is relatively recent because they now have these little M E M S Selerometers that you could also attach to attach to an Arduino or something and you can attach the sensor to the clavicle and then measure the Waves there's the information You can measure vibrations coming from the heart and these this information would be similar a bit to Phonocardiography, but even lower frequencies and it could also be quite useful Optical sensors Many of you have seen those often they get attached to the finger and They shine a light through the tissue and by reflecting or transmission by measuring the reflection or transmission of that light you can tell the heartbeat Because the blood is pumping through the tissue and then it changes its transmission or reflection or whatever and there's also a need to trick to tell how oxygenated the blood is and That's not going to be part of this talk the electrocardiographic Modality is probably the walk-offs of cardiology or has been for a long time and That is about measuring electrical potential difference differences, which I'm going to talk more about later then there's ultrasound which has somewhat replaced ECG and some applications and it is a very real time way of looking at the heart and You can tell a lot about both functioning and so on Then they are lately have been more and more research into using MI and CT But that of course is out of the range of any normal hacker This is from Wikipedia a Video of ultrasound recording in 4d 4d because it's a 3d presentation and the fourth dimension is time and you can actually watch in real time how All the moving parts are moving and you can even tell when something is flowing where it shouldn't and that's why this has been very Very much used for everyday cardiology Also, these devices can cost cost a few hundred thousand dollars right now the most advanced Devices which is more than an entry level computer tomograph or something Now there's also real-time MRI which is also relatively recent and you can Give even more information or get more information because it's magnetic resonance imaging and it looks rather cool Now I'm going to tell you a bit about how this actually works So we need to understand that the heart is Pump and the mammalian heart has four chambers to atria and two ventricles the atria are small the smaller ones and ventricles are the bigger ones and Left ventricle is pumping blood through the main body circuit and the right ventricle is pumping back through the lung circuit Vane's feet say atria and Coming from the body circuit for the right atrium and the lung circuit for the left atrium The atrium is a kind of a priming pump So before the larger ventricles contract the small ventricles pump the blood into the bigger ventricles and for optimal function and these smaller atria have to Contract first followed by a contraction of the ventricles and several things can go wrong with all of that and All of these problems Could lead to reduce cardiac output a little bit about heart cells physiology, how would that all of that work on maybe also an electrical level the heart muscle cells are very specialized kind of muscle cell they Know how to contract a when to contract mostly because the neighbors are Contracted which they sense by an electrical impulse Each cell forwards its excitation to the next so this actually works a lot like laula wave Well, like in the stadium where somebody stands up then the next man stands up and so on and it's much of the same or the same principle in a In the heart excitation you basically basically have a traveling wave of heart cells getting excited and when they are When they just contracted they go into a sort of a dormant phase where they came for a few Well milliseconds for a certain period they can't Get excited again and When we are doing electrocardiography We are listening to the waves of electrical potential to generate it by the excitation of these cells Now what is ACG anyway? As I said There's this traveling wave of excitation and we need to We need to measure the electrical potential difference between two points Mostly or most done at least that would be the left arm and the right arm And then we have a reference electrode somewhere else and There are standardized ways of doing this which I'm not going into right now Yeah, so actually measure this you would need in differential amplifier and I actually don't understand this diagram I'm not an electrical engineer and anyway, this is This would be from Wikipedia and shows how differential amplifier might work and Components and these components are relatively tricky to set up for for a beginner electronics hobbyist and That's then that's a good thing that we have these 80 H to 33 88,000 88,000 232 ECG sensor boards This is a small little chip on this breakout board with a lot of small and tiny components in between The good thing about this board is it just takes one power supply of Ground and 3.3 volts and then it gives you an output an analog voltage signal From zero to 3.3 volts It also has lead-off detection with the LO plus and minus pins, which I'm not going to go into and the SDN pin will Is this is a basic heartbeat detector? Which you could also listen to and then The This heart-shaped outline surrounds an energy that is also beating through your heart if it works The headphone jack is for electrode connection the some of these boards come with an electrode cable or three electrode cables that Headphone jack and or headphone Connector and then you can connect these relatively simply but People say that this doesn't work that way and I also prefer to solder electrodes to the To the three connections at the Top edge here and they also never labeled for right arm left arm and Right foot I think Right leg Now if you have amplified the signal now, you need to read it You need to convert the analog signal to something digital which is a job of analog digital converters and Arduino boards often have an ADC built-in, but it also has problems and these I squared C devices can convert Analog signals at 16 bit versus 12 bit and 860 samples versus 3000 Samples I went with the right one because I wanted to have 1000 sounds per second another Secret about these boards at least The cheaper ones is that if you feed them 3.3 volts instead of 5 volts and They are slower And you just need to know that If you don't know I squared C, it's basically and some people will hate this energy I squared C is a bit like USB for microcontrollers. It's It's a bus for data connection or data transference between the microprocessor and some Periphery devices and it only needs two connections to the pins, which is also quite nice Then I chose the ESP 32 as as a controller board for this For this project and I chose one with an LED and a battery You can put an lithium-ion battery in there, which is good because when you deal with high amplifications and signals and the power line can Inject transference even over a USB Connection and this makes it a bit easier This these particular devices want to be called We most low in 32 That's not really documented anywhere, but if you call them anything else and the IDE won't upload to to this board All is be 32 dual course with Have to they have two CPUs and They are running at 240 megahertz. That is quite nice because they can they have quite a bit of power They can also be programmed with micro-python or cycle-circuit Python. I Decided to use C++ with the Arduino framework Because I needed better Better scheduling and real-time performance For what I'm doing And as you know when C++ is your hammer everything else starts to look like a thumb That was from the pie jobs collection by the way So the general plan is collect samples from I squared C ADC Drive that loop through a timer interrupt Put samples into a buffer That regular intervals send the field buffer off by Wi-Fi I'm using TCP RP now, but I could also use UDP which has advantages and disadvantages and That would I would like to Make a joke about UDP right now, but you probably wouldn't get it now The big biggest problem here is that sending takes at least four milliseconds and if you do it naively then You have a gap of four milliseconds in your one millisecond spaced samples So what you need to do is multitask in with these two cores on the ESP32 the ESP32 can run free IPHOS and that is a real-time operating system The correct way to implement it and I have discovered a few not so correct ways Is to do the sample collection on core one So there I filled the buffer with the samples I collect Then I Then I need another task on core zero which sends the buffers at regular intervals Now what you need to know about TCP IP and Wi-Fi is They need a lot of housekeeping and the microprocessor has to spend some attention on that So your task must not block core zero at As all too long or otherwise nothing happens with your Wi-Fi You have to call Vtask delay and Vtask delay until and if you've ever worked with SMH IO It's basically the same Same principle your task monopolizes its core or threat and Nothing else can happen until you yield Yeah, I am going to try and Switch to my live demonstration Now the problem is it doesn't really work completely. I can I have gotten ECG signal which is set up before but Unfortunately, the connections are not very reliable and I might have some Ideas how to improve that in the future, but right now it doesn't work, but basically it's It sends samples at 1000 frames or 1000 samples at a second and I have the first fast API server running Collecting the samples or receiving them and then putting them out to this okay react app I wrote integration of okay with the X that works over web circuits Here we get some signals, but that's almost Yeah, I've gotten signals that actually look like ECG, but this is really not it Better stops the presentation other you more right there But that's what what you get when you try Light things phone button now we're back to the presentation and Let's do some data science. I promised data science. You can go to Physio net org and download data Did most of the records in BFTB format most of the records are in BFTB like WFTB format and There are several databases collections of samples sometimes they are labeled with all sorts of things and That's a few years ago part of physiology also There are not just ECG Recordings also Of course, there's a Python that your module for that Going to go into that much you can of course plot the data This is ECG data most of the time the recordings are with two leads which means two dimensional signal and Yeah, that's Just about all there is to it What you can also do is to conclude 2D histograms I using data data shader Here from pie data and this is just a 2D histogram of the two lines of the two leads moving together and then you get a picture of how we How the heart all the electrical processes are traveling or how the Traveling wave is moving because Basically with two Two leads you can Intersect the three-dimensional movement of this traveling wave and This shows you how Predictable the heart is beating or how predictable And the electrical connections are and it can look a bit more strange like these recordings from the MIT Bih arrhythmia database and When you see multiple loops, then you cannot already tell that there are hard actions of different shapes and Also in the middle, there's a lot of uncertainty and probably fibrillation going on in the HVA and on the right hand side, there are multiple forms of ECG actions but which are sometimes but Only the one in the middle going downwards is probably the prominent one and the other two are more rare Yeah, it's a nice way of visualizing those Another way is you can use data shader to To visualize one D Only one beat and the way you do it is you first have to detect or know where these Where the big spike is which is the arse pipe of the QIS complex And I'm going not going to go too much into it That maybe for a later talk how to analyze is more but this is Just Basically, you need to know where the arse bike is and then you lay you put all the hard actions over each other and Computer histograms for example data shader and then you get this picture What's remarkable is that this electrical action is not very random but relatively deterministic It's often so that when you have Disease conditions then it would show up as more random We would see more chaos in there or more other thicker lines But when the lines are relatively Thin, then you know that this is a very predictable process and Now I promised you some machine learning also and My main project in machine learning with ECG currently is trying to detect beats normally There are a lot of the beats Detection algorithms most of them relatively simple some more complicated and You might think you could just look for the big spike and be done but unfortunately, not all of not all ECG signals are this nice and So you have to calibrate stuff and Sheen hyperparameters and so on and I tried to use I'm trying to use convolutional neural network with Keras and to Make a more robust beach detector that works with almost anything and In this picture, I have Blue line, which is the original ECG data and the blue dots are the manually manually marked Beach markings manually edited from the database and The orange line is the output of the neural network which says with what confidence there is Heartbeat and this works relatively nicely. This Is a slightly more difficult ECG diagram to analyze for a computer because there is this in the middle base a jump in the voltage level and The less sophisticated I wasn't might just say there's another beat because we are having a spike but at least this algorithm doesn't do that and Can go even crazier like this we have multi-form Heart actions The shapes are different from different heart actions from one heart actually to another and on the right side we have probably motion artifacts and smaller Qis heartbeat looking things and the algorithm still picks that up and This congruent receive with with the manual demarcation and Then I also tried it across the species and this is the buff diagram it's about dogs and The is the one below is for the mouse and They are also from fission edge. They have a zoo database with a few species as we see the algorithm still detects the beats and But It's not that difficult of a task because all the hearts of the mammalian genus or all of the hearts of the mammalian mammalian Species will basically produce the same ECG just different Difference of size if you have a mouse that Little animal like a mouse will mostly have a higher heart rate and This these heart action shapes will be smaller and By just scaling the signal a little the neural network Detects everything just fine now If I ever give another talk like this again, for example heart beats for heckas, too What might be coming next is for example a more reliable hardware Because hardware is Hardware is hard and biology is even harder I Intend to use the only only D screen similarly or simultaneously to the measurements the problem is currently that They both want to be on an ice-cream at sea bus and I still have trouble Figuring that out. I might have a way to make this work But well, yeah, I'd also like to try out mechanical and optical sensors Which would be that we would add another dimension to the data And it should also be relatively easy it would also be nice to show the CNN V detection in real time, which is possible because it's quite fast and the neural network isn't that complicated and I Probably would also tell more about easy gene analysis now that was all for now and I'd like to thank you for listening and session chair for introducing me and Yeah, well, then we may have time for a few questions Yeah, so that's great. That's really good and it's very interesting as well. So we do have one question So let me put that on screen Have you played around with the cardio? Bandage, actually, I know nothing about that but from the CCs come 2019 by any chance There was some interesting easy to experiment with that one as well. Have you heard about that or no? No Yeah, so Yeah But yeah, like all I know about easy G because like I know that I well I'm a Apple fan So I use the Apple watch and then so there's this easy G function that you can measure your heartbeat and stuff Right, so that's the only thing I know Do you think it is accurate? And if I measure it, can I use my data to do some analysis? Yeah, probably I Don't know if you can how you can access it, but I think you can just Access the data. It might even be the same shit like I showed you on this breakout board something similar and I mean you there's all this there's all this regulation about what an easy G should be or but but really I mean as long as you have valid easy G data or you see the shapes are coming out Then it should be usable for some kind of Information in clinical information in some part, but you would have to be careful with that always Yeah, so I think I've also put a disclaimer that like, you know, don't rely on it a hundred percent because you know They don't want to be have responsibility if you know, you have problems and then later, you know, sue them or something So that's another question if I got it, right She used a C++ on the ESP 32 then like with this have been possible with Python In terms of performance, I haven't tried But I haven't I hadn't tried I don't think it is that possible I'm not sure because the basic thing is I want to have one thousand samples a second and Do some networking in between with the other core and that It's important to get one sample every millisecond at the right at the right or Consistent interval and that's harder when you do garbage collection and whatever else the Python interpreter does So I haven't tried at a lower resolution you might Be fine, and it's usually most tutorials and YouTube videos about ESP 32 and Measuring stuff IOT wise Use much lower sample frequencies. And so I had to more or less find figure out myself How to actually do this in a real time way Yeah, I think it's amazing that like you can do so much stuff with open source tools and Python But do you think it's a risk if like people can just do like, you know, analyze their own medical data That you know with a with a small gadget like Apple watch or anything then you can collect medical data From yourself or from a family then people can analyze it themselves And do you think it's a danger that you know people rely on it too much and didn't get them, you know Professional help or things like that. What what's your thought about that? It's extremely difficult right now to do that And at all and I mean the apps will have their limitations always but There's Well, I'd say the applications will always tell you stuff or information about ECG maybe and they won't do the diagnosis for you And you would have to put the diagnosis Diagnosis into context of your other health parameters and then do something about it. So for basic basic prescreening it would maybe enough With artificial intelligence and so on because you don't Expect a very hard a very high rate of disease in pre-screening or screening people and I Don't I don't see a lot of danger You really to do ECG analysis You have to know a lot and then you sort of know what you know and what you don't know and Of course, you can't use anything what I told you for Personal medical advice or something, but I think that's pretty clear that This doesn't Replace any doctors Yeah, that's absolutely true. So I think yeah, of course if you know people if you have you know Suspect that you have health problem, please get you know professional help I mean you could get some you know have some fun project like this You know, but you know don't rely on the small gadgets because you know, I think all of them would tell you that Don't rely on them Yeah, thank you so much Andrea's and this is really really interesting. I think I know that all the other pictures and demos are great So I think now we are going to a coffee break