 Good afternoon. My name is Wendy Sang, and I'm pleased to present this talk on artificial intelligence for automation of chamber quantification and other common measurements. Here are my disclosures. The objectives of this talk are first to describe the use of artificial intelligence in chamber quantification. I'll then review the potential benefits and pitfalls of using these algorithms. I'll also discuss the limitations of these algorithms, and I'll end this talk by providing an approach to assess these algorithms. Now, why do we need artificial intelligence? Well, for many reasons. First, the number of echocardiographic studies performed worldwide is increasing. Here are the numbers for Ontario. As you can see in 2001, about 350,000 studies were being done per year. In 2018, almost a million studies are being done, and despite this, the number of people who read these studies has actually not increased significantly. But in addition to this, each of the studies that are being performed is actually longer. We take more images these days. We also perform more quantification. We do biplane EFs. We do strain analysis. We actually quantify valvular heart disease, yet the time we have to interpret these studies has not increased. We know that there's significant variability in our measurements when we do more quantification. This is a nice study looking at the measurements provided by individual labs of LVEF for clinical trial. And when you look at the LVEF measured by the individual labs against the core lab LVEF measured off the studies, you can see that there's significant differences. And this is due to variabilities in image quality between patients. It's also operated by both the stenographer acquiring the study as well as the interpreter and depends also beat-to-beat depending on which heartbeat you're measuring. When we interpret studies, we usually have a series of images that we've taken in a particular order. And then we have to integrate different information from different parts of the images and different images to come up with what LVEF is or what RV function is or how severe the mitregratiation is. And this can lead to problems with reporting. So this is a nice study looking at errors and inconsistencies in the reports when and compared it against the number of studies being read. And they found that there was an increase in numbers of errors and inconsistencies in the reports, the more studies you read per hour, and the increase in errors was actually not accounted for by the number of finding codes in the reports, which remain stable despite the reading. They attributed this due to differences between the experience. This is also caused by physician fatigue as they're reading more studies. And also the interruptions that constantly as you're trying to do something that caused impaired attention or memory or excited function that affected the reads. Now, artificial intelligence to interpret studies is not something new. A computer-assisted interpretation has been commonly used for ECG interpretation, but it hasn't been able to apply to images because of the limitations of the computer science. But development of convolutional neural networks has caused an explosion in this area because of the abilities of this. So convolutional neural networks are algorithms that are designed to cluster and classify or model data sort of using the way the brain and a neuron functions. And this is a sub area of machine learning, which is where you develop algorithms with the ability to learn without being explicitly programmed. And this is all within the umbrella of artificial intelligence, which is the sort of field of developing progress with ability to learn and reason like humans. Now, one of the things you have to understand about these algorithms is that they don't give you a yes or no. What they do is they take the input, which would be the images in our case, and they provide an output, which is a probability of what you're looking at. So, for instance, if you are looking at the apical 4-chamber view, it gives you a probability that the image in front of you is a 4-chamber view. You got to remember that when you're interpreting these studies. So one of the first earliest applications of computer vision was to try and see if it could sort simple things. For instance, looking at these blueberry muffins versus a chihuahua or a bagel versus a curled up dog. And you can see that computer groups from a lot of these major companies could do this task fairly easily. And these algorithms have actually advanced such that they can now identify multiple different things within an image. So here we've got cars and animals and people all within the image. And it can actually, algorithms can be developed to actually identify all these little sub parts. And with this increasing sophistication in the algorithms, you can see the machine learning and cardiac imaging has really taken off over the last 10 years, especially in echocardiography. Now, what is artificial intelligence in imaging used for? Well, there are four main applications. We're going to focus on quantification for image sorting and measurements of the images, but it can also be used for as identifying red flags or notification tools and for diagnostics in terms of determining what to do with a patient or even diagnosing a patient and predicting therapies. So one of the largest and more most impactful studies was published in 2018 by Zang et al. And what they did was take almost, they had a database of almost 20,000 studies and what they did was first try to see if they could identify four chamber views, three chamber views from these studies. If you look at the middle in this color image, you can see that they asked the computer algorithm first if it could actually cluster things, because if you can cluster studies, it becomes actually very difficult. If there are features that the computer algorithm could identify that would allow it to cluster the images. And you can see here that the algorithm actually did a good job. So these red ones are sort of the parasitonal longest views here. You've got here, you've got a apical two chamber views and the apical three chamber views. The blues are the apical four chamber views and the different clusters are sort of apical four chambers reviews where you can see all four chambers versus you can see only the atrial versus only the ventricles and similarly for these other clusters. So once you know that you can actually cluster your data, then you know you can have the algorithm can develop a way where it can identify these and then you'll see this a lot in these paper, these confusion matrices. So the ground truth is what the gold standard is or what the real the image really shows and this is usually labeled by manually by someone and then the prediction is what the computer thinks it is when it sees the image. So things along the diagonal says that the both the the computer prediction is the same as the ground truth. Anything off the diagonal means that there's been some sort of misclassification. One, these scientists actually put these echocardiographic studies and saw if it could identify the different echocardiographic images, the two chamber, four chamber, three chamber views, etc. They were able to show that they could do an 80, they were 84 correct for specific views and 96 percent for broad group views, which is very good. They also tried to see if they could segment the images, which means if they could find the blood pool versus the walls and then the ventricles in the chambers for the two, three and four chamber views. And here's the sort of an example showing the actual image and then what someone manually contoured and then what the actual CNN or convolutional neural network was able to and you can see that they're very close and when you look at the accuracy for example, the two chamber view for the left atrium versus the left ventricle versus the myocardium, you see the percentages for accuracy were actually quite good. Now convolutional neural networks has also been used to look at Doppler images, PW and CW images in addition to just the B mode images. And you see the accuracy of doing this has been good high at 94.4 percent and the time to do this is actually quite short. It only takes 0.87 seconds per study for the algorithm to identify all the images. Now, once you can identify the images, you could actually pick out things that you want to group together so you could read in stacks. And so this is a data showing that that computer's algorithms can not only identify the images, but they can also stack them so that it could be useful for rate. For reading here, you can create an LV stack and LA stack, or you can even just pull out all the images related to mitral valve and you see that the complete stacks are that the algorithms can do this very well once they know what they're looking for. Beyond identification of the images, you can actually get measurements. If you can identify the image and then you can actually segment the image, then you can actually automate your LVFs and your basal LV dimensions here. So this is a really early study looking at a biplane LV volumes and ejection fraction. And you see the correlation between the manual EFs and the contour. The algorithms were actually pretty good and the bias and limits of agreements were small. And these algorithms were very fast. There were eight seconds and so these algorithms were very highly feasible. And now there have been some preliminary studies looking at LV basal dimensions, also ejection fraction by other groups, and these have also shown that compared to manual measurements, the machine actually is not very bad. Automated algorithms are also being applied to LV strain analysis and the study you can see that the differences between the algorithm and the manual assessments were actually very good for LV strain. And that is one of the things that automated analysis has would really contribute to is reducing the variability. This is data from the heart model study showing that when you have an automated algorithm compared to core lab measurements, you can see there's still some variability and that's completely eliminated if you just use the algorithm completely. And this is one of the important things that when we look at people who cryocereal studies such as those undergoing chemotherapy, if they had their echocardiogram performed in a lab in California and then the next month they have to they're traveling and they want to get it done in New York, you see that if there's any difference it would actually be attributed to a true change in their cardiac function rather than because differences between the labs or the readers of that day or the images or the sonographer that was doing those pictures. The other thing automated analysis, this is very time-saving, you're not spending all that time analyzing and manually comparing things. This is a nice study showing data from using GLS and EF on serial studies and actually showing that you can then track patients to see what's going on with their LVF as a strain. Now beyond chamber quantification, these machine learning algorithms can be used to improve our diagnosis here. They're looking at athlete's heart versus hypertrophic cardiomopathy and the machine learning model can take many different features and be able to provide a prediction of whether or not an image is demonstrating hypertrophic cardiomopathy or athlete's heart that's also been used to diagnose cardiac amyloid as well as diastolic function. Other applications for machine learning include valvular heart disease. They're using artificial intelligence to improve tracking and creating these models so then there's less work to actually create them. They're also being used for automated quantification. Finally, artificial intelligence is being applied to valvular heart disease to improve first identify the valves but also the severity of regurgitation. However this work is still very early on. Now what are some of the limitations to using artificial intelligence on echocardiographic images? Well first image quality will really affect the algorithms and so diagnostic algorithms to be used in clinical practice will have to meet rigorous standards proving utility cost effectiveness as well as comparison versus a gold standard. Now when you're assessing an artificial intelligence algorithm whether or not you're using for image analysis or phenotyping this is sort of an approach you've got to think about what data set was used to create the algorithm because if you have a group of patients with only aortic stenosis and nothing else then it becomes a very biased data set. You want a data set of patients with and without aortic stenosis in order to know how good your algorithm is to identify aortic stenosis. You want to actually see what's within the data sets. You have to assess what it is and look at who's missing from the data set. And then you want to see how they develop the models. What kind of what kind of image preprocessing did they use? What kind of approach did they take? What kind of hours did how much how did they manipulate the images? And then finally how did they validate their study? Did they prospectively trial it? Did they have data from a different center that they applied it to? And that's sort of what you want to know. So in summary machine learning for automated reporting is still very early in development. Automated measurements analysis is being integrated into current programs right now and it will improve accuracy and reproducibility and save time for readers. It will ultimately change the workflow because then you will have a machine perhaps tell you how to acquire the images and then analyzing it for you, providing a report and it will change our workflow. It will also alter how research and clinical trials will be done and it may lead to patient specific improvements in tracking and interpretation and care. Thank you for your time.