 I was able to evaluate a large reader study and I was able to look at how they scored lesions and how they classified them with bioregids and I was able to do an analysis of false negatives and what caused them and what we might be able to do to minimize them and by false negatives I mean under classification in terms of bioregids category for the percentage of cases that turned out to malignancy so basically because we all perceive that the cost of a false negative is greater than the cost of false positive and we correctly perceive that we've developed an algorithm for interpreting images in which we can achieve 98% or greater sensitivity but that comes at the cost of false positives so in the United States body breast hybrids people who are body imagers who also cross cover their partners in breast 10 have about four false positives for every true positive so they have a true positive rate of only 20% dedicated breast imagers tend to have true positive rates somewhere in the 30 to 40% range but you know even for dedicated breast imagers that means that in general we're getting two out of three negative biopsies for every positive and for non-dedicated breast imagers like four out of five negative but we've learned to live with these false positives now we all recognize that the cost of a false negative is much higher than the cost of false positive so you know we really tune our algorithms to achieve adequate sensitivity but even doing that we can get false negatives so when I think about false negatives I classify them in two ways absolute false negatives which are cases that we call by Reds 3 and suggest the possibility of follow-up rather than biopsy that turn out to be malignant but there's also relative false negatives and what's a relative false negative well that's where we underclassify a lesion we may call it 4a when it's really 4b or c now an acr by Reds edition 5 you need three things in order to call a lesion benign you need old-shape parallel orientation and well-circumstried margins that sounds really simple what could possibly go wrong right well haha a lot can go wrong so what did I find went wrong with these by Reds 3 classifications well how did the readers do overall there are 66 false negative reads out of 12,289 total reads you know that's a pretty good false negative rate that's less than 2% so did the readers overall achieve the acr by Reds false negative rate of less than or equal to 2% for by Reds 3 they did but that doesn't mean we can't learn something from the false negatives that they had so I did a very intense evaluation of these 66 cases to see how they managed to get underclassified I also was a reader and so there were 1757 masses I read these as a blinded reader just like the other readers did so I use myself as a reference standard to see how they scored and classified differently from they from them to get an idea of how I might be able to teach people to do better and how to minimize false negatives and by Reds 3 so the top three causes of false negatives were the readers were too loose on the definition of oval shape I looked at what the readers were calling oval shape and I said how in the world can you call that oval that is your workers all get out and so you know oval shape should be oval shape not approximately not sorta it has to really be oval shape the second part is a deficiency in by Reds for God knows what reason I was on the by Reds committee but for God knows what reason we decided to remove the thin echogenic capsule catastrophic terrible mistake circumscribe margins without a thin capsule do not have a good negative predictive value good enough negative predictive value you really have to look for that thin echogenic capsule and I hope we come to our senses in the next version of by Reds and put the thick thin echogenic a capsule or thin hypercocapsule back in it furthermore it's not enough to have a thin capsule in the front the back of the module you have to very carefully search out for the capsule on the sides of the lesion because the sides of the lesion is where invasion occurs it's the coronal plane is the path to low resistance in the coronal plane is on the sides of the lesion and the third thing is that our training under emphasizes the hypercoac elements you know we're caught hypok!!!! from morning until night and the best predict a value for cancer versus benign is not the hypercoac part it's the hypercoac thin Capsule the hypercoac echogenic rim and the hypercoic speculations they give us the best predict a value so you know readers did not recognize the hypercoic elements because our training is terrible. Our training is that we should only look for the dark parts. Wrong, wrong, wrong. The last thing is that unless radiologists scan themselves, they're often given two views through the widest part of the lesion by their sonographers. And so what we found is that orthogonal video sweeps starting in the soft tissue on one side of the nodule and going across to the other showed the answer where the still images obtained by the sonographer did not. So I'm a big advocate of if you if you're not actually doing the scan yourself you should be looking at video looks not still images. So all of these are things that an ACR bioreds can appropriately be considered oval shaped. In the first version of bioreds ultrasound which was edition 4 we separated macrolabulated from oval shaped. In edition 5 we decided to include macrolabulated together with oval shaped. So whether it's macrolabulated or oval shaped you can classify it as oval. So on the top are what I call flat oval. What do I mean flat oval? If I take the maximum diameter in the horizontal plane and divide it by the maximum AP dimension the ratio is greater than two to one. The bottom are all macrolabulated lesions that can be considered oval. But the two on the right number five and number six would be flat oval because the ratio of maximum diameter AP is less than two. And I'll show you why that's important later. But all of these even though they're macrolabulated can appropriately be considered oval shaped in a bioreds edition 5. Is this a problem? I don't think so. I'm okay with what we did there. Now what's really important is a mass roughly or strictly oval shaped. And here's where the consistency and reproducibility fell off dramatically. Readers very greatly in what they would classify as oval shaped. And this was a major contributor to false negative classifications. So triple negatives in particular and mucinus carcinomas in particular in topillary carcinomas in particular can be oval shaped and circumscribed. So let me show you a case. False negative case one. Absolute false negative meaning that readers call this bioreds three when it was malignant. So here's the orthogonal views of the mass. All seven readers and I call this shape oval. Now I don't know what I was thinking. I must have been smoking dope or something because when I look at this there's no way it should be called oval. First of all oval shape should have a parallel orientation. If it's non-parallel you can't call it oval. So clearly you can see on this radial view that this is not parallel. But you can also see when I draw in the region of interest around this even in the in the anti-radial view I was just sloppy. The seven readers were sloppy. We all kind of blew it. We shouldn't have shouldn't have called this oval shape at all. Now here's a video loop. This is a long axis video loop. And notice the hypercoic speculations. I mean the readers just plat didn't see that. This is not an oval shape. You can see that when I draw the region of interest this is highly irregular. It has a thin capsule on front. It has a thin capsule on the back. But it has a thick ill-defined echogenic rim on the ends. And it has hypercoic peripheral zone specules. So I mean there's a lot. There's a lot there that's not reassuring. This is a short axis view. And this is a still image from the short axis video sweep. SAX stands for short axis video sweep. So what I'm doing is scanning across the lesion and the short axis of the transducer. Starting outside in the surrounding tissues on one side and going all the way through to the outside tissues on the other side. Notice that it has a thin capsule on the front and back. But it does not have a thin capsule on the ends. This is what I'm saying. If you're going to call something a thin capsule it should be completely surrounded. The path of low resistance for invasion is between the tissue planes. Whereas between the tissue planes in the chronoplane. Where's the chronoplane on the sides of the lesion. So many of these false negatives actually were quite scary looking on the sides of the lesion. But falsely reassuring on the front and back. So the front and back don't have the answer. The answer is on the ends of the lesion. What's the problem there? Well axial resolution of ultrasound is about 100 microns. Ladder resolution of ultrasound is 500 to 1,000 microns. So we have much better resolution of the capsule on the front and back because that requires axial resolution at high frequency. Whereas showing a capsule on the sides requires a ladder resolution which is lower. And also we have critical angle phenomena. As the beam comes down it reflects and refracts off these sides. So we actually have to manipulate the probe. We have to use spatial compounding. We have to do whatever trick we have available to try to show the capsule on the ends as well as possible. But again this is showing a nice capsule on the front and back. But a thick ill-defined ecogenic rim on the ends. And actually if you look you can see alternating white and black lines. So there's hypercoc and alternating hypercoc speculations as well as purples on hypercoc speculations. So in my scoring system these are all pretty high scores. Even if you call it plump oval there's other features that should prevent the false negative. So if I plug this into my artificial intelligence, actually this is not our, this is just a polynomial regression equation. But I have a polynomial where I can plug in these scores and predict the likelihood of malignancy. You can see my likelihood of malignancy here is 96%. Anything over 95 is bioregist 5. So it's called bioregist 3 by all the readers and myself. In retrospect using the video in my regression equation is really a bioregist 5. There's no way we should have called this bioregist 3. Now these scores that I put in yellow, let's start with a shape score. Zero score is flat oval. So of the seven readers, five called it flat oval, two called it plump oval. So gross overuse of the oval shape. And I was in there, I called it flat oval too. So I made the same mistake the readers made. Now if we look at the purple zone, zero means no specules. One just means some artifactual shadowing. So seven out of seven readers didn't see the hypercoexpecules. Now what about the thin capsule? Well zero means they saw a complete thin capsule. So two readers said they saw a complete thin capsule. One means they saw the capsule in front and back but not in the ends. Two means it was circumscribed but they didn't see any capsule. So four of the seven readers really didn't see the capsule. So basically there were a lot of mistakes made there. This was a grade two, ER positive, PR positive, HER2 negative, luminal B cancer with a Ki67 to 39%. It's a bit unusual for a luminal B to fake you out. Usually it's going to be a triple negative or a mucinus that's going to fake us out. But in this case it was a luminal B that we under cross-side. Now I'll say the same thing for round as oval. Approximately round is not round. It's irregular. So we want to be strict in our definition of what's round and not. Here are my type PPVs for round. And when I compare with the readers theirs was twice that. It was forty some percent. So they were calling stuff round. I was calling irregular. So here's a lesion I called a regular that a lot of readers called round. Notice how hypervascular it is. Notice that it's stiff and the area of stiffness goes outside the hypocaloric central night. It's into the echogenic rim. Notice that it has a thin capsule up front and back. But it actually has a thick echogenic rim on the right side and on the left side as well. And it's you know there are some micro lobulations there. I mean this is this is not round. So now let's talk about what I think is a deficiency in bioreads that we need to fix in the next version. Bioreads just use circumscribed margins. I created a scoring system where I took circumscribed margins and I created four sub categories of circumscribed. So a score of two is the current ACR bioreads definition. The margins are circumscribed but I can't see a thin echogenic capsule. A score of zero I see a thin hypercoic capsule on the front and on the back but also on both sides. So that's a score of zero. I have a complete thin capsule all the way around. A score of one is I see a beautiful capsule front. I see a beautiful capsule on back but I can't really identify it on the on the sides of the lesion. The score of two is I have a well-defined echogenic capsule but it's a little thicker than normal. Usually these are post-inflammatory but occasionally they can represent cancer. So basically the current ACR definition is all the way up at two. I've got three scores that have lower likelihood of malignancy than the current ACR classification. So let's look at a second case now specifically looking at capsule. So here's a mammogram with a mass and we can see that on one view the mass is completely circumscribed and on the other view it's either obscured or partially indistinct. It's kind of hard to tell but the outside is really pretty reassuring. You know most people call that flat oval shape and it's got a ratio of two to one exactly two to one. Again these are from video sweeps and they look pretty reassuring. Enhanced through transmission oval shape but we're gonna look at this more carefully now as I go through the video. Also the Doppler is positive. Again you can't trust the negative Doppler but this is way more vascular than I would expect if I've read. No I'm gonna be. It's gonna be a papillary lesion or cancer. So the Doppler, I don't think you can downgrade with a Doppler. I never trust the negative Doppler but I can certainly use a Doppler to upgrade. So let's load the video tape. Here's what I want you to see. Is this spicule or is this a normal tissue plane? The answer is I don't know but to have it cut off like that, that's not a good finding. If this were a benign tissue plane, it would be continuous with the anterior capsule or the posterior capsule. When you see a tissue plane chopped off like that, that means there's an invasive cancer there. Again, beautiful anterior, beautiful posterior capsule, a little bit of a thick echogenic halo on the right and just kind of micro-lobulation and maybe even some angles on the left side. Again same thing here. Notice that that tissue plane is completely cut off. That's just not a reassuring finding. And again, even though the capsule is beautifully thin and well circumscribed in the front and back, it's a little thick and it'll defined on the sides. Now also notice that I've got some hypercoexpicules in the boundary zone. So now we have a thick echogenic boundary zone on the sides of the lesion. Despite the thin capsule on the front and back, I've got Frank short boundary zone hypercoexpicules. Again, I want to show you the tissue plane cut off. Nice thin capsule front, thin capsule on the back, shown by the green arrows. Thick L defined capsule shown by the pink arrows on the side. Again, if I feed this through my regression equation and again, this is a frame taken from the video look clearly that's not oval shaped. And clearly in this particular case, do I have a partial thin capsule on front? Yes. Do you have a partial thin capsule on back? Yes. What do I also have? Thick echogenic rim and boundary zone spicules. So in ultrasound, if we have a mixture of reassuring and suspicious findings, we always go with the more suspicious finding. So what I say this is a partial thin capsule? No. I would say that this is either a thick echogenic rim or Frank boundary zone spicules. I have to go with the more suspicious finding. What I caught, you know, there's a roughly oval shape, but there's angles. So am I going to say this is irregular with angles or am I going to say it's oval shape? I'm going to say it's irregular with angles. This is going to be very similar to the other case. Once I plug this into my regression equation, I'm going to get a bioreds 5, likelihood of malignancy. I'm greater than 95 or 95 or greater is bioreds 5, not 4a. So what we're seeing again is the same three things over and over, overuse of the oval shape, inadequate recognition of a thin echogenic capsule and complete blindness to the hypercoexpicules. Nobody's looking in the surrounding tissues. If I go back and draw this line, okay, I typically draw two regions of interest. I don't think you need to do this forever, but it's good training. This is the internal zone, and this is all that bioreds looks at right now because they took away the boundary zone. They don't look at the boundary zone. So internal zone is inside the white line. Boundary zone lies between the aqua and white lines. The purple zone lies outside the aqua lines. Everybody's blind to the purple zone. I mean, the best way to see these spicules is a short axis video sweep. You can see these hypercoexpicules pulling in and going out, retracting in and going out as you sweep across the lesion back and forth. If you put your video loop into a sweep back and forth mode, it becomes very apparent. It's much more apparent than any still image. And the yellow lines are showing the hypercoexpiculations. This was a grade 3 ERPR negative, HER2 negative, triple negative cancer with a Ki67 to 60. This is what we expect. So this was a case that had a large biologic component. We know that biologically triple negatives are more likely to fool us. But you can see that by storing video loops and carefully looking at the frames as we swept through it, I could correctly call this Byred's 5. But the two still images that I originally showed you, they look falsely reassuring. And this again is a reason that I've become a big advocate of storing radial and anti-radial short axis video sweeps through the lesion and doing my reading from them instead of two still images. Now, this is showing you that no single feature can achieve a false negative rate of less than 2%. So for the readers, the best they could do with flat O was 6.6% false negative rate. The best they could do with a complete thin capsule was 3%. If our goal is less, is 2% or less false negatives, boom, we fail with single features. What this shows is only by combining a truly oval shape with a complete thin capsule can arrive successfully at less than a 2% false negative rate. The complete thin capsule alone doesn't do it. Partial thin capsule doesn't do it. Or partial thin capsule plus a flat oval shape doesn't do it. Only the combination of the thin capsule and flat oval shape achieved the desired false negative rate of less than 2%. Now, what this shows is that margin alone just completely bombs out. So this is a circumscribed alone 22% false negative low for mid 4b. Circumstribe plus flat oval, low 4b. It doesn't even achieve 4a. So the ACR definition of a benign margin circumscribed is woefully inadequate. Ignore it. Look for the thin ecogenic capsule and hopefully we can get it fixed in the next edition of Byreds. Circumstribe plus plumb oval back in that mid 4b range and round plus circumscribed, you know, sort of upper mid 4b range. Clearly circumscribed with any combination of shape totally fails to define any population with less than a 2% false negative rate. So in summary, also studies are more likely to be classified as oval shaped especially plumb oval than our true positive exams. Part of that is biology, especially because grade 3 and triple negatives and some usinus and papillaries may look that way. But I think the biggest component is interpretive error. People are way too loose on what they call oval shaped. Roughly oval is really irregular and should be called irregular. It has to be strictly regular. The radiologists really were not looking at the sides of the lesion. Invasion occurs in the path of low resistance. Where is that low resistance on the sides of the lesion? Is it on the front of the lesion? No. Because you'd have to go through two planes to make the front irregular. Is it on the back of the lesion? No. Because again, you'd have to go through tissue planes in order to create irregularity on the back end. Where does irregularity occur? Where there's low resistance to invasion? Where is there low resistance to invasion? Between tissue planes. Where do we see the reflection of between tissue planes on a breast mass on the sides? Still images don't give us an adequate view of the sides of the lesion. Video sweeps give us a much better appreciation and you want spatial compounding and some healing and towing of the probe to try to actually fill in thin capsules on the sides of the lesion. The other thing is radiologists just are failing to see the white parts of the lesion. We've been told for years, look at the dark part. It's all the answers in the dark part bologna. That is fallacious. The white parts of the lesion are far more predictive of whether it's benign or malignant than the dark parts of the lesion. You have to train yourself to look for the white parts of the lesion. Not inside the lesion, but in the boundary zone and in the surround tissues in what I call the peripheral zone. And the ACR current descriptor of circumscribed margins is woefully inadequate. We need to add the thin capsule and the thick echogenic rim back into our lexicon. Thank you.