 Welcome back. My name is Steven Seltzer. I'm the radiology department chair at Brigham and Women's Hospital in Boston and what a treat to be here with you today to celebrate this very special occasion and what a great morning we had with such an amazing diversity of provocative and exciting topics of conversation and the afternoon will not let you down either. We'll be hearing from some of the young investigators in our field, winners of presidential early career awards, we'll be hearing from a distinguished radiologist, we'll be hearing from a Nobel laureate, we'll be hearing from a winner of the National Medal of Science and most importantly we'll again hear a patient story. So I wanted to thank Dr. Pettigrew and the NIBIB team for putting all this together and appreciate very much the chance to moderate the afternoon session. Now let me introduce our next speaker who is Carla Pugh. Carla is a winner of a prestigious presidential early career award for scientists and engineers, the P-Case Award. Carla is a associate professor of surgery at the University of Wisconsin, she's vice chair of education and patient safety and she's doing very innovative work in a topic that's very relevant to the NIBIB portfolio which is the use of sensors and simulation technology and we'll hear about her exciting work. Carla, it's nice to have you. Well thank you very much for the opportunity to share my work with you, it's truly an honor to be here and last night we heard some of the history about NIBIB and I have to say I'm pretty sure I speak on behalf of all of the researchers that we are extremely grateful for all of the hard work perseverance and innovation that went into bringing NIBIB to NIH as an institute so we are very thankful. Palpatine is the use of touch in clinical practice for diagnostic and therapeutic purposes. The motivation for my work comes from my own personal experiences as a medical student and a surgery resident in the clinical arena trying to learn what it is that my esteemed professors were experiencing and doing on a daily basis and many times as I watched them perform surgeries and physical examinations I would find myself wishing and hoping that somehow their level of discernment could magically be transferred to my fingertips but obviously that was not possible. There's one story that I experienced that's worth telling and I remember being a second-year resident and spending time on the urology service and there was a patient who was having a procedure relating to the prostate and here's a picture of a normal prostate and here's one of an abnormal prostate where there is blockage of urine and this patient has a problem voiding their bladder. This particular procedure was called a suprapubic prostatectomy and as a second-year resident this would be my first time seeing this procedure and I was very excited and I stayed up all night reading in the textbooks and looking in the anatomy text to see what was going to happen and much to my disappointment this is what I saw for 30 minutes in the operating room the back of the surgeon's hand and as I stood there quietly as a second-year resident intimidated by this very intelligent surgeon and the senior resident fellow who was there I found myself bursting with curiosity as I looked at the senior resident and wondered how is it that you are learning what he's doing and you can't see and so I asked the question and in this day it was a little different I trained on the East Coast and junior level residents aren't supposed to talk in the operating room but I had to ask the question I said how how are you learning I said can I feel what you're feeling and the room went silent and he said sure and he took my hand and put my hand inside the patient's bladder and I felt something that was sort of mushy and I had no idea what it was but this was one of the one story that's pretty similar to some of the other experiences that I had during my training and I actually found it pretty frustrating but I still had the desire to learn and that's what led me to go to Stanford University and get a PhD in education after my surgical training and while I was there I took classes on human computer interactions and learned a lot about sensor technology and thought of all of the things that I could do to change and help medical education one of my class projects would actually ended up being the technology that I used for my dissertation was using force sensing resistors or sensors and placing them on teaching mannequins and I actually chose the pelvic exam not because I'm a OBGYN doctor or a gynecologist but because there was a professor emeritus in the lab who had a model and so I put sensors on the model and this is a picture of a normal uterus most of the mannequins have interchangeable parts and this is a normal cervix and uterus and this is an abnormal uterus with vibrate tumors and so there was no textbook or any information to tell me where to put the sensors but I sort of stood back and looked at where people touched them where they weren't supposed to touch and I put sensors in those areas and you can tell this was a long time ago based on this computer but basically this was the this was one of the models that I built and I used it in the classroom and it was very interesting basically the gist of it is that you can see where someone is touching and with how much pressure and instantaneous immediate visual feedback when you allow them to look at the computer screen and along the ways you know in using this both as a teaching tool and an assessment tool I have these stories that I call the epiphanies and I only tell one there was one student in London I did a visiting research fellowship there who was at the end of her rotation and we did an assessment of their knowledge and we had three or four mannequins that had different anatomy on the inside and after they all went through and did the examination without looking at the computer they did the exam as if they would normally do on a patient we collected their data and they wrote down what their clinical findings were and then we turned the screen around and did the exam and told them what the findings were one of the students was very angry she said I've been doing this exam wrong for two months why didn't I have access to this before I did an exam on a patient and we didn't really have an answer we said well you know this is we're glad that we know we're teaching you how to do it now but it was sort of a shocking scenario for the student and that realization became clear that when you're trying to teach an internal examination to someone who can't see what you're doing they're trying to mimic a behavior that they really don't see and then when you're observing them you can't really see what they're doing and so obviously we then went the next step in terms of building all of the epiphanies and things that we knew that people were doing wrong into the curriculum the other one which was interesting was that there were some students that were naturals meaning they were good they had not had much training they could put their hand inside and they knew that there was something abnormal this doesn't feel like the picture there's some fullness on this side and there were other students who would touch the mass which as you see here there's a mass here in the right at NUXA and I could I know they were touching it because I can see the sensors going off on the screen but they had absolutely no idea that they were touching anything and so we sort of coined the term that there are people who are haptically gifted and unfortunately there are some who are haptically challenged but you can't tell who they are just by you know looking at them this is an example of this is another after I had some success I started putting sensors on every model that I could get my hands on and this is the students at Northwestern using the digital rectal exam sensor simulator and you see they're all looking to the right because they're looking on the screen checking their hand movements the other source of research for me really has been the data that comes out of these technology I can as a surgeon learn how to write a little bit of C++ code just enough to be dangerous much to the chagrin of my engineering students and basically this is a you know less than a second worth of data that comes out and all the sensors are sampled at a rate of 30 Hertz which is faster than any human being can do anything so I didn't know what we were going to do with this data I didn't want to miss anything and this data has mainly been the source of all of my research questions since I was a graduate student this is the data plotted in the line graph formation and for my dissertation the assessment variables if you will in terms of performance that we use with the time that it took to do the exam the number of areas touched during the exam the average maximum pressure used when touching each of those areas and frequency within a sort of narrow mathematical definition when you look at this graph each of the different colors represent a different area of the anatomy and so you can pretty much see what someone was doing from beginning to end and tell a story in terms of what they were doing during the exam now the next slide that I will show you is a slide of a line graph of two medical students are during the course of my research that wanted to use the simulator to teach they were fourth year students that wanted to teach first year students using the model and by that time they had used it and then went from the model to a patient and found it to be very useful so they wanted to use it with students and then just to the pelvic exam is basically you have one hand inside the vaginal vault here and you apply counter you apply pressure on the cervix here and this pushes the whole organ towards the anterior abdominal wall and then you also palpate the other the fundus of the uterus with your opposite hand and so basically trapping the organ between your two hands and these are the two exams of the students I can tell you the the ratings here on this scale are a little different because I had to squeeze this exam to fit it on the screen compared to this one it took twice as long but it's plotted as amount of pressure over time and you can see that right away the student went in did touch several areas on the cervix and then began to do several course palpations on the uterus while holding constant pressure on the left posterior part of the cervix whereas this student touched several areas of the cervix here held constant pressure on the left was not able to find the uterus to palpate it switch positions went on the right then did left and right back around anterior and so on and so forth and still was barely able to touch one portion of the uterus at the very end and we could not tell the difference of what was going on before in clinical practice when you're standing watching so on but now I'm looking at this data so clear rather some students who could fill the uterus and some who didn't it's just part of their technique and their palpation was not there they don't all come one-size-fits-all everybody does something a little different and so the question remains you know should the second student fail or how do we grade this or you know what do we do with this information I then sort of even went out of the realm of Pat palpation any procedure that you can do that doesn't require cutting otherwise you would cut the sensors I can still put sensors on a mannequin and watch other procedures and this is one for intubation and when you look at this the top graph is an experience intubation and the lower graph is one of someone who's inexperienced and if you look closely again I had to squeeze this graph to fit it on the screen it took almost three times or two times as long as the one on the top what you see here most importantly is the sensor this brown one is the center that's on the tongue and it's not sort of intuitive but that's where the laryngoscope rests and so this person experienced anesthesiologist put laryngoscope in the mouth and held pressure and then it passed the breathing tube past the vocal cords and this first blip is the tip but the second one is the balloon and it took me a while to figure it out because I'm looking at this graph and then I go back and watch the procedure and as I start to collect this data all of the procedures that we do have a signature and so my students in my lab could show me any random graph and I can pretty much know whether it was a prostate exam or an intubation or something because they all have a signature but when you look at this performance here this person did several move manipulations of the laryngoscope held pressure wasn't able to see the cords and the other thing you start to see your other sensors here and this one was one of the first ones where I was able to make it clear to everyone that it it doesn't mean that you're doing a better job if you touch more sensors in this case when you start to see since additional sensors going off that's because the person is really struggling and they have severe tilting of the neck and pressure on the teeth and areas that they shouldn't be touching and then they finally were able to pass the ET tube when you could see they really didn't have good control of the over the airway so I'll move on to another exam the digital rectal exam and this one I have more more data but this was a quote that's on the web page for the American Cancer Society and you can read it and I like the term subtle prostate abnormalities and I think this is great because the performance expectations are pretty high but we have absolutely no way of ensuring that any clinician can perform at this level and so this was my interest in doing this work I buy several mannequins off the shelf I also modify them to try and get a realistic range of clinical presentations and these are diagrams show if this is sort of a view through the rectum in an anatomical diagram of the prostate here we built two models one that had a fairly large prominent nodule on the right lateral lobe of the prostate and then another one that had enlarged lobes bilaterally and also a rectal mass and so this is actually not uncommon when you have an elderly gentleman who may have a lesion and these were both pretty subtle and so this was meant to be a easy exam and a difficult exam and so for the first pilot study I had students from a probe 30 medical students 25 residents and 30 faculty examine both of these models as unknowns and when you look at the difference here there's some interesting things going on the medical students don't spend as much time as the faculty or the residents the number of sensors they touch are about the same no different both the students and residents don't apply as much pressure as the faculty frequency is lower and you know the difference is here both on the rectal findings and prostate findings where the students were way off on the more difficult simulator and this was a sort of encouraging one of the things that we found after looking at what's going on with the students usually you would think the students would spend a longer time because they don't know what they're doing ironically what we found is that they actually spent less time in this case because they didn't know what they were doing and this is the scenario I have no other way to show it except that when they learn how to do the the procedure they watched a video and this was the video the same thing the back of the hand and this is all they saw now what they were doing after we looked at the data because we couldn't figure out why they were only touching three or four sensors in the center they were just pronating and supinating their finger in the anus whereas the instructors were doing this maneuver and the students never knew and we never knew that they didn't know and so once we looked at the data and mapped it out and we figured this was going on then we took that back to the classroom and made sure that the students now using the simulator can touch all of those areas both laterally and they understand and many of the students say yeah I would have never figured that out the textbooks or static pictures and you look at the video and it's the back of someone's hand so we titled the paper lost in translation and so we also then got really encouraged and then did a bigger study using the same models but we basically over a two-year period collected data from several hundred urologists at a American Urologic Association meeting colorectal surgeons at their annual meeting and then 200 students that were third-year students during their pre-testing before going into the clinical years and so there were no differences in their performance on the easy simulator they all pretty much figured out the normal rectum and the large prostate nodule not a hundred percent but but not bad and then when you look at the more difficult simulator is pretty interesting so for the prostate exam the urologists were the most accurate compared to colorectal and students but then when you look at the rectum it rectum the urologists were actually less accurate than the medical students and when we got into the details of this and look back at the surveys many of the urologists almost 40% of them did not complete first they admitted that they don't do the rectal exam so they they focus on the prostate and they never supinate their fingers some of them openly admit it but the other part they just didn't even even though the form is said to complete the findings for the rectum and the prostate about 50% of them left the rectal findings blank and so that gave us a quantitative number in addition to the anecdotal comments by some of the people that say they don't do an exam and I think the problem with this is that you know you sort of know that this happens here and there as people become more and more specialized but it's not really clear that it's public knowledge and so the scenario where you have a a patient who goes and sees a primary care physician and the primary care physician says well your your history sounds like you've got urologic problems I'm gonna send you to a urologist and I know you you know the rectal exam is uncomfortable I'll spare you the exam and because urologist is gonna perform it well this has happened more than once it's happened to me where I've had a patient and I'm a surgeon who had gone to a urologist and then came to me and really did actually have a rectal cancer so it happens we don't have a way to quantify it the study that I've done which I'm actually just in the process of publishing is a little window I believe into what may be the truth out there that the specialist focus on the organ that they are a specialist for and we sort of need to understand this is a new evolution in medicine and what the implications are for our patients we then got really sort of bold and decided well we've gotten some luck with the with the prostate exam the digital rectal and the pelvic exam let's try the breast and I say this is bold because the breast is a different beast of sorts if I can use that word the anatomy is has much more variation in it than the prostate or the pelvic exam and obviously a lot wider variety of both normal and abnormal clinical findings in addition the technique you're looking here you some people use both hands some use one there is a broad range of degrees of freedom and so it's a little more complex if you're trying to look and understand the breast using sensor data and this was the project that we were funded for our R01 for an IBIB and it was to validate a set of sensorized breast models for high stakes clinical skills assessment and the first thing was to develop and evaluate these models to see that they can be used to quantify breast exam skills and then we take wanted to take it to the next level in terms of validating it as a high stakes assessment and I'll talk about that again I looked on the market at all the models that were out there and many of them unfortunately and I'm sorry if any of the vendors are in the audience but they're quite awful and some of them are too hard and some of them don't have they have homogeneous material with a big lump and that just is not good enough for medical students or residents or physicians that we want to say you have the minimum competency or even mastery in terms of performing the breast exam so we buy parts and we build you know the layers that we need to represent a wide range of patients these are sort of our basic scenarios in terms of a firm mass which could be either a fiber add no more I cancer we can put it any location make it any size I mean we do no masses we also represent the wide range of normal the breast of a normal 80 year old woman feels very different than the rest of a 15 year old and so we try to also represent that range the same with cystic masses our early excitement was that just in terms of placing sensors on the models and you see the sensors here in color that we can actually differentiate between the two most common types of breast techniques that are used the first one you see here is called the strip technique and you can see by the colors the person started in the upper inner quadrant here and then proceeded with the rest of the exam on this one the person used a circular motion starting at the breast and here you see yellow and coming out and there's you know the variations there so we were happy at least that with one model and the sensors that at least we could tell some difference in the technique the other excitement for me was finding the work of these two researchers who have done over 30 years of research on touch and they have found some very interesting things regarding object recognition and that human beings use reproducibly and subconsciously six specific maneuvers to decipher an object and many of you may have been the subject of some of their experiments or at least a relation where someone asked you to reach into a black bag and pull out the nickel and how do you know it's the nickel there may be a dollar a key or something else in there but you know it because you felt it before you know what it feels like but what they've shown is that what you do to figure it out is subconscious and it's reproducible in terms of if you you know you can tell by it by the weight by the pressure the hardness of it and so some of these maneuvers we were able to actually see in our data the ones that were usable in our clinical exams where the lateral motion the pressure and in sort contour following obviously we're using human body parts and patients that are still alive so we were not we can't do unsupported weight with the prostate because we can't unless it's taken out so these three were the most common ones that we used and so we we actually purposefully just perform those maneuvers on our models to see what the waveforms look like and the first one you see is when you're sort of tapping we didn't have a language for it when you're tapping in a single area without releasing full pressure the second one is actually tapping in within a small radius and then rubbing which is related to their lateral motion in a single area in a multiple areas and you start to see that actually some of these waveforms look differently depending on what you're doing and that gave us some encouragement one of the difficult ones is that you can't really tell what the sensors we were using which only are used for direct force you can't tell the difference between lateral and circular motions because it's it only detects direct force so we then collaborated with a group of engineers that develop biologically inspired hair cell sensors and 3d sensors and these sensors then can detect directional forces and now that we compared the hair cell sensor with also 3d sensor and at this point because of the data is easily discernible we are initially calibrating our mathematical algorithms with the 3d sensor because it gives us a little more differentiation with the lateral and circular movement so this one is vertical rubbing and you can see the blue line is just the direct force waveform but the purple and the red one on below show the x and the y axis on the 3d sensor and so instead of having one signal we now get two signals that help us differentiate that from the horizontal movement which you can see the purple and the red sensor on the right side here look differently so now we have a means of being able to tell the difference in directional forces and that can give us a little more information about the details the other part with our grant is that we have some consultants from the national board of medical examiners in ecf and g and these are the people who actually design licensing examinations and these are the metrics persons who are helping us to understand how we can use this data to decide cutoff scores and for and for performance so we've got a really good team and really excited about this work and I'll just end what's making a little comment about how this relates in the bigger picture with respect to quality of care I think some of you are already thinking about that but traditionally and I am old enough to remember that physical exam used to matter and we used to do physical exams and specifically then order a test that was of relevance based on our physical exam findings and so you would either get blood drawn or go to a CT scan or get a biopsy or a procedure right now what we're starting to see is a wave of scenarios where you don't touch the patient you just send them to the CT scanner and I think that's problematic I checked with my radiology colleagues they think that's problematic as well and when you look at it it actually is market increase in the number of CT scans that people are ordering per year and sorry this doesn't show but they're starting to be a lot of talk about it as well imaging for whom is the patient of the physician and when you look at some of the research studies what you start to find when people get into the details the physicians who aren't as comfortable with their physical exam skills tend to order more studies and that's a problem risk overuse of CT avoiding you know and so there's a lot in here about that and I think it does affect how a patient navigates through the system how be it if you have a cystic lesion that somebody should be able to stick a syringe in and draw the fluid out and send it to pathology but instead you go to radiology you get one exam and it may have been the wrong one because you really never really touched the patient and then you get a second one and then that patient ends up spending a lot more money and spending a lot more time getting to the point of diagnosis and I think that's a problem and we need to do a better job at striking a balance so I think that when you're looking at this it would be great that we could now find a way to assess and certify people that they're competent in doing the exams and get them some actually you know motivation to to do the exam and feel good about what they're doing because we have measured their competency and then they're ordering the right test for the right reason and I could say I have in my conversation at dinner last night with Dr. Seltzer that you know there are lots of groups that are now looking at physician or decision support system that you can't now just order any test without really putting some information of what you've done to show that this is the right test for this patient at that time and again I think that we can sort of add some things in the process that can help certify that clinicians know how to do the right exam and I think that this will definitely have an effect on the quality of care thank you so correct technique that is that's definitely the topic of my research we don't we've never we don't know what it is and so now I finally have a way of of collecting that data that's exactly what I do so anytime I build a new simulator for example three or four of them I take it to the specialist meeting and I have several hundred of those clinicians do the exam and basically it's it's always it's never going to be a hundred percent perfect because we're not perfect in clinical practice but what it does is it gives me a range of clinical scenarios that I know the experts know and I know how they do it I know what they do to get to that answer and you can see that the people who get it wrong they're not using the same maneuver so there are commonalities in what the experts do the ones who get it right and so that's pretty encouraging and exciting to to move forward with respect to the I think your second question relates to mastery if you will and can you can you can you teach someone you know those persons sure I think so I think that in the goal of my my research in the end would also not only just to be to be able to find and define a set of simulation scenarios that would represent minimum competency that you expect for undifferentiated medical student but then another set of clinical scenarios that are for the higher level trainees and the experts who want to maintain a certain level of mastery or even reach mastery and a lot of it comes within their description and what they will do so it's one thing for me to put a lump in something and say you know what is this it's another thing for me to say describe it define it and tell me what you're going to do with this patient and so at in the end when you're going from a specific level of basic competency to mastery it then goes to decision-making as opposed to just your skill in that you have a high index of suspicion that something's not right and you need to do something else to investigate so that's what we're building into the scenarios as well probably so well more an aphorism in the quality improvement literature if you can't measure it you can improve it and you made a breakthrough now that we can measure our performance on how we do things like thank you