 Welcome to this live video teleconference on human factors engineering and medical devices. I'm Mark Barnett, communications director and FDA center for devices and radiological health and I'm going to be serving as your moderator this afternoon. Some of you may remember that one of our teleconferences several months ago dealt with minimizing medical errors over the timely topic back then and it's a timely topic now because medical error is getting a lot of attention in the health care community, in the Congress, in the news media, in the FDA certainly and among the manufacturers of medical products. As we all know part of the reason for that increased attention was the report of the Institute of Medicine that estimated that medical error kills tens of thousands of people each year and in this country alone. What's important this afternoon is realizing that part of that toll is associated with medical devices. We can see that here in the FDA. If you look at recalls you find that almost half of them are the result of design deficiencies and if you look at the medical device reports that we get about a third of those cite use errors and many of those reports involve patient injury or death. It's not that these errors are being made by people who don't know what they're doing and errors can be made by fully competent health professionals. Sometimes that happens when the interface between the user and the device is confusing. Maybe the control panel is cluttered or a display is poorly marked or it's poorly lit or the instructions aren't clear. Sometimes that happens when the user is stressed out or tired or when the environment is noisy or there's a lot of distractions. You just think about the sounds and the stress levels in a busy emergency room. To prevent use errors devices have to be designed with the user in mind right from the start taking into account that person's abilities, limitations, the environment in which the device is likely to be used. Some people call that designing the user into the device and of course another name for it is Human Factors Engineering which is the science that studies the interaction between people and equipment and of course that's the subject of this teleconference. Human Factors Engineering plays an important part in the strategic plan of FDA Center for Devices and Radiological Health and so we asked the center director, Dr. David Feigl, to tell us a little bit more about that. Let's watch the video. As many of you know, our center has recently embarked on a strategic plan that's built around what we call the total product life cycle. What that means is that all of us, manufacturers, regulators and users, must be attention to the safety and effectiveness of a product throughout its existence from the initial concept to the development of testing to marketing its actual use and all the way through to the obsolescence. This teleconference couldn't be timelier because its topic, Human Factors Engineering, is a critical component in applying the total product life cycle concept to medical devices. Human Factors Engineering is a powerful tool to improve the safety of medical devices and reduce the potential for use errors. It means taking into account the abilities and limitations of the intended user when you're designing or modifying a device or as well as using Human Factors principles to understand and anticipate potential hazards. At the earliest stages of design, Human Factors means considering the ways in which a device is going to be used, the capabilities and limitations of the users, the environment in which the product will be used and the possible problems that users might experience. Later, including Human Factors considerations in clinical trials, can enable you to demonstrate the safety and effectiveness of the device in the hands of intended users. Moving on through the total product life cycle, during the period of initial product release, it's very important for all of us to watch carefully for hazards that weren't anticipated. We know from experience that applying Human Factors early in the game significantly reduces surprises at this point. But of course, unanticipated problems can always occur, and if that happens collaboration between industry, the clinical community and FDA can help to find and fix problems early. Human Factors Engineering is important after a product is marketed too. As problems arise, it's sometimes tempting to simply chalk them up to user error and forget them. But it's important to go beyond that and to be especially vigilant to use errors. That means investigating and documenting problems and identifying trends. And it's also important to be mindful of changes in the way that a product is used, such as new environment or different users. As you go about updating or modifying a product, it's important to consider new information on how the product is actually being used and the changing needs of users. Again, Human Factors is the key. And of course we want you to remember how important it is to report use problems to the FDA. This is an essential step because it assures that we're able to do our part, which is to identify trends in the larger aggregate, and that in turn can help you in managing your product. Today you'll be hearing from key individuals from FDA, as well as Human Factors experts from industry and the clinical community. We hope this discussion helps you use Human Factor Engineering in managing your device throughout the total product life cycle. Thanks, and I hope you're going to enjoy this program. Okay, we're back live and I want to tell you a little bit now about today's format. We're going to focus on two main areas using two separate panel sessions. The first one is going to be a panel of FDA people who are going to talk about this agency's current role in the area of Human Factors Engineering. And that's going to include pre-market review, field inspections, and consultation and guidance services. The second panel is going to consist of outside experts from the industry and also from the field of Human Factors Engineering. And they're going to discuss how Human Factors can be incorporated in the design and development process for medical devices. We'll also have some other interesting things to see during the panels. For example, a conversation with Dr. Statlin, the new director of the Office of Dice Evaluation, and there's a videotape segment that we recently shot in a hospital showing the effect of Human Factor design in the real world of medical practice. And of course, as in the past, we're going to reserve plenty of time at the end of the show to take your questions and comments, either phoned or faxed or emailed. Now those numbers should be appearing on your screen now and they're going to reappear from time to time during the broadcast. You can start sending us those questions any time beginning right now, and as in the past, you don't have to identify yourself. If we get more questions the way we have time to answer on the air, we're going to provide answers to them on our website sometime over the next few weeks. And by the way, this broadcast is also being shown on the internet, and the internet version is being archived on the web so you can go back later and review parts of it. The internet address to do that is going to be shown on your screen and we're going to leave it up long enough for you to jot it down. And now let me introduce our first panel, a group of three FDA representatives. All of them are from FDA Center for Devices and Radiological Health. Ron Kay is a Human Factor Specialist with the Office of Health and Industry Programs. Bob Gatling is Director of the Program Operation Staff in the Office of Device Evaluation. And Kim Troutman is a medical device quality systems expert in the Office of Compliance. Ron, let's begin with you and clear up a little terminology because we hear about the term use safety and then we hear about use error. What's the difference? Sure. Okay, if a person uses a medical device that's perfectly operational and they end up making an error or errors and those errors result in ineffective treatment or perhaps injuries and death, that's use error. And now you can design devices so that the likelihood of use error is minimized and when that's done, that results in what we call use safety. Just in a very general way, scope out what it is that you have in mind, what an individual or company would have to consider in looking at human factors engineering. Okay. On a very general level, there's three major areas that... We should be seeing something like a screen, right? I'm looking for it on the screen. If we could get that graphic up there. There we go. Okay, there's three major human factors areas. That's like live TV, right? Yes, go ahead. Okay, the use environment, the device user and the devices themselves. It's helpful to think about these things separately and of course you have to think of how they interact as well. The use environment of course is the environment where the device is used. You can have operating rooms or any environment where light levels, noise levels, vibration, temperature, these types of things apply. At the home, you have situations where for home users there can be inadvertent, abusive devices, bumping and other problems in the home. Community in the shower. Oh sure, that's a good example and that has happened. But also just stress, scheduling and stress dealing with patients that are having severe medical problems can apply a lot of pressure to users. So the environment is very important to consider. Next of course is the users and of course another centrally important component of all this and the important thing there is that users vary in their ability. Their hearing ability, their ability to see, to understand, familiarity with medical procedures, that type of thing. Their expectations about how devices will work based on their familiarity with other similar devices can be very important. We also have to consider their experience with a given device or the training that they've had. And so you can have users that start out with a relatively low level become better after a time perhaps. But it's important to realistically estimate what the capabilities of the user population are going to be when you think about these things. Then of course there's the device itself and for human factors we're interested in what we call the device user interface. And that's all the components of the device with which users interact. What they touch, see, feel, hear and so on. So you can see how these three areas of consideration on the diagram, they come together when the device is actually used in the real world and the result is going to be either safe and effective results which is what we hope for or unsafe and effective results or use error. Okay, thanks. Before we go on with this conversation I want to show a videotape because I think it's going to apply to what we talk about next. Before we go on and talk to Bob Gatling about the pre-market review aspects of human factors engineering, I had a conversation recently with Dr. Bernard Stadlin who's the new director of FDA's Office of Daluation in the Center for Devices and Radiological Health. And I asked him how he viewed human factors engineering and the role that it plays in evaluating new devices and while we're at it, Dr. Stadlin and I also talked in general about how he views his new job and what he sees in the future. So let's watch. Dr. Stadlin, we're going to make you do double duty this morning because I want to talk to you first about your new job and how you view it and your view of the future and also then talk to you about human factors and how that affects the product review process. So let's get into you. How long? You've been here about seven months, is that right? That's right. Seven months. Enough to accumulate maybe 6,000 emails. Emails and meetings are really the highlights of my job. Right. It's the name of the game. Why don't you start out talking a little bit about your background because I think that the audience might be interested in that. We're on about three or four sentences. I'm a native of Minnesota. I have both an MD and a PhD and I'm a board certified clinical pathologist. In many ways I think I've covered the four sides of the square. The first side academics where I was a professor and a researcher. The second side, the hospital side. The third side industry and I think that's very important in what I do now. I've been a CEO and president of a large commercial laboratory and a consultant for various companies. And now I've come full circle to mix metaphors, the square and the circle now working with the government. Both sides of the street. In other words, that's very interesting. How would you characterize the relationship between your office and the industry? Do you think it's what it should be? Do you think it needs to be changed? I think it's getting better. As I understand it in years past, the relationship has really gone on one side of the spectrum from an adversarial relationship to one of collaboration. And I think today we're much closer to the collaborative model. However, I do think that we always can improve upon our interaction with industry. You know, the fact that I'm not going to ask it today about backlogs, I think, is indicative in itself because those really do seem to be a thing of the past and we're all thankful for that. And I'm sure you agree. But beyond that, there are other ways, perhaps, that we still in the FDA can use some improvement in our relations with industry and our performance. Do you think that's true and what would they be? Well, I think we can improve that relationship. I think the first thing is truly communication. It's very important that the FDA communicates with industry as to what we need and the application process in terms of what the new technologies will be and what our expectations are. On the same hand, I think it's very important for industry to communicate with us, continue to have a high level of integrity, let us know when adverse events occur so we are not blindsided. And that's really the basis of a very positive interaction. You said at some time several weeks ago that you believed there were four Fs. And I sort of grooved on that. Do you remember what they were? Well, I certainly do. There may even be five Fs. But the first F is one of our friends. Sure, this is a family show. So be careful. Go ahead. The first F is friendliness. And I think that's very critical. I've instructed the staff to answer phones promptly, to be friendly with the individuals and to have an open policy. I think the second F that is important is fairness. To treat all comers equally. All petitioners have the same opportunity of getting our ear. And to approach things in a very rational way. Obviously, the third F, fast, is critical. It's very important to get the information in and out in a very efficient manner. Companies have a burn rate. And the longer they wait for an assessment of their submission, the worse it is for them. The fourth F is really one of follow through. And I think it's very important that not only for us to be involved in pre-market, but also to work with the other offices as a product is in use and clinically used to be certain that it works. And the last F I would add is firm. Once we've made a decision, once we've asked for something to continue that same policy so that there's predictability. So if we follow the Fs of fairness, of friendliness, of firm, of fast and follow through, I think we'll be doing a good share of our job and the interaction with industry will be a very positive one. Let me turn the question around and ask you to look at the industry. How can they improve? What would you like to see them do in terms of improvement and performance? Well, I think industry can do many things. I think, first of all, in the quality of their submissions. Remembering that the reviewer is just a person looking at a submission in the same way that your English teacher looked at the themes you wrote. If it's well put together, if it's typed, if it's done well, you'll probably get a higher grade. Well, I think it's very critical for the industry to submit applications that are easy to read, that are well organized. I think second is to be forthright, to present all the information as soon as it's available. And as I said earlier, when problems come up to let us know about it. And last but not least, it's very important for us to have the technical expertise ready to evaluate new applications. So if there's a new emerging technology, such as genetic testing or stents in neurology, we need the geneticists. We need the neurologists on board to be ready. This way, we can work together, putting it very simply. We need each other very much. It's a very strong interrelationship. You just talked about new technology and I think that should be a concern if you look down to the future. These technologies are bursting out like mushrooms. That's probably going to accelerate. How are you going to keep up, and I mean not just with respect to resources, in terms of people and money, but expertise as these new fields virgin? Well, that's a superb question. In many ways, it's probably the most important question. Close to 90% of our budget is spent on people. We're as good as our people. It's been estimated in the next five to ten years, and I'm sure you're aware of that, there'll be almost a complete turnover, at least a 50% of the people working for the government. So, we have the grain of the personnel. As new people are needed to replace the people who are leaving, it's very important to get the right people on board. Having been in all other areas of medicine, I realize how exciting this job is, and part of my responsibility is to share with others that this is one of the best chances for people who are in industry today, or in academics today, etc. So, it's really done in the following way. One, we have to see what our needs are. Second, in this Magnet for Excellence program that Dr. Feigl has talked about, to make our workplace in inviting one, and always to keep ahead of the curve. There's a famous saying that we quote in our circles that, I guess, is Wayne Guretsky, the hockey player, skate to where the puck will be, and that's a secret. Well, we have to skate to where the technology will be. We need the people with the expertise, not for today, but for tomorrow. What's your greatest concern as you look to the future? My greatest concern is that in our urge to get submissions through quickly, that we don't have a product that'll have the ulterior effects on individuals, or conversely, in our need to be a hundred percent safe, we don't delay products. Every day is a King Solomon-like decision to kill the baby or not, to let the submissions be approved or not, and to figure out how to weigh the pros and cons are the biggest one. So not only I, but many people in our group have some sleepless nights as we try to make those decisions. Sleepless nights society have cause for optimism? I'm very optimistic. I think we're doing a very good job. I think the numbers demonstrate that. The quality of people is excellent. You know, I've said to many of the folks who I've known over the years that, and I truly mean it, I've never found a more committed group of individuals more competent than the people that work with me, and my job is to keep up with them and give them an image of where we want to go in the future. Thanks. Let's turn now to to the subject of human factors and how that affects the product approval process. Where does human factors fit in with the total product life cycle that we're talking about all the time and with your your program of product approval? I think it's critical. You cannot look at a product by itself. How is an individual going to use that product? Is it easy for the individual to understand the instructions? Is it user-friendly to turn the various knobs and all the rest? And all the things we do in the Office of Device Evaluation relate to evaluating studies, evaluating what the companies have demonstrated in the field, and I cannot think of one device that doesn't deal with the human being interacting with it. So human factors are critical in what we're doing. What are your staff looking for with human factors just in a general way when they evaluate a job? I think our staff is really looking at two things. One, in the design of the device, did the company take human factors into account? And second, in their presentation to us, did they underscore that they looked at this in preparation? And that's really what it really comes down to. Did they take it into account and they have they told us about this in their submission? Are you considering, I mean by you, I mean the Office, considering human factors more now, let's say, than you did five years ago? We are. You probably are familiar that in 1999 there was a very controversial but probably very correct report that the Institute of Medicine came out with, and they talked about errors and mistakes that healthcare workers have made in our making in this country, and tens of thousands of individuals may have died from some of these errors. Most often times there are prescriptions that are written illegibly, prescriptions that are not filled correctly, or just errors in judgment. And as a consequence, one can have the best machine, the best instrument, the best drug. But if the interaction is such that errors occur, the patient will suffer. So we're very cognizant of that. We always ask the questions, the design of the device, the training of the user, the ease of use, and the possibility of human error. And all these things are taken into account as we evaluate the safety and effectiveness of the devices. Dr. Sattlin, thanks very much. This was a real pleasure. Thank you, and I appreciate the opportunity. Okay. Okay, we're back live. And Bob, you heard your boss talk about the increased emphasis and interest in human factors. How does that translate to what goes on on the ground? I mean, what is ODE looking for? Well, what Dr. Sattlin said is absolutely correct. We're much more sensitive to how we look at the user and the use environment for products when we're reviewing the safety and effectiveness of those particular products. Well, get specific. What about for 510Ks? Have you actually changed the criteria that the reviewers use when they go through a 510K? No, for the regulatory criteria of getting a product to the market, it's still whether the product is as safe and effective as those products that are already out there on the market. And as you know, the 510K process is how most products get to the market. And last year I think we had over 4,000 that went to the market that particular way. And we're still looking at the comparative of this new product that wants to go on the market to those products that are already out there and can be linked back to those that were on the market for 1970s. So how would you expect a manufacturer, I'll say, to go about determining the adequacy of human factors engineering, say with a 510K product in this comparison? Right, I'm hoping what they're going to be doing is in their early developmental stage they're using those design controls and they're considering human factors when they're really getting into the initial stages of that device development. And if they've done that then they're going to have an easier time getting through the regulatory process because the product is going to work and work correctly. And also when they get to the market they're going to have a better product also. I'm talking about things like doing mock-ups, simulations. Sure, those are excellent ways to do that. They can go out and actually talk to the users out there who use those similar types of products and gather good information and feed that back in their own development process. Ron talked about environment and he talked about the user. It seems to me that if you have a 510K product that has been traditionally used, let's say in a hospital study or by healthcare professionals, and then you have a shift in which it's now going over the counter, maybe used at home. I would think that would be, you think about environment and users changing, that would be a signal. Uh-oh, we better look at human factors. Is that, was that the case? You're absolutely correct. When we get a device and they say they want to go into the home, even though it may be prescription, but it's going to be used in the home environment, or it's going to go over the counter, we're certainly much more sensitive into the labeling, the information that may be passed back to the user of that through the training by the physician or the healthcare professional. What about with PMAs? Anything special or any changes that you foresee? Well, we've been doing PMAs about the same all along. In a PMA, we're absolutely looking at the safety and effectiveness of that device when it's used. It's not, we're not comparing it to another product on the market although there may be a control group. And in that, they need to show how their development in the early stages in getting to clinical trials led to a safe use of that particular product that everybody can use it correctly, know how to use it, and that will result in less adverse events occurring. You know, when you talk about clinical trials, you're talking about a well-trained, well-motivated clinical investigator. You're talking about a very circumscribed group of patients with a particular disease state. Then all of a sudden this thing goes out and it's used by physicians who are not as well qualified. The number of patients goes way up. That's a concern, I would think. Absolutely. And when we see a lot of PMAs, there may be only like five investigational sites and only maybe five or ten physicians who have actually used the product. And our concern is that while they're very good at that and they've learned how to use that, when you move it out to the more general community of the physicians who are going to be using it, how is that product going to work in their hands? And there may be some additional training courses that they need to provide, or we may actually require a post-approval study to evaluate how that product ends up out there in those first ten or twenty sites. If you don't, most products do not require post-approval studies, so most of them go out there and they're used. And the question then gets to be what Dr. Feigl talked about, I think, in the tape, and that is somehow getting the information that you accumulate in the real world about how that product's being used somehow fed back into the pre-market process. Is that valid? That's right, absolutely. What we want to be able to do is get that information back. We'll be looking at the Adverse Events reports, some MDR reports that are coming into us, and our reviewers should be looking at that and trying to feed that back in. If they see a situation that they've already seen out in the real world and this new product has the same design characteristic, we're going to ask questions about that. What about the phenomenon that's sometimes called technology creep? I mean, you've got a class two device, common device, and over a period of time, the manufacturer keeps tweaking it. The model changes somewhat, and the next year changes again, it changes again. That must be a tremendous concern to you, because if they don't use human factors every time they make those changes, you've got a real problem. How do you keep track of that? Well, it is hard to keep track of that, and we're really concerned about that, because products maybe have started on the market in the 1980s, and they've made new changes, new technologies, new features, and stuff have been added. We're just hoping that as part of their design controls for these new features, that they continue to bring in those human factors things in their development and that they've looked at the whole product, not just the change they're going to make. Let's get back to the PMA situation and talk specifically about how does it look in this submission? What do you want to see in terms of human factors? Sure, as Dr. Stalin said, we're expecting that the submission or the sponsor to include in that submission information in a summary form or more details about how they considered human factors and their development of the process, and that goes all the way back to before they started the clinical trials, and they may have made and found changes that were required as a result of the clinical trials and have made those changes in the newer models, and we want to be able to see that. Now, if they went through a clinical trial and they discovered a problem and they changed it successfully, do you want to see a record of the problem and how it was changed? Because after all, you end up with a good product anyway. You still want to see that progress? Absolutely, especially for PMA products, we've seen where in the clinical trial they made one or two changes or they may have come to a point where, ooh, we need to really make this big change and they'll make these and then when they send in the application, we want to be able to see. They got to that point, they made it, there was something going on. They made the change and then they, the change fixed it. So documentation is really important then? It is absolutely critical and we want to see that. Have you seen situations in your experience where as a matter of fact using human factors and your involvement has made the product better? Sure, there was a, I think it was a monitor many years ago and it came in and it had an alarm situation that it would highlight and it had a little tiny screen and what it did was it had a code for the alarm of what was going on. Well unless you knew what the code was you wouldn't know what to do to fix the particular problem. So we talked to the manufacturer about it and they ended up putting a little placard on there that decoded those codes. The other thing was it had an alarm, an audible alarm that went off but because of the use environment which is in the operating room they found that all these alarms going off would be best if it just went silent after a while until they could get around to fix it. But they didn't have any other way to notify somebody that the alarm situation was still there. So they finally just put a light on. I thought my alarm clock was running. It doesn't come back on again. So they left it with a light so that the light would stay on it. They let people know that there was still an alarm situation with that device. Kim let's start talking about the quality system regulation and how that applies to human factors. One of the things it says in the regulation is that you have to determine the needs of the user. I mean how do you do that? What does it mean? The needs of the user. The needs of the users can be just what these gentlemen have been talking about. There is in the requirement for design validation the fact that you are actually validating against the user needs or intended uses. In the regulation human factors gets played into in several different areas. First of all we have the design inputs up front. When we go into looking at what the design of a product encompasses it's often people going out asking doctors and asking people what do you want this device to be? What do you want it to do? So someone then has to take those inputs and put it into engineering specifications. In doing that it's very important to understand in a hospital environment or in any other environment that it may be used what are some of the norms? What are some of the standards that that are expected? There's certain color codes in the hospital that mean particular things. There's certain ways that people are used to doing things. It's very important that right up front in that design input some of these human factors are considered. So when you start changing or start refining if you will these requirements into specifications then you've got an engineering task of verifying okay here's my input I want this to weigh that much I verify it and yes this device weighs that much but when we get into design validation that goes back to well have we really met the user needs and the intended users? And they were talking about challenging the design right under simulated use conditions? Right and simulated use conditions can be great in the regulation it talks about either actual or simulated use conditions. Design validation requires what we call clinical evaluation and that's a broader term than you'll see in ODE. Clinical evaluation can include clinical trials but goes into some of the things that Ron was saying looking at the history of devices especially for class 2 devices what kind of predicate devices were there what kind of problems have there been? What other things do I need to consider? For example a defibrillator if you've got a portable defibrillator that defibrillator can be used in the hospital it could be used by an EMT and an ambulance and nowadays it can be used by a fireman or a policeman or they even want to put them in stadiums and so forth. So if you looked at a simulated use condition you can't just pick one of those environments. You can't pick the hospital which would probably be your best case scenario the most highly trained and sophisticated people using that device you need to look at the range and it's also not fair just to pick the other extreme to pick maybe the uneducated as far as EMT skills from a lay person's point of view because there are other situations in the hospital that you may need to consider. So whether you use actual or simulated use conditions you have to make sure you consider all the user needs and intended uses and don't just be careful not to pick one and narrow it down to one. When the FDA goes in and inspects a manufacturer where do they expect to see all this documentation you're talking about? When the inspector goes in the first place they're going to look is the design history file and when we talk about design input or design validation you're not going to necessarily have one little neat file that says okay human factors there it is human factors considerations are going to happen throughout the life cycle of the design I think the design input is a very key one. I mean if they don't consider up front some of the things that Bob was saying when you get into clinical trials you start seeing those things manifest and now you've got a lot of money invested a lot of time invested you've got patients invested into this so if they can eliminate some of those potential errors early up front in the early design phases that's optimal but like I think Dr. Statlin said you can never you can never guess everything so hopefully you do you do run your clinical evaluations or your studies throughout the design to make sure you try to get the broadest now the other place that you're going to see some of human factors influences in the corrective and preventive action system a couple people have talked about that here already we see a lot of that in the field the field really is going to see human factors in both the design as well as the corrective preventive action where the manufacturer basically is hearing of information that something's not going right and it is very easy for the manufacturers just to chalk it up and say up they're not using my device right but the quality system use error that's right that's exactly what they call it up the you it's the doctor's fault they didn't do it like we told them to in the labeling but there's a requirement both in the quality system regulation as well as in our labeling regulation that says if the manufacturers become aware of the device being misused or used off label they have a responsibility to take appropriate corrective actions whether that be additional labeling whether it be additional training whether it be safety notifications and so we will also when we go and investigate see human factors coming forward if you will in the corrective and preventive action and you have to be real careful fixing something that appears to correct one thing may induce a problem somewhere else for example we had a manufacturer who had some heating it was a home use device and the unit was overheating it reminded me of your humidity in the shower example and so they did something to open up the vents of the back of the machine so that there'd be more airflow well what was happening is the company was getting complaints if you will of toys and pennies being put in the back the vents were big enough now it's in a home use that there's children that were around they were sticking pennies and toys and so they fixed the air or they fixed the problem of having good flow or air ventilation but at the same time they opened the vents up and in a home situation it was just an inviting toy you know put something in a slot so you talked about the responsibility the manufacturers and so on but what if they don't fulfill those responsibilities I mean what action does the FDA take specifically when in fact human factors are not adequately considered well the first place the investigators are going to look is in the design and the corrective preventive action they will show up on a forty three and FDA forty three which is the list of observations at the inspector issues at the end of the inspection and often those take the manifestation of inadequate procedures to consider human factors or inadequate procedures to consider the user needs and intended use whether that be at the design input stage whether that be in the evaluation of the particular design validation also they can we often see this in the corrective and preventive action system we have an example where a company was out trying to basically compete with another major competitor the device had a cinching mechanism which the first person on the market used an an o'clock wise direction so that was cinched in a o'clock wise direction for whatever reason the person trying to introduce the new device put the cinching mechanism so that you would do an in a counter o'clock wise situation what happened is when the doctors in the controlled clinical trial got there they were very educated they were very attuned to the labeling they were told to do it in a counter o'clock wise direction so that's what they did however when the device went to more general purpose use throughout they were finding that doctors all over the place were doing it in the o'clock wise position for two reasons number one the competitors device which had already been out there did that so they were used to doing it and second it was the old adage lefty loosey righty tidy I mean and so these are simple things that you would think from a human factors perspective should have been considered prior to that that being marked I got some good news in my ear here and they're telling me that that questions are piling up we've got a lot of people out there and I'm delighted that you're doing that keep those questions coming in and we are going to have time at the end to do that but that means that we have to keep our comments here fairly brief so that we do have enough time at the end okay let me go back Ron and talk to you a little bit about what you guys do because Kim and Bob have been talking essentially about a regulatory function on the part of the FDA pre-market and essentially post-market but what you do in your field in the FDA really is not as much regulatory as it is consultative and you work with the rest of the center how do you do that? yes okay we do that in several ways and I'll try to keep this brief one thing we do is provide training to the center in fact we've got training coming up next week on the 26th but we also work with the office of device evaluation Bob's group we look at pre-market submissions of new devices and particularly where use issues are thought to be important and of concern we work with Kim's group the office of compliance and I've had some experience in working with health hazard valuations where devices that are out in the field are running into problems there's injuries deaths that sort of thing and we determined the nature of that whether we can what should be done about it on occasion we can work with the manufacturer to fix those but that's not always the case we work with the field inspection team do you let on do inspections with them? yes occasionally we don't do a lot of that but we are doing more of that and we look at what is how human factors is represented in the device design history file basically if it's been done adequately we look at device inputs and outputs and all that stuff we also look at the corrective and preventative action programs or the CAPA programs to see whether complaints to the manufacturer about their devices have been handled adequately particularly from a use perspective and simultaneously while we're on those inspections doing that we are training the field inspectors themselves who often don't have a real good understanding of human factors or how to look for it or how to apply it in that way that's what you do for the center but now beyond that and what's probably more interesting to this audience is that you actually provide some help to the industry too you have some recent written guidance in fact right? yes we've we've actually written several guidances and we're going to post the names of those up on the screen at the break the most recent one medical device use safety incorporating human factors engineering into risk management that's the title that's the title yes and I've finally memorized it and that came out in June of last year and it provides general guidance on understanding use error and methodologies for applying human factors and it also represents some degree of philosophical shift by both the agency and the medical device manufacturers in that it describes how hazards or potential hazards due to the use of a device should be included in the risk analysis process as any other hazard would that this use is another type of potential hazard not only that but it goes beyond that and says as far as the strategies for mitigation for hazards that are identified you know it's not always helpful just to state that one exists but that when one does is stated for a given use related hazard that it should be demonstrated that that mitigation strategy is effective give some general advice if you would for if I'm a company that I haven't been doing human factor very much what advice would you have in a general way based on your experience well it's it's makes very good sense for a company to get into human factors and to have a human factors capability either in house or being able to being equated with a consultant that can do that for them you've made the statement several times that it's not as difficult as it seems what do you mean by that well a lot of people are afraid of human factors because it seems it's kind of a an odd thing for us for people to swallow and companies people have talked to me confidentially and said that well they're a little reluctant or their management is a little reluctant to get into human factors because they think anytime they have to do that or if they do some sort of human factors analysis that they have to do an elaborate experiment and they have to have a large in size and they have to have a lot of clinical trials where you do have to have big numbers big N numbers statistical significance and what you're saying here is that that may not be the case always in this right human factors can be incorporated into clinical trials with the big N sizes and that's fine but when you're doing applying human factors upstream in the design process those types of analyses don't necessarily have to have large Ns you don't have to get into parametric statistics you know just a little bit of descriptive statistics is fine but you can learn a lot of information that way now in addition to giving written guidance you will actually consult with and help companies directly is that right that's true yes we do that and we we encourage manufacturers to call us to get in contact with us I believe we're going to have our phone number and email on the screen at some point here we go we're going to have your we're going to have my home number yes your home number on there and you will you will do 24 seven service for people exactly exactly yeah right and Mark just kidding Ron's group does a lot of international standards work too right that's correct yeah we participate on both national and international standard setting committees with respect to human factors standards before we close this panel I want you to see a videotape and then maybe comment on it Dr. Matt Wenger is is professor of anesthesiology at the University of California San Diego School of Medicine he's also a staff physician at the San Diego VA Medical Center Dr. Wenger has a special interest in human factors engineering as it applies to patient safety in a hospital environment and so we asked him to talk about the relationship between the design of the device the clinical user and the safety of the patient Dr. Wenger agreed to be interviewed in his hospital and to show us how human factors engineering affects the safety of some of the devices that are used in surgery let's watch in the clinical environment one must consider three critical factors with regard to usability the first is that devices are going to be used by a wide range of users and that diversity of users includes not just differences in training and experience but a variety of other factors such as sleep deprivation distractions workload the second issue is that there's a wide range of use environments for a single device from a patient using the device at home to clinicians using the device in a complex operating room procedure the final factor is will the devices user interface effectively support what the clinician needs to do at the time they need to do it the operating room is a good place to see some of these usability issues in action this is called a pressure relief valve it's used to vent excess gas in the system when it's closed the gas stays in the system when it's open it's vented there have been a number of situations where this valve was closed without the clinician appreciating it and the excess pressure in the system caused trauma to the patient's lungs the anesthesia machines have been designed with a number of safety features to prevent use air for example unintentional movement of the knobs that control the gas flow is protected by this bar the different knobs have different shapes to help someone identify oxygen from air in addition one cannot turn the oxygen completely off there's always some residual oxygen to deliver to the patient on some machines you cannot deliver too much nitrous oxide without the oxygen rising to match it and one can see here that there an alarm is going off from the excess of nitrous oxide this device has a variety of usability issues related to the fact that it's so complex there are so many different ways that one can manipulate the controls in order to modify the display it's a full keyboard as well as variety of other knobs and controls and it can be very time consuming in the operating room to use when the clinician has many other clinical responsibilities this is actually an example of a very well designed device and that it's intuitive that if you squeeze the bag you're delivering positive pressure the space oxygen to the patient it's called an intuitive mapping which is in contrast to many other types of device design where the relationship between the user action and the device's response is not intuitive and sometimes it's counter-intuitive when a device is designed in isolation where the user's needs and requirements haven't been adequately considered that device is potentially unsafe usability affects safety and if I can't use a device in a crisis situation effectively then I'm not able to respond to the patient's needs there have been a number of situations where drugs have been administered in the intravenous line but never got to the patient because stopcock was in the wrong orientation or they injected into the wrong IV tubing and didn't have it running the use of the devices the manufacturer often considers it for a specific indication but once it's out there in the in the world the indications often expand so that the user population and environment of a device's use typically expands significantly after it's released one of the most important problems we have in the operating room is delivering sterile supplies to the field and therefore packaging becomes a very significant problem when the design is such that we have difficulty opening the the product in a sterile fashion one of the problems is a packaging like this where you you have to pull it open like this and then you're supposed to drop it onto the sterile table like this this is very substandard it makes it very difficult to dispense the item onto the table without contaminating it or this one there is no method of opening it at all that will allow for a sterile delivery product labeling remains a problem many drugs are packaged in ways that makes it difficult to distinguish them from each other for example here are two drugs with blue tops that are very different drugs yet appear somewhat similar and the labels are very small so in a in a hurry drug that's put in the wrong place could be grabbed and given to the patient and this happens very frequently intravenous infusion pumps raise a number of usability issues here in this patient after heart surgery is on 10 different infusions through six different pumps I figured out which drug is being administered to through which infusion pump can be difficult in addition to issues with regard to patient safety these heavy pumps on the tall IV poles can tip over and injure care providers when a device's user interface design is flawed it can promote use air in addition users may have problem recovering from errors and finally in a crisis or unusual situation the device's use may in fact make the situation worse a real problem in the operating room is the large number of cables and wires that are not only a tripping hazard it's not uncommon to forget to plug something in and the battery runs out or to plug in the wrong thing to attach to the wrong device to the wrong cable and these are obviously clear usability issues the position of devices in the clinical work environment can result in problems particularly with monitoring in this case an electronic record keeping system located optimally for use it blocks the vision of the clinician of a number of the other essential monitors and can often be in the way when the clinician is trying to interact with those other devices one of the issues you can see very clearly here is they're they're managing the patient's airway but they're having to turn you know 120 degrees behind them to see the monitors many hospitals are trying to go to a 100% electronic or computerized record however there are a number of usability problems and there's much more difficult in a computer to enter data that's very easy to do by hand the increased emphasis on electronic record keeping has required clinicians to learn how to type or to type more effectively in order to enter the data rapidly without distraction from other clinical tasks effective device design will allow a clinician to use a device correctly the very first time without the need for training or referring to manuals clinicians will however become more effective at using the device with additional training or experience one of the issues with surgical devices is that many of them are designed in a way that it's more difficult for the surgeon to do what they want to do because of awkward hand positions awkward body positions and this can sometimes be frustrating and even affect the quality of the surgical repair an effective design will incorporate users in the design process from the very beginning and will consider carefully how a device will be used in the wide range of environments that it might be used in and the different types of users there is a tremendous body of knowledge about human factors usability and interface design outside the medical domain and I would encourage both industry and the government to draw on that expertise in their efforts to make medical devices more usable I want to ask our panelists now for just a quick reaction to to that tape, Ron, before I ask you though I have to say that what amazed me is watching that thing was the comment of the nurse that she couldn't get the product out of the bag without violating sterility which is going to be so simple it's remarkable, impossible not to commit a user in that case what did you think in general about the video? I thought the video was great that example and others very good insight and a window on human factors in the hospital operating room one thing we need to remember is that that environment is a subset of all the environments and those devices are a subset of all the devices you also have different specialty areas in the hospital, of course you have doctors offices and of course, as I said, nursing homes and the home users and different devices throughout so we have to remember that this is a snapshot of a larger spectrum of device use and use environments Kim? From a compliance standpoint you saw the two blinds that were hanging down one time a manufacturer came in and as a proposed fix to a user error, as they called it because the doctors and nurses were clamping the line they wanted to put a label on one of the lines and say, do not clamp what do you think the probability of those nurses and doctors with all those lines A, seeing it with all those invenous lines hanging down and then following the directions so when compliance sees these types of corrective actions and say, no, this isn't going to work and the company doesn't understand well, why not? This is a good example as to why it would not work in the actual user setting Bob? I thought it was a great educational video because when you're looking in the operating room you see all these different devices up and around and all these users that are trying to interact with them yet when we actually evaluate devices we're only looking at that device one at a time one at a time and so I think that was real educational for me to own that one Thanks, thanks for a very good discussion It's time now to take a 10 minute break and during that break you're going to see on your screen some important addresses and phone numbers and websites where you can get more information about human factors and engineering and how it affects medical devices and we're also going to include some information on how to purchase a tape of this program also we're going to give you a website where you can get a schedule of upcoming teleconferences from our FDA studio so you may want to jot all that information down Anyway, we'll see you back here in 10 minutes OK, we're back live and ready to introduce our second panel and this time we're going to be talking to non-FDA folks who are experts in various non-regulatory aspects of human factors we're going to cover how human factors can be incorporated into the design and development of medical devices some of these panelists are employed by device manufacturers or consulting firms but today they're not representing those organizations they're here simply as individual experts in this field when we finish this panel we're going to call back all the panelists and answer your phone and fax and emailed questions Our first panelist is Dr. Pamela Jumar a technical fellow and senior human factors scientist at Medtronic in Minneapolis she has over 20 years of experience in designing interfaces between humans and computers for medical, commercial, space and defense systems at Medtronic she's led the introduction of human factors engineering into product development and her current work is focused on the instruments that physicians and patients use to interact with pacemakers and defibrillators Dr. John Senders is a consultant in human factors and human error and he teaches intellectual property law and cognitive science at the law school of York University in Toronto he's worked in the field of human factors since 1950 at the U.S. Air Force Aeromedical Laboratory at Bolt-Berenek and Newman at the University of Toronto and at the University of Maine Timothy Badaki is a consultant with Hazard Management Associates he's worked with corporate clients who manufacture pharmaceuticals and medical devices advising them on product safety programs loss and liability prevention and risk management he also trains staff in these areas and he's been an expert witness in legal cases involving human factors warnings and usability engineering Michael Wicklund is vice president and director of the New England Research Center of the American Institutes for Research where he consults with medical device manufacturers on a wide range of design and evaluation issues he's one of the lead authors of the Human Factors Engineering Process Guide under development by the Association for the Advancement of Medical Instrumentation he's also an adjunct associate professor in the Human Factors program at Tufts University Dr. Mark Brecht is founder of Brecon Consulting and a senior partner at Human Factors Plus where he consults on usability and interface design as a specialist in human behavior Mark works with designers, engineers and systems managers to ensure that products and services are compatible with the needs and capabilities of the people who use them he's recently helped clients set up programs to incorporate human factors education, design and testing into their product development cycle and with that introduction Mark let me start with you and let's say that I'm in a device manufacturing firm I've not done human factors things before very much how do I get started in terms of people facilities and so on well I think one of the important things to do is to consider how you're going to acquire those resources that you'll be needing and those resources will vary depending on the stage of development of the product that you're working on the types of products you're doing and everything else obviously for a company that's never had it before their intent their purpose and need is to really get an internal advocate somebody that's going to be supportive of what needs to be done and can push that through at all levels from engineering to management in my work I've often worked very closely with industrial engineers and industrial designers those are the people that typically recognize the need for human factors to support their design work for instance but may not have either the mandate or the resources to do that themselves and they'll go to the outside you can go out to the outside if you don't have anything current through consultants perhaps bring in someone that can temporarily provide that for you you can start having that consultant maybe work with your internal people to start educating and training internal designers at some point you may want to consider bringing in your own human factors people maybe start with junior human factors folks that that consultant can work with on a liaison or a mentoring basis even to build it and eventually you're going to work to the point of having an internal staff of some sort what about facilities I mean how complex does it have to be I'm thinking now about one-way mirrors and that sort of thing well that's the model that everybody always talks about is having the observation room for doing usability studies you've got very expensive video equipment elaborate one-way mirrors and lighting and all that kind and obviously that's great and it's good for a lot of kinds of testing that might get done but you can learn an awful lot with a lot less than that I for instance have worked to gather data about how to design a product and to do usability testing everything from walking down a hallway and talking to somebody in their office to in the medical environment frequently going out to hospital settings meeting with nursing staff in a break room for instance showing them a new product you can go out to medical professional conferences and recruit people off the floor of the conference to come up to a hotel suite and use a new product all those are possibilities you talked about getting a champion within the organization I don't want to have other folks comment on it as well where does the real responsibility lie in the firm for one thing and does it start top down or bottom up? Do you need, how much do you need upper management buy in before you can get anywhere? Anybody want to comment on that? Well I personally think that you have to have receptive management and that means that management has to submit even if only to a little education in order to know what it is that they're supporting. What about, go ahead One of the things I absolutely agree, John, with that that there needs to be the management commitment but I found that at least recently there's been a lot of working from the bottom up that as the FDA has done their job of communicating the need for human factors a lot of the program people the design level people are starting to pay attention and ask what do we need to do and how do we go about doing it and pushing that up through their management so that things like budgets and schedules can be adapted to meet that. Mike? Well I think that one of the keys to success when you think about companies that are going to embrace human factors for the first time is that the people doing the hands-on work whether it be in-house people or whether it be consultants have a win have a get a really good running start at demonstrating the effectiveness of human factors what it can contribute in a practical way to coming up with products that are safer but also products that are going to be more competitive in the marketplace in other words have a good first win. Yes. Mark I think all of those things are very important and in addition I believe that you have to develop your quality system your process to include human factors from the get-go rather than trying to superimpose another component but to build it in into the process that you're using for new product and existing product design review. Pamela several, I'm sorry go ahead. I was just going to comment on this one. In that I think that's one of the things that I've seen is that we've started working with companies now to take their existing design process and identify the places where human factors should go in according to the FDA's commitment to that and actually getting them incorporated and that starts pushing upward because now the requirement is there internally as part of their procedure and they start saying now how do we meet that? What does it mean to do usability testing? What does it mean to have a risk analysis done from a human factors perspective for instance? I think it's very important because the management commitment most often will be there but management may not necessarily know what it is and it may be to the other quality engineers to put it in place. Pamela, we've heard several people talk about determining user needs. How do you go about doing that? What are user needs? From the standpoint of human factors engineering not from the standpoint of what else they may want to have on the device. I think it's important that human factors engineering is part of the project team right from the beginning because marketing will be identifying user needs but a lot of times those are a list of features, features that the customers have asked for, features that the competition has in their product, features that are responses to problems that have occurred in the field. Human factors engineering is gonna look a little differently. They're going to be looking at the user need as a safe usable product and they're gonna derive a different set of design inputs and user requirements. They're gonna do that by really looking at the end users, who are they? What are their capabilities and limitations? What are the tasks they're gonna perform using the product and what's the use environment? So for example, let's say that this is a product that's destined for in-home use by elderly patients. Human factors engineering is gonna drive a set of requirements like the fonts need to be larger, they need to be on a high contrast background. We're gonna avoid colored fonts. Any auditory signals are gonna be in a certain frequency range and these users need to have a volume control. Different set of requirements. You're not necessarily starting from scratch. You may have a product that you already manufactured that's somewhat similar. You can use that maybe or somebody that somebody else makes. Right. The other thing the human factors engineering person is gonna do is actually go out and study use of the existing product, similar products, concepts, prototypes you have. They're gonna do things like usability testing, cognitive task analysis and you're gonna learn a lot from that. You're gonna discover problems you didn't know about but you're also gonna discover opportunities to make a great product. What about having marketing do this or how the clinical people do it? Is that fraught with problems or can you do that? Well, I think it's two different sets of skills, two different kinds of questions. Human factors engineering person is looking at it in terms of a human being with capabilities and limitations interacting with equipment. They're looking at those kinds of questions. What are the users doing and what's in use environment? Is there noise and what is the lighting? Marketing person goes in that same environment and they're looking at issues of what are the features, what's the curb appeal, what are the messages? One needs to get the messages. You've used the term the preference trap. What do you mean by that? Well, they're gonna be looking for kind of curb appeal and messages that the customers wanna hear and they're gonna look at price points. You need both to do a product but it's complimentary skills. Yeah, let me interrupt you. Just we've got a lot of questions now. We want even more. So keep those questions coming in, phone them, fax them, email them and a little later on we're gonna be answering those. Anyway, you're talking about the ability to use the device. I mean, that is important as well, right? That's key. Yeah, okay. Mike, let's talk about usability testing. How do you go about setting that up? Mike, there you are. Yeah, well, I think usability testing comes in many different flavors and depending on how you're approaching things, you might set things up differently whether you're using consultants or whether you're doing an in-house effort. But I think there are some common things about any usability test that you would set up. principally is you've got to decide how rigorously you're going to approach testing. I liken usability testing to shampooing your hair in the sense that when you add shampoo to your hair you're cleansing out all the bad stuff in your hair and people who are very thorough do it twice to get it extra clean. When we do usability testing we're in essence trying to cleanse the flaws from a device and accordingly you might get most of the flaws out of the first test. But you'll want to come back later when you have a more refined design that's been improved based on the results of the first test and test again. What I recommend to most of my clients is they do it three times. One, to evaluate early concepts. One, to evaluate a detailed design that's still preliminary. And one, that's almost complete and might serve as a validation test of salt. You can't give an absolute number but what about sample size? Just roughly in most cases how many people you're talking about when you do this? Well, that touches off a debate among human factors people. Someone who believed that- Maybe even at this table. Very surely at this table. I think it makes sense to run relatively small samples and do it more often. So I would typically recommend that you work with a sample size of say six, eight, 10 people. Make sure they're representative users that will give you a fairly broad view of different experiences people might have with the product from first use experience. People are sometimes accustomed to doing clinical trials where it is in six of eight or 10, it's 200 or 1,000. And when you have that many people, if you make a few mistakes in choosing your sample, it's okay. But if you're dealing with eight or 10 or 12 people, you really, it seems to me, can't make mistakes in terms of the kinds of people you choose. How do you choose them? Who do you want to get and how do you already prevent getting people that are not representative? Well, I think you can put a lot of thought into how you choose participants in terms of their background experiences, the degree of training they've received, their various use environments and so forth. I think you want to do a good job in coming up with a good screener. But I would caution about trying to be too precise about the way you get participants for a test. I think the key thing is to be doing testing and there are a lot of things you can't control. For example, if you were going to try to get people who were arguably less bright at using a device because you wanted to test the device on a range of cognitive abilities, that's really a tough thing to recruit for if you call up a hospital, say you want the number of nurses to come by within the next week to interact with your product. It'd be very difficult to control for that kind of thing. In my experience watching hundreds of test sessions, I think these things come out. If you make a commitment to getting what might be considered a reasonable sample of people, you'll learn a lot. I'd just like to comment on that because from a practical point of view, I think it's important that not only the manufacturers understand, but the FDA understand that you really don't have the control and latitude that you'd like to have in recruiting. When folks were in graduate school and doing nice controlled studies, it was easy to recruit. But in the real world, you can't do that. You're lucky if you can even get to nurses. They're busy people and it's tough to get those people to give you even 20 minutes of time. So you take who you can get and you learn as much as you possibly can from that. But you aren't always in as much control of who and how you're going to recruit as you might. Well, talking about difficulty in recruiting, this might be a good time to ask about simulation as a kind of substitute. What do you think about that? It's a really valuable tool and one we use all the time. Everything from very simple drawings that we're telling people how it will work to increasingly doing PC based simulations that recreate what the thing is going to look like and sort of how it's going to work long before we can have an actual working prototype. And that's very effective. Pamela? Yeah, today with things like laptop computers, you can take a simulation of something out in the field and get feedback very early in the process. And that's the name of the game is getting it in front of users and having them try it early in the process. And we were talking about numbers of users. Research shows with a handful of representative end users you can find key usability problems. And even if you only have a handful, that is much better than a couple of engineers on their own trying to speculate about what real users would do. Which leads to the question of do you really want to use the engineering staff or the developmental staff as your subjects and you probably don't. You absolutely not. I think that the difficulty with the design people is that they know altogether too much about how something's supposed to work. And as a result, they can't imagine what kinds of errors people are going to make in using it. And what you need are people who are skilled mischief makers, people who can imagine things that the designer would never have thought of and find out whether or not the device becomes dangerous as a result of all these inputs that were not planned. While we're talking about things not to do, I think probably we're agreed that marketing research is not a good source of information for human factors as well. Because it's different. It's totally unrelated. It's not an adequate source. It's not an adequate source. It's one source of information you should consider. One of the things when people do clinical trials, they'll do it on a large number of people and they'll say well, 2% of the population had a particular adverse event and that's written off and that's the end of it. I mean, it counts as the numbers. But in terms of human factors research, are you guys interested in, let's say you tested in 12 people and two of them couldn't use a device. They really screwed it up. Do you want to know why they did? Do you want to go back and investigate that? The issue is one of the investigation of error and it used to be that what you did is count the number of errors, which is what you just did. But in reality, from the point of view of design, you're not interested just in reducing the frequency of errors. You're interested in reducing numbers of errors of particular kinds, particularly the ones that might injure or kill a patient. So you're very much interested in the content, the actual behavior and not really very much in how many of them there were. Tim, let me change the topic now. Let's say your device is now already on the market. You've done all this and yet you are having problems. You're getting complaints, you're getting reports of the FDA and so on. How do you work that into the human factors spectrum of activities? Well, Mark, I think as Dr. Feigl said in the earlier comments, it's really not user error. What is important I think for designers to keep in mind and to remember is that accident prevention is an engineering problem. Human error is a given and it's not an explanation. And we have to investigate those things in order to determine what in legal circles would be called the foreseeable uses and foreseeable risks with the product. I think a great deal can be gained from all of this that we discussed from the medical device reports as well as the information in human factors testing because ideally what we want to try to do is to design safe products. In order to design safe products, we have to first identify potential hazards and risks and then design them out. And the key items there are what is foreseeable and what technology is available to help us design out those risks and hazards. You're talking about designing them out but now we have our product already on the market, already being sold, already being used. Sometimes people say, well, instead of designing them out or changing designs, let's do another instruction manual or let's do another label. And sometimes you get the feeling and maybe this is overstating it, that's kind of a mandate. It's kind of the easiest way out. Is that fair or not fair? Well, I think it's probably the cheapest way out in a way that most people frequently will look at the problem. But if you're having particular types of errors, I think you need to evaluate what the degree of risk is first and then based on the degree of risk, apply the approach. That might be modifying instructions, might be implementing a new warning, but frequently those are the last choices in kind of a hierarchy of risk management. I think it should be recognized that if you're doing a really good job of usability testing, the kinds of things that will emerge later and that might require a band-aid are probably going to be caught or a large majority of them are going to be caught. I think practically speaking, if you're seeing problems in the field, it's going to be a long period of time before a manufacturer is going to be able to address the problem with any kind of engineering design change. I mean, you might be talking about years at least, right? Certainly not something that will be solved in a matter of weeks or a month or two. So really you're limited to band-aids or you're limited to taking the product out of the hands of the users until it can be replaced. So I think it speaks again to the value of usability testing. You know, it's a really good filter. One thing I might add there is, I think there's an important need to track what you learn. You've got a product in the field that is having problems. So you're trying to do your best to fix that, but you need to take that and move that into the next project's design history. I'm glad you said that because that gets into that total product lifecycle we talked about, getting that feedback through to the next one you designed. Well, that's particularly important because in almost every complex human enterprise, it is estimated that somewhere between 70 and 90% of the adverse outcomes and adverse events are due to human behavior of one sort or another, usually called human errors. And yet we spend much more engineering effort on increasing the reliability of the, let's average it and say 20%, which is the machine problem, and practically none at all on the 80% which is due to people. And even if we were to correct all of the possible failures of the machine, we'd still be left with 80% of the failures because we're aiming at the wrong thing. Talking about people, there's an element in the culture of the hospital that may be working against us. I'd like your view on that. And that is that hospital workers, nurses, biomedical engineers take pride in working around a problem, that is in overcoming it in their day-to-day work. They consider that part of the job. And yet what really should be done is instead of tolerating and working through, letting the manufacturer know that they're having a problem. What about that? Mark, a lot of nurses are like Marines. They're taught to overcome circumstances no matter what they are. And they will gerry-rig things and make things fit that aren't really meant to fit together. I think it's real important that in this process that the manufacturers begin to solicit and that the hospital clinical engineers and biomedical engineers begin to become educated in first in human factors and usability engineering and the value that it can have to manufacturers. Those folks inside the hospitals are in a particularly good position to be on the hospital's equipment or technology assessment or buying groups. And it's there that I believe they can influence the purchase of equipment by asking simple questions like how has usability engineering been performed with this product? I think that would be a tremendous help. I've met with groups of nurses and doctors for many years now and I can look back a decade ago. I definitely had a sense of that can-do attitude coming from nurses, more so than doctors actually. But I would say there's a little bit of a trend I've noticed recently which is that people are becoming a little bit more aware of good design. They're seeing lots of exemplary products out there and I think they're getting more demanding. So in a focus group where you would be exploring people's vision and needs regarding a new product, you'll hear people now going as far as to say, well, if it's not working for me, forget it. I mean, get it out of my hospital. And I think that's a little bit of a sea change. That's interesting. Pamela, one of the things that people may say when they're new to this, well, can I afford it? How much is it gonna cost me? Can I really afford to do this? How would you answer that? I hear that question a lot. And of course that's gonna depend. It depends on the device and the complexity, how far along it is it in the development cycle, how many different kinds of users, how many use environments. But here's what I can tell you. You're already spending the money. I would tell the manufacturer, you're already spending the money. You're spending the money potentially in finding problems late during a clinical when it's very costly to fix them. You're spending the money possibly in the lost sales because the competitor has a product that the nurses like better. You're spending the money having meeting after meeting and focus group after focus group with engineers and marketing and clinicians arguing about a design feature where the question might have been better resolved using something like usability testing. Much better to spend the money upfront and end up with a better product and a safer product. Let's start answering some of the questions that you've been sending in. And by the way, we've got a lot of good questions here. They're all in writing, they're faxes and they're emails. We'd like to hear from you on the phone too. So pick up the phone and call us. One of the advantages of doing that is that I will give you the chance to ask a follow up question. Call us and we'll answer and then I'll say, does that answer your question? And you can say no, it doesn't and ask it again. So we wanna hear from you on the phone as well as on paper. But anyway, keep both of them coming. I've got a pile of them here. So let's begin and this one says, and by the way, both paddles are now back. We do have the FDA folks on another set so that we can all pitch in here. It says, what types of tests are recommended for evaluating device instructions? What kind of documentation will an FDA device evaluator be expecting? Anybody on the FDA panel wanna? Well, I'll try to answer that. When we're looking at use instructions, we're just gonna look at readability and unless the manufacturer actually provides some sort of a test of the results that he's actually done a mock up on that, we won't know about it. Okay, anyone else wanna add to that? I think it'd be important that the manufacturers go through a task analysis of sorts in the development of the product. It's frequently found that a lot of folks will design the instructions after the product is built and you wanna design them as the product is evolving and I think that's where you conduct a task analysis to get the proper sequence of steps in the use. Okay, there is a significant literature on the design and evaluation of instruction sets. You find it in the human factors literature and in the applied psychology literature. I don't have them at my fingertips but I could certainly provide them to you when I get back. I don't particularly care for it when folks dismiss the value of instruction but I hear the voice of Matt Weinger who said, we don't wanna be instructed, we don't wanna read this, we wanna walk up to it, we wanna use it intuitively. So I think it's important to get instructions right and I think they can serve a value for certain people with certain learning styles but I think the prevalent learning style and one we'll see becoming even more prevalent is that people want that walk up intuitiveness. Okay. I'm not sure you can get it. A bread knife is almost intuitive but I've seen people try to misuse it in cutting bread and a hammer is almost intuitive but the things we're talking about don't come anywhere near that. Most of them need some training, a great deal of experience, possibly an advisor while somebody is using it in the first instance and a reference book to find out what to do in strange circumstances. Well, in an eye, that's the labor of the issue. We okay, because we had a lot of them coming in. Go ahead, one more comment and then we're gonna go. Just that, from a practical point of view, you mentioned that the documentation may not be available until near the end and that's the dilemma because whoever's writing the documentation has to have something to write against and the software's changing and all that. There is a real practical concern there and I think in the same way the device gets user tested there needs to be more emphasis put on user testing of that documentation as soon as it's available and on an innovative basis as well. There is, I wanna take a phone call and we've had a lot of questions. So let me take a phone call from Orange County, California. Orange County, you're on the air. Orange County. It's due, a comment made earlier from one of the FDA ladies and it was a comment to the effect of a manufacturer needing to take preventative action if misuse or optimal use is encountered on any significant level by a manufacturer. The first question would be what regulation would require that and the second one is I wanted to focus more on off-label use. What would you expect a manufacturer to do if they did encounter significant off-label use or what would you recommend that they do to try to rectify the situation or I guess make people more aware of the off-label use indication? Well, two things since I'm the only lady FDA person who has the question was to me. A 20.100, which is corrective and preventive action requires you to analyze your data and if it's coming in through complaints, service reports or any type of reporting feedback mechanism, you have an obligation to try to prevent or correct that happenings. Also, 801.5, which is the labeling regulation also has some regulatory requirements that talk about information that you gain. Once it comes to the attention of the manufacturer, there are certain responsibilities. You can't give a one-stop answer to your question. It depends on the significance of the problem. It depends on the root cause of the problem. Like some of the gentlemen or I forget who mentioned it, redesign does not happen overnight. So there may be a series of steps. In other words, you may take a first step that is quote corrective and might be a bandaid at that point in time. But it may get you or buy you some time to feed that into a redesign or perhaps a design change that could better address the question. So FDA's expectations, if you will, will depend on the significance of the problem and the strategy that the manufacturer lays out. That can be a short-term strategy as well as a long-term strategy in dealing with the issue. And oftentimes it may depend upon having more information come in and getting some more outside information as well. Collar, does that answer your question? That's great, thank you. Okay, let's go to a written one here. It says, doesn't grandfathering devices into the market permit poorly designed devices to enter the market? Should all devices be evaluated individually? I guess that sounds like an FDA question. I will try to handle that one. Well, the way our law was written in 1976 that allowed for that grandfathering clause and that law has not been changed since then. So what we want to do is if we want to move toward that then you would need to get seek a legislative change for that. This one says, does anyone else want to add to that? Okay, this one says, since the FDA doesn't actually do human factor research and evaluation, they have to rely on the manufacturer's word. Isn't there a lot of room for the manufacturer to fudge the evaluation? It couldn't be written by a manufacturer, surely. Anybody want to? I think that's a... Go ahead, that's something that's universal. The FDA doesn't test product that evaluates. No, we don't have a testing lab. We do do reviews, but of course, we can only do a limited amount of that. There's, I think there's the potential for fudged data anywhere, whether it's human factors data or any other kind of data. Of course, anybody that does that is running the risk of being caught and charged with fraud, and at least that's my understanding. Also, the people, at least in the human factors staff, do have enough understanding of human factors that we can usually evaluate the quality of the human factors content of a submission. So I agree that that potential exists, but it's one of the challenges we have to try to make sure that that data is good and accurate and appropriate. But I do have to add, we do have some testing capability. I mean, we have our Office of Science and Technology, but that is used more on global issues as compared to individual manufacturer application-type issues. They're global issues, maybe for a particular brand or group of products that we have. We have enough... Kim, what I was... No, I'm sorry, go ahead. What I meant by that we don't have testing capability, we don't have a human factors lab, not yet anyway. Where I have worked in facilities where we did have a human factors testing lab and we would bring people in and we would do testing as yet we don't have that. But we do do other types of testing. Okay, let's go to another phone call. Again, it's Orange County, California. You're on the air, Orange County. Yes, to some extent the definition of good human factors design is subjective. How do we as manufacturers reconcile this subjectivity? Well, I think that's been a concern to the FDA in terms of based on my assessment of things they've said in various standards meetings that I've participated in. I represent the Amy Committee Association for the Advancement of Medical Instrumentation and Pam does as well. And we've been working on a process guide that Amy will be publishing in a relatively near term. In fact, Matt Wanger is the co-chair of that committee. And that will be a guide that tries to lay out in a process oriented sense what it is to do good human factors. And what it is to do good human factors is to lay out a program of activities that will systematically help you identify user needs, translate those into product requirements, come up with good designs and evaluate those designs. So I believe the FDA, as far as I can tell, is really committed to that kind of guidance and will probably be endorsing that standard when it comes out. Anyone else? Yeah, I think that's a good answer. There's a human factors engineering process and do you have one in place and are you following it? That's kind of a high level measure of good human factors. But I would also say if you're talking about looking at a device and just saying does it have good human factors? As part of your design process, you should upfront define some concrete, measurable, testable usability specifications. In other words, you have specifications for reliability and maintainability. You should have specifications for usability. What is it that makes this system pass or fail in terms of usability? So for example, I might write one up upfront that says a nurse of this type with this kind of experience and who's used this kind of equipment and familiar with this task can take it and perform these tasks with no errors or in a certain amount of time. Whatever is really critical to that product. But you can define some usability specifications and make that part of the product requirements. And as you design and develop that, you're gonna be testing against those specifications. Mark, you want to answer? No, that's... You had your hand up but it was gonna say the same thing. My point exactly. Yeah, okay, anyone else? Yes, just adding to that. Then of course there are a bunch of things we know that represent good design. We know how far apart buttons should be so that you don't accidentally strike one versus another. We know what size font should be on a screen. We know that certain color codes are representative of certain kinds of actions. And so just in terms of an inspection of a design you can go through and you can determine whether something is ascribing to good human factors, practice as described in textbooks and in very detailed standards. I just thought of something. The guy who designed my car radio should be watching this so I was talking about little buttons that you can't figure out which one is which while you're driving, that's great. Here's another fact. As to the best of my knowledge, the product is not submitted along with the submission. How can the FDA evaluate human factors by reviewing directions for use, labels, and so on and not the actual device? It's like reading an automobile user manual rather than taking a test drive. Right, I think that defaults to me. Of course, we have a memorandum of agreement with the Office of Device Evaluation for doing reviews of what we call patient labeling, which also includes user manuals. When we do that to the extent possible, assuming that the device doesn't weigh hundreds of pounds, we try to get a sample device if possible. When we don't, there's typically pictures of the device or drawings of the device in the user manual that we can refer to. And after a while, a lot of it is just experience. Admittedly, it's not a perfect situation for doing that evaluation. And again, what we're trying to do is do the best we can with what we have and keep the process moving and give feedback to the extent that we can that will be helpful to making that information more usable for the users. And a lot of times we recommend to the manufacturer when we review that material that they do test the content of the user instructions with a representative group of users to see if they can understand it and see if they have any problems with it. That's sort of a blanket recommendation we have. Occasionally, we see that that's been done. We'd like to see more of that, of course. Now I've got a comment that if the device is actually exists in a physical state, one of the field inspection team of FDA, if trained in human factors, could go in and do a preliminary evaluation, at least on the major aspects of how well it's designed because there, the machine exists. This is an interesting one. I'm not sure that I understand it, but it says how should a manufacturer document that they considered a user need but did not address it? I imagine that means that that was considered and it was okay and therefore how do you document that? Yeah. Well, from my experience in conducting usability, we frequently get into that situation where there's some issue that comes up. It'll be noted, it'll be discussed among the team members and then for a variety of reasons, maybe it's not critical enough or significant enough, possibly it's too late in the design cycle to do something about it, but it is at least noted and the remedial action that's taken, either we're gonna fix it, we're not gonna fix it for the following reasons or we're gonna defer that for a future generation of product is noted and that becomes part of both the usability test reporting and ultimately the design history file. So in my experience, that's one way that it can be. Do you have a comment on that? Oftentimes the inspectors can also see it coming out through their risk management program through some sort of risk analysis where they may acknowledge the fact that there is an issue and they may choose to not do anything but they can offer the fact that they may be mitigating it through other means as well. That's where we often see it as somewhere in the risk management. Yes, I think that's a critical point. Again, I don't think you wanna try to superimpose a human factors approach on a company that integrated into existing systems and processes such as the risk analysis that is being performed by companies for a number of reasons. Human factors should be a portion of that. Okay, I have a, go ahead. I was gonna say there's a lot more here I wanna get to, so make the comments briefs and we'll try to do as many as we can, yes. Well, I was just going to say something sort of opposed to the philosophy of good human factors engineering, which is you often hear the expression that the customer is always right and that tends to be applied to sales. I don't really think that's the case when it comes to human factors. I think the customer's needs need to be viewed in balance with the business objectives and the engineering feasibility of things and good human factors engineering is finding a balance point between those but making sure that you don't make a decision and present a design that is unsafe. But when it comes to more preferential matters, users can tell you what they want and you may have very good reasons, rational reasons not to go there. You know, well, you talked about that earlier about the preference trap. I mean, what they want is not always best for safety. I think if you design products according to what people were telling you in the focus groups where they're not embracing the engineering limitations and so forth, you would get products that were caricatures of the products that they really want. You really have to apply the judgment because the users aren't designers. They're just telling you what their needs are and you have to interpret them. Well, here's another one. It says, since the vast majority of devices are approved by the 510K process which requires a predicate device and a demonstration of substantial equivalence, how would a manufacturer with novel human factors features get a device approved without going through the more costly PMA process? In this sense, the 510K process seems to hamstring innovation. Okay, well, I'll try to take that answer. What we're looking at in a 510K is really the substantial equivalence of this new product to the predicate and we do see a lot of technological changes that occur and that move along as technology and new things become to the market. So it is possible to do that. What we're looking for that pulls it into the PMA realm is where you're changing the intended use or its performance is not as good as what's already out there on the market. We have a phone call coming in now from San Jose, California. San Jose, you're on the air. Hello? San Jose, go ahead. Hi, I have a question for either Pamela or John, or Ron, sorry, from the FDA. What is the ideal role of the human factors engineer or group in a medical device company? Go ahead. Pamela would be better for that. Oh, thanks. You know what? I see human factors people in different roles. There are, you may have a corporate level human factors person who is worrying about overall process and getting process documents out there and standards and worrying about that. If somebody's really doing design where they're really gonna be working on the usability of a particular product, I believe they need to be really close to the engineers that are doing the work. They need to be an integrated part of that team and not just somebody outside who comes in and is seen as adding extra work. So there are many different roles. You can also have human factors people again more at the corporate level that are doing training. They're giving courses or they're bringing in courses on user centered design and human factors engineering. So there are different places for them to be. But if we're talking design, I think they need to be really close to the design team. Anyone else? Yes. I just added that one of the things that some of my colleagues have felt was important that has to do with the placement within the organization and in particular the charging for it that to be simplistic about it, human factors support, design support to a program ought to be free. That is the program needs not to feel like they're gonna have to pay for every bit of that. Otherwise they won't make full use of it that if within the corporate structuring it can be a service kinds of facility, that often works best. And that may not work in every case for all organizations but at least it's one of the things that needs to be considered. How do we pay for human factors? And that's important too early on because if they're saying how much is this gonna cost me you need to first demonstrate the real value of it. And then you can do that better if you're not directly charging to the project. And when I said you need to work closely with engineering I don't necessarily think you should report to engineering. I think there are built in conflicts and trade-offs on a product. Engineering's worried about cost and schedule. Human factors is really looking out for safety and usability and there will be trade-offs to make and it's good if they're not all reporting up to the same person. Yes. You know frankly one of the things that companies could do immediately is to put into their process that a member of each product development team becomes the human factors advocate for that particular device. And in a larger corporation than the in-house human factors consultant assists each one of those product development teams with situations that arise. Well I tend to agree with your point about it would be a great world if human factors was sort of this overarching thing that was applied to all projects. I think just from a professional standpoint though there are a lot of other kinds of engineering professionals who might think their service to the product development effort would be equally valuable and should be overarching as well. So I think I have mixed feelings about whether I want to sort of be a line item at a corporate level or whether human factors should really earn its keep by demonstrating as about just the way mechanical engineering and electrical engineering and packaging engineering and industrial design might have to. Caller does that answer your question? God. It must have because she's gone. That was if you're still there Caller that was a very good question. I think a lot of got a good conversation out of that. We've probably got just about three or four minutes before we have to close. Let me do a few more of these written ones. How would the experts here apply human factors engineering to the design of class one exempt devices like wound dressings? You know from my experience some of the most severe problems that have occurred with medical devices have been from some of the simplest items. Elevated toilet seats for example. People fall and break hips all the time with those things. They're multi-million dollar lawsuits. Just because the product is simple doesn't mean that it should not go through a process. The process may be simpler but the risks and hazards that are foreseeable have to be identified. Let me do one more in the time remaining. Make it a quick answer. Does the focus on human factors by the FDA suggest that the FDA may take a more proactive stance on known problems? How will the FDA publish such issues so that manufacturers know that the issue must be addressed? Well, I think from a device of a evaluation standpoint we're going to try to get that information back through the reporting mechanisms that we have already. And if we can get that we will try to get that back in as we review new products going on the market. OK. One of the other things we have for pre-market approvals is a manufacturing section and a part of submitting some of your design procedures. We are specifically looking for in your design input references or consideration of human factors as well as we mentioned the design validation protocols having some consideration to human factors and that's all prior to product approval. I think we're going to play it safe and end it right about there. The clock is telling us that this program is just about wrapped up. I think we can find the best takeaway message from this whole human factors teleconference. They go back and listen to something that Dr. Wenger said in that videotape we saw earlier. Let's go back and listen to it again. An effective design will incorporate users in the design process from the very beginning and will consider carefully how a device will be used in the wide range of environments that it might be used in and the different types of users. OK. I hope that says it all and I hope that the message that you've heard today from these folks takes hold. I also hope you enjoyed this teleconference and that you got some benefit from it. We certainly enjoyed bringing it to you. Until next time, this is Mark Barnett.