 Hello everyone. Welcome to another session of the Harvard Medical School Organizational Effects Consortium. I'm Kelsey Berry and I'm one of the co-chairs of the consortium along with Dr. Charlotte Harrison and Jim Saban. And today I have the honor to be your moderator, at least for the first half of the session, as we discussed how the world's largest professional psychiatric organization, the American Psychiatric Association, has engaged empirical data in its effort to uphold ethics in organized psychiatry. So first to note just about the series, the Organizational Ethics Consortium, which is now in its eighth year, hosted by Harvard Medical School Center for Bioethics, provides an international forum for discussion of organization and health system ethics. Organizations can sometimes seem like monoliths, right? Navigating the world as powerful agents in their own right. And in that regard, we often ask in this consortium what it is right or good for a health organization to do, especially in moments of social, political, or health crisis. But organizations are made up of people too, with their many individual choices and actions. And so we also want to know how organizations might cultivate ethical norms of behavior among their members and activate individuals as forces for good in the world. What tools do health organizations need to effectively uphold ethical norms at an individual level and keep their ear to the ground as new ethics needs might emerge in a constantly changing society? Especially, for example, when the organization's members are diffusely spread out in hospitals, offices, and private clinics across the nation. And so that brings us to today's program. The American Psychiatric Association counts as its members about 38,000 psychiatrists and is tasked with setting ethics norms for the practice of psychiatry. Today, we have the rare opportunity to hear brand new data on the APA's ethics process and to more broadly consider the role of empiricism in keeping organizations on the ball. So we have with us the co-authors of some forthcoming work as well as two esteemed discussions to share their commentary on this topic before turning to audience questions later. So let's take a moment, therefore, to touch on audience participation. There are two main ways for you to participate today. First, you can submit questions for our speakers at any time using the Q&A feature at the bottom of your screen. And then second, we also welcome you to use the chat box to share general comments and reflections or to seek technical assistance if you need it. So with that, I have the privilege of introducing our main speakers. Dr. Michelle Hume is zooming in from Wisconsin right now, where she's a forensic psychiatrist at the Mendota Mental Health Institute, a state psychiatric hospital in Madison. While completing her residency, she was an APA leadership fellow and in that capacity work with the APA Ethics Committee, where she began work on the research that you'll hear today. And then prior to that, she was focused on pediatric ethics and policy at the Food and Drug Administration. And then we also have Dr. Phil Kindilis with us, who's a forensic psychiatrist and medical ethicist whose work focuses on professional ethics, global mental health, and empirical ethics. And this is a little bit of a homecoming, we think, for Phil, who completed his residency at MGH and his ethics fellowship here at Harvard Medical School. But nowadays, Phil is a professor of psychiatry and behavioral sciences at George Washington University School of Medicine and Health Sciences. And he's also director of medical affairs at St. Elizabeth's Hospital, president of the Hellenic American Psychiatric Association, and has been involved in APA Ethics for a long time now, as far as I understand. So we are very thrilled to have both Michelle and Phil with us today. And I will turn it over to Michelle to get us started. Thank you so much, Kelsey, for the really kind introduction. I just want to share my screen here so that we can get started. All right. Okay. So today I'm going to talk about some organizational ethics that Phil and I had the privilege of doing with the American Psychiatric Association. So for those of you not familiar with the organization, the American Psychiatric Association is the largest and most influential psychiatric professional organization. And as Kelsey said, with about 38,000 members. And what's really important is our APA president and things like that at times testifies in front of Congress. It really is the public face of the organization. And one of the roles of the organization is to promulgate specific codes of ethics and things like that, that are used by its members. So there are a variety of ethics training opportunities available through the APA. The APA has a standing ethics committee. And Dr. Lazarus, who is one of the commentators later was actually a former chair of the APA's ethics committee. And so we will have a great perspective on that. Over the years, the ethics committee has promulgated principles of medical ethics and annotated the ones from the American Medical Association in ways that are specifically applicable to psychiatry. So again, you know, sort of lots of shared values. And I'll be talking about that a little bit more. In terms of actually handling ethics complaints against members, there's a variety of resources that the APA has as well as the APA has an ethics office that is staffed and available for questions from any member at any time. So in psychiatry, we spend a lot of time thinking about things like boundaries. We often have very personal conversations with people in the course of therapy and so maintaining appropriate boundaries is something that we really spend a lot of time talking about and thinking about. And that's very important to us as well as, you know, the conduct of psychiatrists, confidentiality, obviously often some of our patients don't even want anybody to know that they're even seeing a psychiatrist. And then one of the things that you may hear about later is something called the goldwater rule, which says that it's unethical for a psychiatrist to offer a professional opinion about a public figure. And this has come up numerous times over the years, but particularly in the last four years, it was something that came up quite a lot, unless that psychiatrist, of course, is personally examined the person and so on and so forth. Again, we think a lot about outside relationships and conflicts with patients and a whole variety of things. So this is just a flavor of some of the principles of ethics that in psychiatry we consider important. So if there is a psychiatrist that somebody feels has violated one of these principles, one way of handling that is to send a complaint to the American Psychiatric Association. And the association delegates the handling of complaints to district branches, which are sort of smaller, you know, often state organizations or regional organizations across North America. And their procedures, it's a peer review process, it's very well spelled out for how to adjudicate those ethics complaints. And so district branches have a standing ethics committee often to do that. And there's a variety of outcomes if the complaint proceeds. One is more of an educational option for the psychiatrist saying, hey, you shouldn't do this, so on and so forth. Alternatively, the district branch can decide to sanction the psychiatrist if they so choose. So the APA has done some prior surveys of ethics complaints over the years as a way of trying to use empirical bioethics to try to understand our organization. And so there is a couple of different prior surveys. One is, you know, from the 1950s to the 1980s, and then again from 2004 to 2007. And so we'll talk about, I'll talk about in a little bit more about that data. All right. So in the fall of 2019, at one of our ethics committee meetings, we had a very long discussion about, you know, gee, it's been 10 years or more since we've had empiric data on the handling of ethics complaints. And we wonder how the organization is handling ethics complaints now, and so on and so forth. And so we just decided that we needed updated data, which was the impetus for the current survey. And so Phil and I sort of sat down and constructed the survey. And already here, we're talking about values, values of the organization in terms of the questions that we ask about for the survey. And so immediately we're talking about ethics and values just simply by the types of questions that we ask. And you can see the sorts of things that we ask about on the slide. The survey was approved by the APA's institutional review board. So again, in the analysis, lots and lots of value laden stuff. And by the way, I mean, in some respects, this isn't hard, the tallying and so on and so forth, but you need a good statistician. So make sure you have one of those. But again here, in terms of how we tally the questions, coded the questions, Phil's going to talk a lot more about this. But that's all about what we value and what we choose to listen to. And so this question of values is sort of inescapable from good survey design. So in terms of data, there were 95 total complaints that our district branches reported over the last three years. Many of these complaints didn't go forward either because there was no jurisdiction by the APA, the person wasn't an APA member or something like that. There was a simple lack of evidence or the complainant said something originally and then never followed up. Okay. There were 22 out of the 95 complaints that actually underwent the peer review process. And out of those, there were two psychiatrists that were required to undergo educational interventions and one psychiatrist who was sanctioned. So as you can see here, there's a wide variety in terms of the number of complaints reported by each district branch over the last three years. There are many that didn't have any and some that had quite a few. In terms of the type of complaints, this in psychiatry, in terms of the kinds of things that we talk about is pretty standard and stuff that you can find in the literature. There's always practice issues. I didn't get the right medication, something like that. Boundary violations, we shouldn't have sex with our patients, things like that. Financial and billing concerns and report issues, all of these things are very typical if you look at the medical malpractice and specifically psychiatric malpractice literature. So in that sense, not too many surprises. So what was very interesting to us, though, is the overwhelming majority of our district branches rated the importance of conducting ethics reviews as important or very important. And they cited a variety of reasons for that and things like the importance of community trust and patient protection. And one theme that came up that I'll talk about a little bit more later is an obligation for psychiatrists to review other psychiatrists. So we ask about as well what kind of ethics support to the district branches valued from the APA central office. And they talked about educational materials and procedural guidance and some things like that that that were interesting to us. We ask about challenges of the ethics review at the district branch level. They often talked about procedural challenges, logistical challenges, and then again, things about what was within the scope of district branches to adjudicate. We also asked some questions about the relationship between district branches and and medical licensing authorities because the other possibility of a patient has a complaint about a physician is to complain to the state medical board about that physician. And so we asked we wanted to understand the relationship between our ethics process and the more formal state licensing authorities. So there were a variety of responses. Very often there wasn't a whole lot of crosstalk between our district branches and state licensing authorities. At times we had district branches refer cases that were were outside of the scope of what they could review. So in terms of additional results, this is more just out of interest. The number of complaints was significantly correlated with the size of the district branch. There wasn't any regional variations. Most district branches thought there wasn't a change or a decrease in the number of complaints over the last three years. And then we asked about the number of hours for resolving a complaint, things like that, as well as you see. So I mentioned this before, but one of the themes that kept coming up is that psychiatrists need to review other psychiatrists. And I think the reason for that is psychiatrists are trained in psychotherapy. We have all of this sensitivity to boundaries and these kind of dynamics that are just a little different than what other medical specialties have. And so that was really important to a lot of our respondents. And again, thoughts that complaints need to be heard, even if they weren't serious, things like that, just a way for the concern to be addressed. When we compared our current data to the prior surveys that were conducted, the bottom line is the number of complaints seem to be declining over time. And we have fewer psychiatrists now that were found to have acted unethically than in the past. And I think that's going to be an interesting topic of discussion later in terms of why that may be true. So just a few final points. Empirical bioethics is an inherently value laden discipline. And when you want to engage in it, you need to have real clarity about the organization's missions and values. Like I said, even to just design the survey and to analyze the survey, that needs to be clear. Organizational policy that is informed by data collection is a model for maintaining contact with evolving themes in the professional community, things that really matter to people. And it also serves the transparency and the service missions of organizations like the APA to make it clear to everyone that we're trying to do what's best to hold our members accountable. Finally, I just want to acknowledge my survey team, as well as the assistance of the council from the APA in doing the survey. So thank you. Thank you, Michelle. And we will turn it over to Phil to keep us going on the topic of empirical work in organizational ethics. Thanks so much, Dr. Berry. And thank you to the consortium for bringing us in for this meeting. Dr. Berry and Harrison, you've made this an educational process, not just a little logistical one. And I really appreciate that. It's a real pleasure, too, to have those discussants, two such distinguished figures. I mean, for those of you who don't know, Dr. Lazarus was like the third psychiatrist to be head of the AMA and to really emphasize ethics and appreciate your emphasis on improving the healthcare system during your tenure and physician health, the graying of the profession. And for those of you who don't follow the work of the council on ethical and judicial affairs, where Dr. Lazarus is the first AMA president to be appointed there, this is some of the most thoughtful and well-written ethics in the profession. If you want to look at the different controversies and the different issues that we have as a profession, go to the Sija website and just read some of the stuff on end of life care and gifts and how to deal with impaired physicians. I mean, this is the touchstone for so many of us in the profession. So thank you to Dr. Lazarus and Dr. Chen, an old friend who does work with vulnerable populations that make her just a unique voice in psychiatry. So thank you. Thank you for that. So my job essentially is to get into the nitty gritty and show you how this is done and that there is a method, there is a favored method for doing empirical ethics for organizations. And that's the survey. So it's a mixed method approach that does kind of quantitative stuff and qualitative stuff all at the same time. So you ask certain questions, how many complaints per year, and then you ask opinion questions that are open-ended where then you code or categorize the answers, you determine whether you're a lump or a splitter, you make decisions on what it is you're going to include or not. And as I say frequently, I mean the decision making on data in research is the black box of empiricism because you, I, we make decisions all the time about what we're going to include, exclude, whether something's an outlier, whether it's a representative of the sample, how you clean data, all the different decisions come with the values that Dr. Hume has been talking about. And this issue of IRB approval, this is a, this is a, an element of accountability to the community, right? It's part of the social contract. You're going to transparently talk to peers who understand this work and say, look, I need some oversight on this. What do you suggest? Can you approve it? Because as Michelle said, the types of questions you use and that, and how you analyze them matters. So in academic surveys, you try to be as balanced as possible. And you don't do the kind of push polling that you see in politics, right? In politics, you can write a question in a way that drives, that determines the answer. So, you know, your opponent is a Satan worshiping, you know, infant eating, you know, blood sucking candidate. Who would you support? Him or our wonderful person? And again, this is true conspiracy theory, right? You may have heard all this baby stuff over the last few years. And again, you can drive answers in this way. And you can drive it by choosing the, the, the order in which you write the questions in which you put them in sequence, because you don't want to get all yes answers and then it becomes automatic. You don't want to get all no answers. So there are lots of principles of survey methodology that again, make, make values, put values on the front, front burner, but also that allow you to generate a certain quality of data. So, you know, what do we do? We had a short survey, we'd have to be short. The people we were talking to were busy. We used this quantitative, qualitative mix to open and closed ended questions at the same time. We had a coding team of three people, Michelle, myself and my chief resident here. And we went through a kind of a training process. This is how you enter, this is how you code. This is how you categorize. This is how you interview. We have a bit larger interview team. So there's a little bit of training involved so that we could systematize what was being done. We ran a pilot with members of the APA ethics committee developed a code book, which is how we're going to categorize the different kinds of answers we're expecting and that we saw during the pilot. And we made this part of an iterative review, both in terms of the historical surveys that Dr. Hume was talking about. So an organization has to have some kind of a repeated process, some kind of an approach to observational and empirical data. And the survey itself, the tool itself has to have a kind of an iterative process. So every five surveys, we would get together as the coding team and say, what are you seeing? How are you coding these responses? This is what I saw. I think I'm going to have to switch this coding category to a slightly different title and shove these answers back into that code. And this is again a way of making sure that something is systemic, systematic and repeatable. And then of course, there's has to be a process for settling disagreements. And that's something that came to the co-PIs to me and to Michelle Hume. So again, you see there's a way of making this systematic of making it comprehensive, but you have to have a training process and one that's repeatable, observable at the same time. And then you've got to interview people. How are you going to interview? There's going to be a script. There's going to be potential answers, potential prompts that you have to develop and you have to agree on. So you're using similar prompts and then you develop this instruction guidance and decide how you're going to contact non-responders. I mean, we had a good response from the district branches, but we also went after them three times if they didn't respond. So we made phone calls and emails. And then in the analysis, Michelle mentions having a good statistician. I was a student at the NIH many years ago and the chief statistician had a talk every summer for all the interns and folks. And it was called, why didn't you come to me first? So you're going to develop something or design something, but you need to have the right sample size, the power to answer the question that you need to answer. So there has to be a previous literature that you can analyze and say, look, with a sample of 100 people, I'm likely to get a distinction between geographic areas, let's say. So the statistician can help with that. And usually they're part of some kind of an academic core in the medical center or you can contract with someone as we do with the medical schools for a few thousand dollars a year to get some statistical hours. But statisticians are critical to design and analysis. So here, let me just give you some examples. I don't have a lot of these for you, but the way in which we coded some of the open-ended responses, a lot of them were what we called practice issues. And these were complaints that complained about someone not treating them without an office visit. The psychiatrist said, oh, you have to come in for me to prescribe something. I didn't like that. So they filed a complaint. They didn't like the diagnosis was one of the complaints that was given to them. They were told that they had a personality disorder and that was insulting. So all these different kinds of things we thought we could lump into something of code that we call practice issues. And there were jurisdictional answers identifying the complaints that they had. Do we have a jurisdiction over non-members? We do not. There was a complaint from an inmate about how they were treated in the correctional setting, which was very interesting and inspired a discussion on scope as well. And then lots of financial issues. And here, the idea of scope and jurisdiction, how are they different? And this is something that we look to the APA principles for. And then distinguishing things like procedure from logistics, which are too closely allied ideas, but we have a process. We have a published APA process for complaints. And again, because we know the field, we know how the APA works, we can shoehorn these answers into a procedure code, these answers into a logistics code. And that's the other thing. When you're categorizing or coding things, the values that you bring from the organization matter because you're a part of it. And the way you know what the important questions are to ask and how to analyze them is because you've been part of the system. So if you come in cold, you have to talk to people. But if you've been with the organization for many years, it's a lot easier to know what the resources are and how to make these category decisions. There's also this Leichert kind of approach to these surveys. How important do you think things are? And then with the statistician, you can decide if something can be lumped into very positive answers like four and five or very negative answers like one and two. There are ways of cutting the data where you show how important something is and that you can make the sample large enough to generate something significant. And that's what we do here with high Leichert scores, answers being the importance of ethics review because of professionalism standards, trust of the community, the need of psychiatrists. And Dr. Hu mentioned the Goldwater Rule. There was only one mention of the Goldwater Rule in complaints, but I can assure you that there are plenty of direct questions and challenges to the APA Ethics Committee on this issue, especially after the last four years. Although there have been psychiatrists who write books like Bush on the couch, Trump on the couch, these kinds of things that violate the Goldwater Rule. And I'm glad to talk about that professional standard in these times. Oh, I have here a picture of the St. Elizabeth's, the bluff over the city where we were built 155 years ago. This is one of the original photos. And again, an element of public sector where the environment, the geography lead to holistic treatment and recovery. Finally, and I already see questions in the chat about why there's a few complaints. Are we more ethical than other professions? We're just better people. And again, our sense is that everything's going to the licensing boards now rather than to us. And we've turned more to an educational option in enforcement so that we're not seen as the ethics police. This is something that Dr. Hume likes to talk to us about. Because it's very important that we're seen as collaborative before we become prescriptive. So we can talk about that in the discussion. But a lot of discussion, a lot of answers and responses to our open-ended questions telling us exactly what's going on out there, that the boards are getting more of this. And they do tend to be more conservative, I should say. They don't have a lot of psychiatrists on them. So these boundary issues that we run into are not always understood or they're understood more harshly than we do. And we did ask questions about when people are referred to an ethics committee rather than a licensing board rather than the ethics committee. And again, you see the kind of values and the answers. When something is egregious, it's like, well, what is that? What complaint is egregious? And a lot of legal stuff and this distinction between law and ethics is a classic one. There's a lot of ethics and a lot of law in ethics. And you see that that's where a lot of the ethics committees that chairs that we talked to, where they were making these kinds of legal judgments. This is more a legal question. This is more an ethical question. And for those of you who've been in this business for a long time, you know how hard it is to draw these distinctions. So that's it for me. I mean, I just wanted to make the case that there is a method. It is systematic. It is imbued with values. And it's the first step in an organization's development of their organizational ethics. So you have to have something descriptive before you have something normative. So you have to know what's out there, know what the issues are for your community. And then you can say, well, we probably shouldn't be doing X. And these are the standards that we should be following instead. So that's that's it. And I'm looking forward to our discussion. Well, thank you so much, Michelle and Phil. Really letting us kind of peek behind the curtain, I think of APA ethics and what goes into such a robust process. And in part also really showing us how ethics and empirical work are so deeply interrelated. And a culture and practice of empirical inquiry can certainly be ethically productive for an organization and revealing areas of need and areas where attention is warranted. But it's also ethically demanding, right, given the many value based choices that are going into the design and interpretation of this kind of empirical work. So for those of us who are really being trying to be thoughtful about the potential for, you know, ethics in our organizations and in the world, there's just a lot to think about in the course of engaging empirical inquiry, not only how it serves our aims, but how ethics can serve the aims of developing good empirical work. So we are very fortunate to have two discussants who I know you've been hoping to hear from. So both of them have thought deeply about ethics in organized medicine. And then also how well conceived empirical work can bring ethically relevant insights to the fore. So I'll introduce both and then we'll turn to Donna first for her commentary. But Donna Chen, Dr. Donna Chen is joining us from Virginia right now, I believe, where she's an associate professor at the Center for Health Humanities and Ethics and in the Department of Psychiatric Medicine and Public Health Sciences at the University of Virginia. And there she does quite a bit. So in addition to her clinical work in psychiatry and research focus on the health needs of underserved communities, she also oversees the ethics and professionalism component of medical education and does organizational ethics for her institution. And Donna was previously at the National Institutes of Health where she trained in consultation liaison psychiatry and in research ethics. So we'll hear from Donna in a moment, but I will also introduce Dr. Jerry Lazarus, who's coming to us from Florida today. Jerry is a clinical professor of psychiatry at the University of Colorado Denver School of Medicine. And he first became involved with ethics at the APA right after its birth in the early 70s, and was chair of the APA Ethics Committee quite early on before heading over to the American Medical Association, where he was first delegate and then speaker and then ultimately president of the AMA. And he's now serving his fourth year on the AMA's esteemed Council on Ethical and Judicial Affairs, as you heard from Phil. And some other activities of Jerry's currently on the Ethics Committee of the Department of Defense and working with the Center for Bioethics and Humanities at University of Colorado to integrate the lessons of the Holocaust into medical training. So we're pretty excited to have both of these commentators. And I'd like to bring Donna in first. Go ahead, Donna. Thank you, Kelsey. Can you hear me? Yes. Okay, great. So thank you for inviting me to commentate for this wonderful, wonderful talk. I have tried to limit my discussion to five minutes because that's what I was hoping to do so that we can get conversation going. So I have, I don't know, three maybe four observations I wanted to make. The first is I think many of us think about organizational ethics as kind of what an organization does. And the APA as a professional organization definitely has ways that it manages its kind of ethical processes. But it's also an organization of professionals. And so what the APA does affects its members in their own offices, in their own potentially private offices. And that's some of what you heard about today in this really wonderful talk. But as some of the folks in the Q&A have noticed, this particular way of tracking how our profession is doing and how the organization is doing, focuses really on a very downstream way of counting and looking at problems. When you look at complaints, right, that's obviously really important to learn about when we figure out what we're, how we're actually doing as an organization and how we are helping our members stay doing the right thing. But there are so many factors that go into whether a complaint is made, how it's tracked, how we find out about it, that it is a, it's one small lens into how the profession and professional ethics are going. Another thing that the American Psychiatric Association does, which was touched on a tiny bit, not in depth, was that it has a huge kind of educational component. And that's another aspect of kind of what I would, what I tend to see as the organizational ethics of the APA. The APA has an office and a committee, an ethics committee that helps answer questions. They do what in many organizations we would see as ethics consultation. So any member can call in and get a consultation from the APA. And many of those end up generating educational materials. They end up in opinions that are, get tracked over time that are associated with our ethics code. And those are updated every year and available to everybody. And so it would be really interesting to actually study those, the questions that come in there as another lens to what are the kinds of ethical issues that our members are facing and what it's a more proactive approach. It's a more preventive approach. But that's another view on what kinds of things are within that area, I think of ethics in what psychiatrists are doing and what they're facing. So kind of another, I guess, piece to this that thinking about this talk really got me thinking about was, you know, when we create these educational materials, we in some ways have an ideal vision of how they will be used and how our members, whether, how our members will follow the kinds of guidelines that the materials support them in doing. And so I think there's an interesting process to think about from an organizational perspective as to how these educational materials that might present an ideal vision of how people should behave, when do those become enforceable as ethical norms of the profession in a way that not doing that would generate a complaint. So one area that I was thinking about when I was thinking about this is that the APA as many, many organizations has really, really taken to heart how we might address structural racism within our profession and, you know, addressing some of the historical ways that we contributed as a profession and potentially as an organization. And so there are a lot of really, really useful educational documents and statements that the APA has made in this arena around structural racism, around implicit bias, even around explicit bias and outright discrimination, really putting these conversations on the table. And so however, like many organizations, the efforts around diversity, equity and inclusion actually exist slightly separately from the ethics processes of those organizations. Sometimes they overlap, but in many organizations they're actually separate, they're separate processes. They're separate processes. And so it got me thinking, and this I hope we will have time for discussion about, you know, at what point would the kinds of educational materials, the kinds of, you know, visions around how we all ought to behave, at what point would those, would the ethics processes of the APA step in to monitor those to potentially oversee them? And at what point would they entertain a complaint in this area as a complaint about the ethics of one of its professionals? So I think that's it for me. Hopefully I didn't talk too long. Not at all. We really appreciate your drawing our attention, Donna, to kind of the challenges of measuring ideals of conduct and looking to ways to identify how to hold accountable to ideals of conduct that are perhaps only recently being really highlighted as essential. So really appreciate your bringing up this topic. And we'll hear from Dr. Jerry Lazarus now. We'll have time to come back to this big question that you've asked, Donna. So, but first we'll turn it to Jerry. Jerry, take us away. Thank you, Kelsey. And thanks to Phil for those very kind words that you mentioned about me. It's a pleasure to be with you all today. In a sense, this goes back to the beginnings of my involvement with ethics, because I was on one of the first district branch ethics committee back in 1973, when the APA annotations were first published. And then as Kelsey mentioned, went on to become chair of the APA ethics committee. I also wanted to acknowledge that also Dr. Rebecca Brandel is currently on AMA's CJA. And Dr. Jim Saban was also on CJA a couple years back. What I wanted to comment on was a bit more about what the AMA does, which in many ways is similar, but in many ways also different. I think one of the things looking back when we started in 1973, we didn't have the kind of data that Phil and Michelle are giving us today. And I think that's extremely helpful to be able to track the kind of complaints that are coming in. And I think it's also fair to say that the APA was really at the forefront in pointing out sexual misconduct, boundary violations, as Michelle mentioned, even though this had been part of the Hippocratic Oath, I think we found that there were not enough aggressive and assertive actions against psychiatrists at that point who were involved in boundary violations. So I think the APA was really at the forefront in that. But trying to contrast what goes on at the AMA, as you all probably know, or if you don't know, the AMA was founded on the bedrock of code of ethics, as well as standardized educational programs. And it's enshrined in this really big book, Code of Medical, it's really thick. But that's what Phil was referring to. So the Council on Ethical and Judicial Affairs at the AMA has two main functions similar to what the APA Ethics Committee does. First and really foremost is to work on ethics policies for the association. And the way this is vetted is the committee comes up with a report, which it delivers to the AMA House of Delegates, which in one form or another represents every physician in this country, because it's represented by all state and specialty societies. And then recommendations are made and either accepted or not accepted or referred back. And then eventually, when approved by the House of Delegates, these go into an opinion that is published in that big book that I showed you. So that is really the north star of ethics policy, which drives a lot of the rest of the policy of the AMA. So for example, if you looked at the recent controversies about vaccines, vaccine mandates, masks, confidentiality issues, these are all addressed in these opinions. And these are all available online for free if anyone wants to take a look at that. So it really addresses issues that are current and also emerging. So then the other part of what CG does is it does adjudicate complaints that have been brought through state medical licensing boards. And as opposed to getting these complaints from patients or others directly, the approach that AMA CG uses is to only adjudicate, since the time I've been on, that's the only ones that they do that have been adjudicated through a state medical licensing board. And then we have a similar process where we have hearings for that physician. And then a determination is made about a sanction of one kind or another. So it's quite different in some ways from the APAs. And we can talk a little bit more about why the number of cases at the APA has fallen off. And I might have some thoughts about that in addition. But I think it's also fair to say that the complaints that we look at, and Phil mentioned this, how serious are they or how it reaches? I think on a continuum, they are probably the more severe instances of complaints that are brought against physicians, not always. And because the primary obligation of the state medical board is to protect the public, sometimes the actions that they take against physicians may be actually more severe than we might take in more of a peer review process with our own colleagues. But the state medical boards also do, if necessary, they will require a physician to have ethics training or boundary violation training or medical records training, all kinds of different courses that they're required to take to try to help in their rehabilitation and return to practice and finishing up with their ethics complaint. I did want to comment a little bit more about the data that Phil and Michelle brought forward. Because again, and I think this reflects a little bit of what Donna was talking about. I think this is really important data, but I think it's really the tip of the iceberg. Because it really is only based on the complaints that have brought to the district branch. And the other mechanisms that we use both at the APA and at the AMA, to gather information about what current ethics issues are relevant to physicians and for psychiatrists in particular, those are brought through the APA's board and the APA assembly and through its district branches. But I don't think we are tracking those as much as we probably should. And as I was thinking about the empirical ethics part of this, I think we should be thinking about both at the APA and perhaps Becca and I can bring this back to the AMA as well. Although we at the AMA track the kinds of the number of complaints we've had that we've adjudicated in the sanctions, we haven't actually indicated what they're for. In other words, what the complaint was about. So I think the fact that that was gathered through the district branches I think is extremely important. Of course at the AMA level, the complaints are public knowledge. So one could go back and look at what they're about. But I don't think we have that information. So we have the one piece, which is the adjudicated ethics complaints. But we also have additional information about what's coming forth from the members about what the current ethical issues are. And I don't think we're capturing that in this data. So I think we may be picking up pieces of issues that may be of ethical relevance, but we may not be always picking up the issues that are of relevance right now in a broad fashion, which don't result in complaints. So I think the approach similar to the APA, where I think it is really moved from more of a disciplinary process when I was involved back in the 70s and 80s, to more of an educational process is something similar to what we are trying to do at the AMA and have been trying to do because there are tremendous resources regarding education and ethics. We've got the Journal of Ethics and we try to provide guidance to physicians of any specialty by allowing them to take a look at the ethics code. We get hundreds of calls from physicians and others about the code, what to do when you have an ethical dilemma. So much of this, again, is hopefully before a physician has crossed the line and either done something mildly, severely, or egregiously unethical. And I think that's what we are aiming to do to try to educate the profession, to try to keep them out of these ethical dilemmas. I'll just make one more comment before I finish and that is the AMA, as well as the APA ethics for the most part, has been written with the individual physician or individual psychiatrist in mind. But really more recently, we are also paying attention to how that individual physician should also within their organization try to influence the values and the ethics of that organization. And I think that has been a change over the last number of years to make recognizing that physicians often are working in organized systems of care, they're employed, they're in different offices where there's integrated care, and not just in their own private practices. So we want to make sure that whatever ethics guidance we give will be transmitted to the organizations in which those physicians are working. So I think I'll stop there, Kelsey, and look forward to the Q&A. Thank you so much, Dr. Lazarus and Dr. Chen, for bringing us these additional perspectives and essentially extending the issues that were brought forward by Drs. Hume and Candelus into related context. So adding to the sort of scope of the work that's been brought forward for us to discuss today. At this point, we're beginning the open discussion period of the consortium session. This period has two components. One is discussion among the speakers and the other is discussion with the audience. So first, I want to give a quick reminder and invitation to audience members, please feel free to contribute your thoughts to the chat. And if you have a question, please put it in the Q&A where it's easier for us to track and make sure we have seen it. I think also to our four speakers, we're about to start this discussion by inviting you to pose questions to each other. We thought we would ask Dr. Chen to kick this off because she had posed questions in her commentary and perhaps would like to select a couple to lead off with and then just invite other panelists to jump in. Dr. Chen. Actually, I wondered if the folks in the audience knew what the Goldwater rule was. I know I didn't bring that up in my comments, but I guess that's one thing because it's come up several times. And so I wondered if, and Phil had said that he could comment on that. So I think that was one thing. And the other is I really do, I would love to hear the people who, you know, run the ethics committees for these big organizations, how some of these ideals end up becoming enforced by the organizations as kind of ethical norms and rules. So can I take a crack at that? Because I think that the questions that Dr. Chen and Dr. Lazarus raised can't be answered empirically. And then we can make policy from that. So if we do a survey of the members, right, we could have something like the structural racism task force tried to do. What are the problems out there? If we survey the APA ethics office to see what the phone calls are that come in? Or if we look at the assembly to see what kinds of proposals are being. Now, all of this is imperfect. And the structural racism surveys, for example, again, they had to do these things. They had to look to the membership to see what their understanding was of structural racism. But when you get only 500 to 700 responses out of a potential 18,000, because that's what the access was, that's not effective data, right? So you get a 0.04 response rate, percent response rate. We had Michelle on ours 65%. It's very hard to make a judgment, right? So the data matters, the quality of the data, how it's gathered and how it's analyzed. And we run into these problems, but we still have to do it, right? We still have to survey members. We still have to take the next step. For Dr. Lazarus question on, you know, how we look at all the potential routes for ethics complaints, you have to choose a regression stopper at some point, right? So for us, it was to repeat, to continue the iterative process that the organization had taken in the past. So we look at the prior surveys, and we say, ah, we can improve on this by doing a well-constructed questionnaire and doing something similar. And we stop there. So the next survey could be of members or of complaints at the same time, but the quality has to be good. You can't have a response rate of .04%. And you have to, I mean, even the responses for the structural racism task force out of APA, there was still a percentage of people who responded, well, I don't see it. There is none. I don't know what you're talking about. Don't know. Again, that's not helpful. I mean, we know that there are people who don't see it, but it's not helpful to, it's a waste of our resources to get answers like that because we're interested in moving forward. So two responses there to kind of bring in Dr. Lazarus and Chan at the same time. And on Goldwater, again, Barry Goldwater was the conservative, the Republican candidate for a president in 1948 when I was at 64. And the psychiatrist did a survey, one of the failed magazine did a survey of several thousand psychiatrists who said he was unfit, that he was schizophrenic, that he was, you know, a danger to humanity because he was conservative and would use the A-bomb. And obviously the outcry was impressive. And the APA wrote a particular section of its principles about, you know, making comments about people in the public domain. We shouldn't be doing it. We value informed consent. We value confidentiality. So unless you've got permission from, and again, Rosalind Carter, Tipper Gore, I mean, important people have talked about their mental health issues, unless you have permission, unless you've examined someone, you should not be making statements about public figures. It's a violation of all these kinds of principles of informed consent and confidentiality and good science, frankly. There was a book, I think, recently, about 27 psychiatrists have an opinion about the president, and everyone had different diagnoses and different approaches. So it's not a particularly useful statement. So I'll stop. That's very helpful. Thank you. Do others want to speak? Michelle, did you want to? Oh, I'm sorry. Sorry, Jerry. Go ahead. I don't know if it's my turn to ask a question or I did have a comment. I'm just going to ask if Michelle wanted to say anything further about the questions that that were brought forward by Donna. I, you know, I'm going to defer to Jerry because I am interested in what he has to say about this. Jerry? Okay. I'm not sure what this is, but I'm going to respond to what Donna brought up because I think the important issue here is how the organization sort of looks at ethics. And I would say that at the AMA level, and again, not being on the board of the APA, I suspect it's very similar. The organization often looks through an ethics lens at many of the questions that come up before us. Not always, maybe not as much as we would always like, but oftentimes it does. So for example, during this last year, during the pandemic, there was a whole section of the AMA website that was devoted to ethical issues related to the pandemic, you know, about masks, about vaccines, about those kinds of issues. And so, and a lot of physicians, a lot of organizations were looking for guidance on how to deal with this issue. So it was right there. So that so, and I think when Donna brought up the issue of disparities, I think again, the council, the ethics policy also looks at that. But I think the ethics lens is something that I think the AMA pays special attention to. There's also one additional part on the organizational side is that recently, the organization has convened a committee that looks at complaints about behavior amongst its own members within the governing bodies or its committees. So for example, with the House of Delegates where you've got, you know, maybe a thousand people or the committees, and is tasked with adjudicating complaints against those individuals. So it was formed mainly around issues of harassment of various kinds, sexual harassment, and so on. So that's the way organizationally the AMA has dealt with it, and Michelle might have some other comments about APA. But I did, if I have a chance, I would just want to ask my question to Michelle also and Phil, because, you know, it does seem as I see the data that you have that from the time that I was involved with the APA that the ethics committee has moved considerably away from being more of a disciplinary or ethics police or functioning in that manner. And I know there are lots of reasons for that. And it's striking to me that of all the complaints, that there are only two educational interventions, one sanction. So, and I'm wondering, I'm wondering if it's really, it really is functioning as a peer review process. And the fact that the district branches are telling you that the psychiatrists think that they have an obligation to talk with their colleagues when there's a problem, I think that's absolutely correct. But it sounds like it's more of a peer review process than sort of the ethics process that I remember. So that's really my question for Michelle and Phil. So just to clarify, your question is, you know, is this, is it a peer review process versus versus what? I'm not quite sure. Well, an ethics procedure where you have a process that you go through where you either have a sanction or you don't. A peer review process where you're basically discussing with colleagues, you know, it sounds like there's been an ethical complaint here, a problem. What do you think about it? And how can we help you deal with that? So, I mean, the peer review process, the procedures that the APA has for handling the peer review process handling, the process is actually, it's actually fairly involved and it involves, you know, getting more information from the complainant, sometimes, you know, records potentially, all of that kind of thing. And then it's from that review that is done typically by the district branch ethics committee that then there's a decision made about, you know, what is the right outcome here and does the, you know, first of all, did the psychiatrist violate the ethical principles in some way? And if they did then, you know, sort of, what do we do about it? So it is, I mean, the procedures are actually very well spelled out by the APA for the peer review process and it is quite involved. Okay. Yeah, I mean, it's definitely a peer review. I mean, Charlie, you know, we used to have something called a grievance process where it was much more collegial and it was for things that didn't quite rise to the level of a complaint. And that was very peer review and education oriented. But with the shift in the last five, 10 years of this kind of educational component, those things, I mean, they still happen, the peer review counseling, the peer counseling happens. But there's also education for the patient. So if the patient says, you know, the doc won't write me a disability letter, I'm filing an ethics complaint, there's education that goes on in that direction too. So, you know, this is allowed, you need someone independent or what have you. So there is that peer flavor throughout the process. And I think that's what's valued by the membership. Because the other aspect of this chair is whether it's etiquette or whether it's ethics. And a lot of these complaints seem to be etiquette issues. And the kind of education of how you talk to a patient or what a patient should expect goes on throughout the process. So it may not get to the end. Again, the other stuff where you're defrauding me by billing too much and things like that. I mean, that goes everywhere. That'll go to the board of licensure, it'll go to the district branch. People file lawsuits, you know, so the lawsuits to recoup the fees that they paid to psychiatrists, you know, I didn't understand the cancellation policy. Your fees are too high for the introductory. I mean, it's not all massive issues that you and I discuss when we craft these answers. There's a question also about cultural issues. And I hope that attendees understand that the cultural formulation is now a part of our, I mean, it's part of the DSM. It's part of our, the appendix, how to do a culturally informed interview. So when we have patients from global, representing global south, we are applying, you know, principles of exploitation, principles of insensitivity, principles of, you know, violating the rules of confidentiality through the cultural lens. It's not just the ethics lens that Dr. Lazarus is talking about. It's this cultural lens as well, but how people should talk to each other, how they should behave, whether the space between them is culturally appropriate, whether the kind of eye contact that they make is considered a part of the problem. You know, are they being paranoid? Are they being disrespectful? So the cultural formulation is very much a part of psychiatric evaluation and of the ethics process. I just wanted to say one more thing, Phil. And then it looks like there's some other questions. But as you probably mentioned, the difference between etiquette and something else. And I think there were a lot of concerns about many of the earlier editions of the AMA code that had more to do with etiquette issues rather than the ethical issues. So I mean, the reframing of the code and is really more around the, you know, the significant ethical dilemmas that, and opinions around those that need to be addressed by physicians, not the etiquette challenges. Yeah. So taking very seriously the increasing challenges. And it sounds like there, in some ways, this begins to address some of the question about evolution within the organizations that Donna was bringing up. And, you know, there was her, the component of her question about how things, you know, how ideals end up getting into a position of being enforced within a, within a professional organization. And we could say the same for other organizations in healthcare. We have aspirations, we declare, we make statements of our goals and our values. And then, you know, how do those statements begin to translate into more sort of enforceable and targeted aspects of running an organization? And what's the contribution of empirical work to, to those, to that process? I wonder if people would want to comment. I think you have illustrated one way. Is there anything more that people would want to say about that? Do you envision other ways? Well, you know, if you look at the AMA code and the opinions, we have shuds or shells or maize. So I don't think that, you know, we, the AMA, the seizure itself doesn't act as the sort of ethics police going after everyone that violates any of those opinions. I mean, the hope is that physicians will look to the AMA or psychiatrists will look to the APA and the AMA to determine whether something they have done or they're considering doing or something they have seen rises to an ethical problem. And we get, and I know the APA gets these calls also. We get hundreds of calls, as I mentioned, questions about, you know, is this, is this an ethical thing to do? And, you know, we can go over that with them. We can direct them to the opinion. You know, at the end of the day, the physician is going to make a decision about what they're going to do. And hopefully it's the right decision that benefits a patient and doesn't hurt them. I mean, but, you know, we can't, we can't go after each ethical decision. I mean, we try to set the bar high and set it high at doing things that are for the benefit of the patient primarily, as well as for the benefit of society and ourselves. But the balance is usually towards the benefit of the patient. And that's how we do that sort of ethical analysis. And I know the APA does the same thing. Yeah, I think that's exactly it. I mean, I think that's the answer to Dr. Chen's question. I mean, it's there are there are standards for exploitation and for maintaining the best interests of the patient. And we fit a lot of these complaints into that framework. In fact, we went back to the educational commentary that we did. We've renewed that every so often. And we added some of the racial language to it so that it's very clear that we're talking about racist attitudes and racist behaviors. It's a kind of a move towards anti racism. Can I comment real briefly on this wonderful, brilliant question on restorative justice? There's a question from Rebecca, see, would you describe the APA review process as comparable to a restorative practice in academic integrity, such as, you know, a college or university would consider, I don't think we're quite there. I don't think we're there to facilitate the disagreement between the patient and the psychiatrist, because there's so many protections of confidentiality in this in the process of the, I mean, you'll guess who the patient is, if you're, if they're filing a specific complaint, those things are kept separate for so long that it's not quite restorative justice. It's not quite the facilitation between the complainant and it's more educational in two directions that don't often meet because of the protections. I mean, you can't identify people. You can't even identify the psychiatrist when it comes to the district branch. We say we have a complaint. If this sounds like anyone you know, first of all, get off, recuse yourself. And then we can discuss it in vague terms. And then we, there's an investigation that specifically talks to the psychiatrist and the patient complainant, but they're kept separate. So it's not quite restorative practice where, you know, people kind of figure out their own thing with a facilitator, but a brilliant question. I mean, I'd love it, but there are just too many liabilities, both the institutional and legal. There are some protections that the psychiatrist requires. Really helpful. And I think you just started to dip into the audience questions, which we'd like to do next. Kelsey, let's see what you're, what you're seeing in the chat and the questions. And also if you want to add a moderator question, please feel free. So there's, there's quite a bit looking for more context, right around the declining number of ethics complaints over the years. And I'm sure that some of what you've said have, has already spoken to that in some respects, but I just wanted to raise that and give you a chance to, to use a little bit or if there is an appreciation of perhaps what is causing a declining number of complaints, the audience would love to hear that. And I'll just add a second piece to that, which is to say, how might a declining number of complaints be interpreted from the point of view of the organization as it considers the role of these empirical data as a potential guide for future work on ethics in the organization? Well, I will, I will try to tackle some of, some of that question. We had, you know, lots of our members when we asked them, you know, about whether they perceived that the number of complaints were declining and all of that kind of thing. What one of the questions we asked was why, you know, why do you think that's true? And, and people had all kinds of ideas about that. And some of it was a little, maybe a little bit self congratulatory, I would say in terms of sort of saying, well, we're better trained now, you know, we have all this ethics curriculum that is part of medical school, which is certainly true, you know, all of that kind of thing. I think one of the other things that is, that's very clear from our survey data too, is that there's the perception that a lot of the, a lot of the really, you know, more egregious complaints are going to state licensing boards, because there is the perception that, you know, the state licensing boards have a little, have teeth in terms of, you know, actually being able to take somebody's license away or something like that if they actually did something that rose to that level. There was, you know, there was one of the other questions, I think, in the chat is, you know, do, do, does the public know about this avenue of being able to, to complain? And, and in some cases, yes. And in some cases, I think no. And, and so, you know, making a complaint is not hard. You call up the organization and sort of start there, but, but, but, you know, does the public think to do that? Or does the public think, you know, gee, I don't, I don't like what my doctor did. They did something, you know, egregious and unethical. I'm going to go to the state licensing board, you know, that kind of thing. So I think that, I think that there's a couple of, I mean, I think there's multiple reasons why it is that overall, the number of complaints is, you know, relatively low given the membership. I think to weigh in just a little bit on this as well, sort of thinking back of the history. Because early on, when we first started out in this process in 1973, the state medical licensing boards really were not, especially sexual misconduct, were not taking it particularly seriously. And would often not do much about it. And I don't recall, you know, how that message got out to the public, but then we had an upsurge in cases. I don't think we had any more cases at that time than we had in previous times, where we still have these cases. But there was then an avenue for a patient to complain and get, you know, some investigation and some action against the psychiatrist. And what happened after that was these complaints, when you had the ethics complaint, became very serious for the psychiatrist's ability to practice. So for example, when the National Practitioner Data Bank came to existence, these actions were reported to the National Practitioner Data Bank. We made an expulsion or suspension from the APA public, we put a notice in the newspaper. So it could affect their privileges at a hospital or getting insurance. So the stakes were very, very, very high. So that's why, but since that time, the state medical boards have taken these kinds of complaints very seriously. And, you know, the psychiatrist or physician has, you know, a legal process to go through, attorney representation at the state medical, state medical board, so the protections for them are, you know, much more reasonable than at the point that we were doing those in the early days. Now, I'm not saying that that is the primary reason, but it's probably one of the reasons why there's been somewhat of a decline. I'm sure there are many other reasons in addition. Michelle and Phil might agree or disagree, I don't know. No, I mean, they see a lot, the board see a lot more complaints than we do, than the district branches do, that's for sure. And there are civil lawsuits too. So I don't, yeah, I don't think that people know as much to go through the district branches, there aren't complaint forms in outpatient offices, as someone has suggested. But, you know, they do ask. So, hey, you know, this is not right. I need to complain. Docs tell people how to do it. It's like, look, if you have a complaint, you talk to the board, you call the district branch, here's the number, because that happens too. I mean, we see that with these cases. There's also a question about what happens if there's insufficient data, you know, can people be wrongly convicted or referred for disciplinary action? And generally not. I mean, if there isn't data, if it's not confirmed, if it's a rambling, psychotic letter or something like that, then the psychiatrist is not thought to be, you know, eligible for, you know, the educational option or something more serious. So I think these investigations also help. So you send a couple of psychiatrists, you talk to people, you look at the record that's available, you make the best judgment possible. So I think that just on really kind of on this theme of considering perhaps the public's behavior as a determinant of the complaints that might be coming to the district branches, as well as, of course, the kind of shifting areas of responsibility and enforcement of different institutions. But I just wanted to develop some of the questions that I'm seeing coming up in the audience Q&A. So this work is really kind of demonstrating a commitment to a bottom up approach and kind of this deep listening to those who have complaints communities in a way that could potentially help identify blind spots, right? That the organization might not otherwise really be thinking about from the point of view of ethics education. And what I wanted to ask is how do data like these, which have the potential to guide that practice and thought, if they do rely in some respects on community's awareness of their role in raising their voices, what challenges might this raise in consideration of the fact that there might be kind of a squeaky wheel effect that could dampen what you learn from communities that are not as accustomed to raising their voices in the context of perhaps problematic encounters or who have not been listened to perhaps in the past and therefore are not quite as likely to complain in the ways that are described if there's a barrier or a lack of information about how to do so. Does it maybe suggest a role for the organization in considering more education of the patient population to be able to help spot concerns or how would an empirical approach kind of address some of the concerns about the underlying bias in the data itself? Boy, I think we could write a dissertation on that. Yeah, the data has to be good. It has to be descriptive. It has to be real. Yeah, and this was our issue with COVID. We got a lot of questions, the APA Ethics Committee on the obligations of institutions. So institutions weren't prepared, generally speaking, for the pandemic. We're prepared at a certain level for emergencies. We have a certain amount of personal protective equipment. We have a certain number of respirators for my thinking of intubators. So we're not quite prepared. What is the organization's responsibility? And certainly seeing the phone calls come in to the ethics office, it was a lot about that. Why am I being put in close proximity with the patients now? There should be social distancing. They should be masked. Shouldn't we be doing more telemedicine? So we came up with some guidances that Dr. Lazar pointed out. We had online forums. Dr. Brindell did question and answer sessions. We did teaching sessions for the community and for psychiatry. And it's this idea that Jeremy was talking about, this AMA code, which was born in a pandemic, by the way, right? So this is Percival's ethics born in the Manchester epidemic of typhus whenever it was 17, late 1700s. And it tells us that there's a balance between what the organizations can do and what the individual can do. I mean, you can advocate for patients to be socially distant masked. You can resign. You can go to your state legislature. There are certain avenues that are acceptable. But ultimately, you may have to decide that you can't work in this environment. And we've seen that. I mean, we saw people leave the med surge hospitals during the height of it and go practice elsewhere. So I think we can use empirical ethics and empiricism generally to answer these questions. But it has to be couched in terms that we recognize because if it is something that's beyond the pale, I'm ready to sue the institution for not being prepared. And this is why Zeke, Emmanuel and others said, look, these are the ways in which we balance the public interest against the individual practitioner. You have to consider the protections of the staff along with the protections for patients. And here's a way to do it, for example. I mean, what's your triage process and the great commentaries over the last year? Do we know if we're tracking the questions that come in to the APA, the ethics office, or the ethics committee, these kinds of phone calls? Because I do think that would be a really interesting other window into the kinds of ethical questions and ethical dilemmas that psychiatrists have. It's kind of on the front end. No, I may jump in. Oh, sorry, Phil, do you want to give a... No, no, I don't know if we have a metric. I don't know that we have a metric for that. But what happens is whenever a question comes in, people are referred to an opinion or a particular section of the code or the commentary. And if there's no settled answer, they'll appoint a panel of three members of the ethics committee to come up with a quick answer. So I think we could count those things. I just don't know that there's currently a metric for the phone call. Yeah. Don, I think that was really what I was referring to as well, that I don't think we're doing that at the AMA either. I don't think we track that. I think that would be another interesting thing to track. And I also wanted to respond to Kelsey's question because I think what you're saying is what are the limitations of an organization like the APA to sort of look at the overall community? And what's its obligation to gather data from the overall community? And that's an interesting question. I don't have an answer to that. Yeah, I think it's a similar question for the AMA. I mean, there are organizations and systems, integrated systems of care, hospitals, public health settings that do gather data on what their community thinks about X, Y, and Z. So maybe that might be in the form of a coalition or some group together. But I think it's a question of what, as Phil brought up, what are the limitations of what we can do with the empirical part of it? And already what Phil and Michelle gathered was great data, but it was a slice of the overall continuity of what's goes on in the community. So it takes a lot of energy to do that. Good to think about that. I think we brought forward a really important range of questions. And honestly, it's a sign of an especially valuable consortium session if we wind up having to stop before we're finished. I have to say, I think this is a fascinating, complex conversation that many people will be continuing to think about with respect to our own organizations. I really want to thank our outstanding presenters and discussants, Michelle, Phil, Donna, Jerry. You know, you've led us through some principles and good practices and empirical bioethics in an organizational setting. You pointed out the inherent value ladenness of the process. Clarified a number of ways. Honestly, when carefully doing this work, as well as transparently sharing the work can really contribute to the evolution of the ethics oriented measures within an organization. I think all organizations are, you know, hopefully growing and, you know, responding to changes in their environment in our social environment, as well as the professional environment. And this has been a really helpful forum for exploring how empirical bioethics can be an important piece of that development. I want to also thank our very thoughtful audience. A recorded version of this webinar will be available on the YouTube page of the Harvard Center for Bioethics, also Facebook page. This consortium is going to meet again on January 28, 2022. And again, monthly through the spring, as we continue to explore a range of perspectives on issues in organizational ethics. Programs will be announced on our webpage at the Harvard Center of Bioethics, and we look forward to seeing you again. Meanwhile, Fourth Fridays, our usual slot, are going to be associated with major holidays and end of year observances for the next couple months. And we hope that everyone enjoys those and returns for a healthy new year to see us next year. Thanks, everybody.