 So first let's start off with a new thing that's having a cross ophthalmology that may or may not be applicable or needed in our residency program That's night float. So night float is something that Became really in vogue in internal medicine and other specialties When the original work hour restrictions came into play And so in any given intern year you'll have a month that can be on night float That might be for a senior resident junior resident. So how that's looking in some programs. It would essentially be a Month where one of the residents or two of the residents would actually cover all of the evening call And that might be a just during the week where rather than be here during the day They're actually covering the overnight it could be a hybrid where you have a night float for the weekend So for the weekend, they might be on call from 10 o'clock to 8 o'clock 10 o'clock to 8 o'clock and then have a Monday off something like that as This discussion came up at the AUPO meeting It was actually quite controversial There's some really good evidence that night float is no less effective and no less helpful and no less healthy Than the current system that we have So really that one I'll just put out there for residents. We certainly would be interested if that's something that might make sense for a program Integrated intern year something that We all know about well, but integrated internship is now being mandated across the country And even though there were several well, there were three full months of opportunity for input From the program director's chairs around the country when this was rubber hitting the road at our AUPO meeting in in January It was as excited as exciting and as heated as that room has ever been several chairs stood up and say said things like do you Recognize that there are some programs that may not be able to do this And if we're shutting down programs with an ophthalmology, are we really gaining or are we harming our profession? There has not been any significant change in the number of ophthalmology residents We've been training now for almost 15 to 20 years So our the number of ophthalmologists being trained in our country is fairly static despite the fact that our population Continues to increase with the grain of our population What will ultimately happen? There will be an integrated intern year For for everyone across the country and whether that's an implantation over one year two or three years that remains to be seen And the most compelling argument for this was if we stepped back and we decided how do we want to train ophthalmologists? We would never set up a system where we give away one year of training to go train in medicine Where we have zero control over what they learn how they learn whether it's a transitional year or preliminary year We would Design it in a way where they can get experience in internal medicine They can get some experience in surgery and they can integrate with the rest of the body of medicine But we would do it in a way that was deliberate and thoughtful to prepare them to become ophthalmologists in the end So with that I'll just pause for any thoughts questions comments concerns. Dr. Jardine Do you see That's an amazing question and you you've you've well, let me just do this before I answer that question anything else regarding Integrated intern year questions That's a good question. I think for some of the chairs. I Gathered this was the first time they heard about this was at the meeting rather than previous meetings or during the comment period and so Let's just take New York. There are a number of New York programs Which are smaller which may not even have an affiliated medicine program with them That that's present outside of New York as well And so that is a pretty heavy lift for them as opposed to a program like Iowa who would have close relationships for their internal with their internal medicine or their general surgery program and and that was really it it was really just just something that That that kind of came as a surprise to some of these chicks no one Literally no one argued This is a worse way to train. Everyone said this is better It's just the implementation how we get there to make sure that we don't leave anyone out along the way Just a comment. I'm stunned how long it's taken for this to go through us as we put this program in 35 years ago. I'm just amazed that this didn't really catch on for that long It's a wonderful way to get training extra training you know before people Yeah, yeah, thank you Nick like I say even among the most I'll just say vociferous opponents of Implementing this now There wasn't a single Person even there that was trying to argue that this is a worse way to train so it has taken a long time in and it does I think the biggest reason is it just hasn't had a champion to really take this and push it through academic medicine is You know we do things the way we've always done things and it does take a lot of Energy to change the inertia of how we've always done things And Yeah, so so to two final comments on that it's really interesting Until the ACG me mandated this Programs really didn't have anything they could go to their graduate medical education offices and say Other than would you help us because we'd like to do this now that it's a mandate It actually gives programs a lot of power so that's kind of an interesting dynamic that I think was was lost on several people there The other piece is payment. The ACG me is not telling you really how to do this. We could Decide as a program to pay for all four years and Decide exactly where residents go and how and how they rotate and just own four years of ophthalmology Or we can partner with an internal medicine or general surgery and love that the residents to match into their programs and hopefully have some you know, we'll just say Sway on what the residents are able to do during their training But at the very least they have to have three months of ophthalmology in their training during during their intern year That's kind of the baseline. So More to come on that and Griffin. I'll get I'll get your other comment as well here in a moment National ophthalmology surgical wet lab curriculum so what this is There are there are a number of programs around the country where Residents don't really get a wet lab experience outside of sending them to an industry-sponsored event and the challenge for academic centers is more and more though those lines The I'll just say the restrictions and regulations from academic medicine on what is allowed in terms of industry Participation and training. It's becoming more restrictive over time and and there's many good reasons for that so Many of you in residency attendings you might have you know flown down to the Alcon course that that's on there That's on the Alcon campus in Fort Worth and gone down and participated and really dug into the Alcon machines Currently Bausch and Lomoff or something where they'll fly you to their institution also Abbott Johnson Johnson will do the same and and that's just becoming more expensive for industry and Less and less residents are able to do it. ASCRS has done a great job filling a role here As well the the question to this body from industry has been Would you like to partner with us in a way that we could give all of the residents across the country an Equivalent experience so that we would know Every PG y3 entering their PG y4 year has had this set of curriculum This type of wet lab experience and and at the very least that's the baseline that we can say For an ophthalmology resident how this would look whether it's large regional wet labs like ASCRS has done or the old core program Has done or maybe a program like ours with resources We could just deliver that curriculum and do our own wet lab how that looks it really to be determined But that's going to be a heavy lift coming in in the near future That's questions a little less controversial now, but next year's meeting I think it will will be a little more controversial. All right Griffin to your to your Question comment Currently applicants are applying to greater than 80% of all ophthalmology programs So essentially every applicant is now applying to essentially every program across the country We talk about getting over 500 applications for four residency spots And that is real and that's really now the same same way across the country all programs are inundated Judith can tell you the the Herculean effort it is to try and give the dude due diligence and credence to each one Of those applications many programs just just for simple ease will say if you don't get a 245 or above We're just not going to review your application because they just have to somehow weed down the 500 applications to something that's manageable Each application that we get right now It's about 20 pages of Data reading and some of that is very dense the personal statement the letters of recommendation and The amount of time it takes to really dig into any of these applications for for many of us who have been on the committee it is intense and So we've talked about this at house that faculty before talking about different issues potential options One of the key issues right now. We use San Francisco match San Francisco matches an early match Historically the early match was done Because it was a competitive specialty. We wanted to give the unmatched people an opportunity to match into something else Now there's a couple of things that have changes and Griffin points out the integrated intern year The integrated intern year now when an applicant is applying to ophthalmology They're going to match into their intern year at the time they match into the residency programs And so one of the the big big arguments for an early match is going away And I think potentially the bigger issue and I don't know if Tara's here. I know she's on the VA rotation probably getting Tara did almost Didn't go into ophthalmology because she and her significant other Richard Going to try and match together and essentially what we're asking 10% of applicants to all Residencies right now are doing early match or sorry are doing couples match I'll just say that again because I messed it up 10% of all applicants in medicine to any residency are doing a couples match so we're now asking 10% of the the Potential applicant pool to be willing to give up matching with their significant other that that's a big big problem For medicine, so there's a lot of things that that are happening around this The the real elephant in the room is that the Academy AUPO and San Francisco match are Incredibly closely in twine together We were told for all of the kind of ideas and things San Francisco match is not going away Going to the NRMP is not an option with the rest of medicine right now And so our opportunity is we now do something with San Francisco match that might address some of these issues Thoughts comments questions I'm gonna jump into one thing which I want to allow for some discussion because I think I think if you're any jerk reaction To this is the same as mine. We could have a pretty interesting discussion for this All right a narrative letter of recommendation a narrative letter of recommendation is a traditional letter of recommendation So if you've been around long enough either on the admissions committee or certainly in retina fellowship If you get a letter from Paul Sternberg, you know what it's going to be if you get one of Randy's letters You know what it's going to be and these two right great letters They give you the information that you want they use terminology that's clear These type of letters are the exception When we get letters for ophthalmology residency, it is just a total potpourri of information And it's really really difficult to get some of the just just baseline key information that we need to get and So is there a better way? So let's just let's just present this as an option By the way, when you get this guy's letter of recommendation, you don't know what it means yet So that that's part of the the issue here Okay, and letter of recommendation at its core is a measurement tool So if we think about a measurement tool, you need it to be reliable right measure the same each time You also need to have it actually measure What it is you're trying to measure and with this as a benchmark we can see that narrative lack of recommendations Quickly run into some problems. There's incredible variability in the type and depth of information. There's lack of uniform descriptors There was a UC Irvine paper that just came out which talked about Key words that are indicators for matching an ophthalmology and if you use things like superlative or exceptional how likely someone is to match and and particularly for a junior faculty as we're trying to figure out how to write these letters We're just you know, we go look at some letters that we've seen before and that's about the extent that we have So this is called a standardized letter of recommendation This is actually available now and we did receive some application So it's going to standardize a certain portion of the information we get and then it still allows you to have a narrative portion This is what it standardizes who you are Whether we as a letter writer feel there in the top 1% 5% 10% 20% 50% We're gonna put aside the whole challenge of what called grade inflation on letter of recommendations because that's present in narrative letters It has the risk to be present here But let's just say in a perfect world. You actually got this accurate information That would be extraordinarily valuable to really understand where someone stands in this person's experience How long have we known the applicant? What's the nature of the contact? How many days you've actually spent with the applicant? What was the grade and ophthalmology on your service and they're they're These are the only questions right with this rubric here So there are seven questions commitment to ophthalmology passion enthusiasm for the field work ethic Grasp of exam skills findings level of curiosity asking appropriate questions works well with peers and co-workers Ability to communicate carrying nature to patients. How highly would you estimate the candidate will reside on your institution's rank list? the rubric is Top 1 out of 10 top 1 3rd top middle 3rd lower 3rd cannot assess So If we could get this information that would be incredibly powerful to be able to compare applicants to other applicants now Let's just pause here for comments questions concerns Acknowledging the flaws in the other way it's done presently. This seems just totally flawed to me because Hey, how do you rank somebody within a percentile when a lot of these metrics? You might have like very few data points right what the narrative allows is if your relationship falls outside of these kind of Very rigidly defined parameters You can still express what the person has done well or what they've shown if you have to answer these Like either the the the person evaluated the person might just create inflate which makes it useless or Cannot access what do you do with that if you're the person now about in these letters or people are gonna get unfairly kind of placed in these percentile scores and like the top one Five ten that was on the first question is gonna be inherently subject to till all the biases we carry with us all the cognitive biases, right? Maybe just how much we like the person how recent they experience all those things. I Don't know. I just feel like it's it it'll become a way of Sifting through applications quickly that doesn't really then there's already so many ways to do that, right? We keep trying to like box ourselves in as people into these very kind of rigid Definitions that I would hesitate to do that more so as we try to evaluate humans So a number of challenges brought up I'm just gonna maybe pull out to and make make sure that I heard this right so one We'll talk about great inflation of bias, right? So I'm gonna set that aside and I'll address that here in a moment the other is These are seven characteristics and then you're going to again have this overall What is your experience with them and then these you know? These are just kind of straightforward. Okay. Are these the right questions to ask Does this allow you to capture everything that we should in an app? I think it's a great valid question. Should these be the questions or should there be less should there be more? The more you have the the greater the risk of people checking boxes And then the less you have the less data you're going to get Let me present a challenge inherent in every letter of recommendation that we write is a conflict We have a mentee that oftentimes has done an incredible amount of work for us They they've perhaps worked on research projects. They've devoted time and energy We are invested in their success and actually there's a really compelling argument that I Had an I talked with Paul Sternberg about that as a chair of a department One of your core obligations is to help your medical students match in top themology when they want to so we will just accept There is an incredible amount of investment in that individual and then the the other side of that is we also have an obligation to our profession If I receive a letter from someone saying this is a slam dunk top 1% of ever worked with and this individual is someone who is not a good trainee who ultimately Is not going to be in a good addition to our profession Then that letter writer, right has not been at the very least Accurate and potentially not honest and so that's that's the tension that we're dealing with here We know in letters. There is great inflation and we know in in narrative and in these standardized letter of recommendation up to 30% of these will have Inflation so the question then is If we were to be it if we could get all of our letters in this format And I'll just continue on to give you a little more. Would that be better? Overall than getting all of our letters in a narrative format We had a problem in reviewing abstracts for our vote so we wanted to Have it be educational for the applicant in a way how to write an abstract But we also wanted to reduce the burden Because we get like 200 that a person would have to review So what we did and I'm wondering maybe you've already done this But we came up with guidelines is what you need in the letter What what are the program directors and the committees and residency programs looking for what do you need? What do you absolutely want to see but you still allow people to be able to add You know what they wanted about the applicant so it's sort of a hybrid I love that and did in the back. Did you hear what she said? I just want to make sure except was that was That's really brilliant and the answer is no that hasn't been done But that really is potentially kind of a middle of ground So let me let me show you what else is is in it So you do have an ability to write a narrative portion And that narrative portion allows you really two paragraphs to capture what it is You want to say about that applicant and the irony is When when we did this just informal poll asking program directors and others to raise their hands How much of the letters of recommendation are they reading it was over half that we're willing to say They're only reading the first and last paragraphs already, which is what this allows you to do a 200-word limit So this is what a 200-word limit looks like You don't have to read it Charles Barkley and something about Doug Marks But the question would be with that first and last paragraph just Switch to what's their overall percentile and a quick gestalt of what what are their boxes look like on the next part, right? And if that's a trend towards the right or it's not one five ten I'm gonna read the paragraph. You've lost any nuance in it. Why would it not go that direction? That's fair. So if I find hearing you right if we're back here and everything's over here on bottom third middle third They're probably not going to read that portion that that's a risk That that that I think that's a valid point in concern Uncomfortable with these kinds of things where you categorize people like that and for lots of reasons, right? You know, it might just be Maybe people had a bad day, you know, you only I mean, I like the questions that say How much time have you spent with this person? You know, I think that gives us information that that's very helpful, but I think some of these are No, I think it's difficult. So personally Judge if you don't mind just could you Define the problem in other words Is it that there's so many applications we need a more efficient way to get through so many applications that's You know driving looking at a standardized way. It's quicker Or is there a flaw? With the narratives, is there a flaw? it's a The core system is too cumbersome and time-consuming or it's flawed. I think I think the primary issue is this the reliability right variability can be fly and secondary it's it's just a time issue and I'll get to some some some papers published in an emergency medicine to start this a long time ago and they There's a dramatic time savings for both the writer and the reviewer That is I'd say kind of tertiary to all of this. This is the issue. I mean This is I would say number one two three Well, we're trying to quantify something that may not be quantifiable and that's that's the main issue that we're looking at here I'm not sure if any of these will get away from what we call the like war with God effect Which is everybody's above average And so if you get if you get a letter that says yeah, this guy's in the top 50% They're gonna just get tossed on the pile. So that doesn't mean anything, but everybody can't be taught 10% That's the problem when you look at these letters after gosh 10 years of Everybody was top 10% Every single letters of this guy is talking percent of this girl's top 10% And it's the Lake Wobegon effect because again, if someone's worked with you you want them to get a residency You're never going to say they're the top, you know, they're a G35 percent That's not going to get him a residency program So I don't know how we get around that Lake Wobegon effect because again every letter says every applicant is top 10% Great point in court one of the issues and you'll notice it only goes to top 50% there is no below Right to to your point and I'll just make one comment take up other Comments and we'll wrap it up. I Had two chairs tell me They want to see every letter that comes across their desk be in this format But they would not be willing to use this because they have to advocate for their students in a way That they feel this could could undermine Which again gets back to that that that real challenge that we have here We're not as a profession giving accurate assessments, right? Just objectively accurate if we were we'd feel much more comfortable saying okay This person's in the top half of students that I rotated with so that becomes a Real question for us as a professional Are we doing a disservice to ourselves by not giving accurate information when we try to get our mentees matched? I think one of the things is when you start thinking about this like in literal terms Residents, so I can only say one of them is in the top 10% today. I know which one it is, too Anybody yeah for four residents that does become interesting right because they've already differentiated themselves to get into Ophthalmology and then get into our residency, which is more competitive. That's a great point Griffin couple more comments that we're gonna wrap it up Yeah We had a hundred applications we had one quarter of the applications To go through I understand one time on each one of them. I would be more interested in that qualitative data So I feel like this is a band-aid to a deeper problem that would be Numbers down and that's I know you've already met Jeff. It's not legal Barriers Just less come and sorry I want to make sure that at least I'm Jean I Back when I was applying years and years ago I think I applied to 11 programs and I was told oh my god You're playing you know that you're crazy to apply that many programs And now I've talked to Telling me something like 80 or 90 and that's average And they say they feel that they have to do that because everybody else is doing So I kind of wonder if you know if it could do something with I don't know anything about this match But if the match program could have say maybe 20 included for a certain price And then they have to have a much higher price for any other It is a logical idea That that benefits San Francisco match only because if I have 80 Programs that everyone else is applying to I'm gonna play the programs and I don't care if it's an extra $10,000 or $20,000 My career I love it. All right, we're gonna wrap this up So Let me just run through Paper So I guess Griffin the question then is is this adequate? to comment on the Qualitative component of who this individual is because let me just show you show you a couple things standardized letter of recommendation has been shown to improve the inter and intra evaluated reliability content consistency and Of course the nature of the relationship between the rider and the applicant Emergency medicine has been doing this for almost 20 years This is this is their information 92% say the standardized letter of recommendation not the narrative 92% of the standardized is The top factor in deciding who should be interviewed 99.3% Say that it should continue to be used You don't see that type of Numbers for something when it becomes It's it's emergency medicine and it's different I Love all these comments Mean writing times just as an FYI So we know what we get from these people. We don't always know what we get from others Is this a potential step forward? Or not, I'm just gonna leave that there and turn the time over to our other speakers But thank you for your attention We literally are in the middle of some of the fundamental changes going on up Thelmaic education With with Griffin now really playing a pretty big role there as well in terms of medical student education as well So with that, thank you for your time