 Good afternoon, everyone. My name is John Hagan. I'm the Executive Director of the US ETDA, and it's my pleasure today to introduce the, and to facilitate and moderate the next panel discussion, the survey of our indoctrits, tips on compliance, privacy, and institutional data resources. Today we have Pat Green and Peter Inaudi from RTI International. And so this plenary panel presentation will explore the survey of our indoctrits. Oh, can you guys mute? I hear an echo. Peter and Pat. Can you mute your audio? Pat? Okay, the echo's gone, so we're good. This plenary panel discussion will explore the survey of our indoctrits or SED. We will begin with an overview of the survey and its uses, best practices to motivate respondents, and improved access to survey data using new data tools that can be used for analysis and planning. So Patricia Green is one of the RTI International Senior Experts in Education Research. She brings an academic background in sociology to her leadership roles on many of our major surveys and studies for the National Science Foundation, the US Department of Education, Foundations, and other major private and public sector clients. Currently she directs two surveys for the National Science Foundation, the survey of our indoctrits, which is a census of all students receiving research doctorates in the United States, and the survey of graduate students and postdoctorates in science and engineering, which provides data on trends in graduate enrollment and financial support. She's also designed and directed studies across all levels of schooling in the US and China, including the administration of international assessments in US elementary and secondary schools. So Peter Inaudi is a Director of Education and Research at RTI International, with more than 20 years of work experience in corporate and academic settings. He has expertise in survey methodology, data analysis, quality management, and systems design. Since joining RTI in 2006, he's led data delivery and dissemination tasks for multiple large federal surveys, and he was the Project Director for the National Science Foundation's early career doctorate survey. Currently, he's leading several methodological tasks for the survey of her indoctrits, including exploring the use of data analytic techniques to develop a taxonomy of research topics from dissertation titles and associated metadata. That sounds really cool. He receives a bachelor's degree from mathematics at Harvard University and a master's degree in sociology from the University of North Carolina Chapel Hill. So without further ado, I'm going to turn it over to Peter to run the slideshow and go ahead guys. And Peter, go ahead and start your screen sharing. Yes, please. Oh, and Peter, you would need to unmute if you need to talk as well too. Pat, can you hear? Yes, we can hear you, Pat. You're breaking up a little bit, but let's see if we can go forth. Peter, we're having a lot of audio glitches. I wonder if Pat needs to do the same thing, get out and come back in. It seems like it did improve it. I don't know if you can begin her presentation for her. I can try. Hold on a second. Hey, Pat, we're having big audio problems. Can you try and jump out and come back in like you did? Okay. Audience, please bear with us. So I should mention, and I should have mentioned this earlier in the session, if you have questions for the panelists, please enter them in the Q&A tab, and then I'll call upon folks along the way. And at the end of the session, if you'd like to ask your question live, that's great. If not, I'll be happy to read it to our panelists and the audience from the Q&A tab. If you'd like to chat amongst yourselves, you can certainly use the chat tab. Okay. So Pat is going to try logging in with her Mac as opposed to her other computer. So what I'm going to do is I'm going to start and hopefully I haven't practiced her section, but maybe I think it'll be okay. Welcome together. Thank you. And hopefully she can do it. When she logs in, she can take over. Okay, great. Thank you. So our agenda for this plenary is to give a brief overview of the survey of earned doctorates, talk about institution and student participation, new changes or what we've done for privacy and confidentiality and some new and exciting things we're doing on that front, and then talk about data uses by institutions and others, and then how to access your data. So that's part of that last section. So in terms of the overview, the survey of earned doctorates is a census of all research, of all research doctorates in the United States, okay, who are getting the, there are the people who are graduating this year within a year cycle that are getting their degree. It should be a census of all people with, with research doctorates. So it's not professional doctorates. It's not practitioner level doctorate degrees, but rather all PhDs, EDDs, you know, SCDs and other types of doctorate degrees. It is, yeah, oh, I can hear you perfectly. Yes, it sounds wonderful. Okay, go ahead and take over. Yeah, so we, I was just starting, so go ahead. Okay, so I think you were just talking about research doctorates. The professional doctorates are not included in the study. It really is oriented toward research. So next slide. Okay, so who sponsors the SCD? There are four sponsors. NSF is the lead sponsor, and they are the ones that are responsible for actually conducting the survey. NIH, the Department of Education and National Endowment for the Humanities also contribute to the study. And both Peter and I work for RTI International and we're under contract to NSF to actually conduct the study. All phases from, you know, developing the instruments to collecting all the data to analyzing and reporting. Next. Okay, what is the purpose of the SCD? It is to collect information on the doctoral recipients' educational history, demographic characteristics, and post-graduation plans. You know, this study has been going since I think it was first done in 1958. So it really is possible to examine trends in many areas, both the representation of women minorities and foreign nationals, emergence of new fields of study, and post-doctoral employment opportunities. We examine fields of study every other year to see if there are new fields that should be included in the taxonomy. And just aside, not this year, we changed. And we're now using the SIP codes, course and instructional program codes, rather than the SCD field of study codes, although we'll still be reporting by those codes. This dataset is used by many people, both at the institution level, often at the state level and certainly at the national level, to determine if there needs to be changes in the way money is distributed across institutions and graduate programs and to benchmark, you know, what is really happening. Next slide. Okay, so there are two ways that doctoral recipients can complete the SCD, either online via a web instrument, which is like almost everyone, and via caddy computer-assisted telephone interviewing. If students haven't responded like when they're on campus and have been asked to do it and they don't respond to our emails and other contacts, the sort of last resort is to go to telephone interviews, and that's an abbreviated interview. We no longer use the paper questionnaire at all. Zero paper questionnaires this year. Last year, we had maybe 10. And so we're very pleased with that. Next. Okay, if you want more information, this is the SCD information website. The address is up in the top corner and also these slides are all online on the website with the presentation, so you can consult that if you want to find this website. Next. So now I'm going to talk a little bit about institution and student participation. You might have noticed that I changed compliance in the title to participation because really this study, you know, it's not, there are no federal regulations really regarding it. We have no sticks, you know, it really is because people want to do this study and contribute to it. So next slide. Okay, the role of institutions, I have to thank all of you. I mean the institutional participation is really what allows the study to happen. First, most institutions make the initial contact with students when they file for graduation. And then secondly, institutions confirm which people actually did graduate with each graduation ceremony. So, you know, this is a critical activity and we really couldn't do the study without that. Next. Institutions also have a role in non-response follow-up. When we get the graduation list from the universities, we match them with the completes that we have on file, people who completed the web instrument, and then we post on the, there's an administrative tool, institutional coordinator, administrative tool, I can't. We post all the people on the graduation list and whether or not they have completed the survey. Some institutions prompt non-responding students, other ones, we ask them, the school to provide their grad address roster so that we can begin prompting. And then near the end of data collection, we ask institutions to provide critical items for non-responding students. Next. Okay, so you can see there are three times that we go to institutions to ask for information. And each time, fewer and fewer institutions and students are on these lists. Ideally, if people really responded right away, the only thing we would ask for is that initial graduation list because everyone would complete the survey. But we do go back and ask for address rosters for the non-respondence. And then at the very end, the missing information rosters for the total non-respondence. The data elements on that request are shown on this slide. It's very basic information that analysts need to really analyze the SED and also it's used as a sampling frame for the survey of doctor recipients. Next. Okay, SED non-respondent follow-up is also done by RTI. RTI prompts non-respondence via email providing them a link to the survey. If they don't respond to our emails, then we send hard copy mailings with links. And then finally, we have telephone interviewers who contact non-respondence. And at that time, they can choose either to complete a short caddy survey that really basically asks only critical items or if they prefer the interviewer can automatically send them a link to the survey again and they go on the web. And I would say it's about 50-50, you know. Some of them haven't responded because we had bad email addresses or bad snail mail addresses. And once we get a phone, they are willing to participate. Next. So we've been very successful with this methodology. Approximately 92% of doctor recipients complete the survey cycle each year. And I think, you know, people as they file to get their Ph.D., they're usually pretty darn happy. You know, it's like the end of a very long road and they're very proud of themselves and most of them are willing to share their experiences and respond to the survey. And I have to thank all the institutions because they work hard to have their students complete the survey. Next. So here are actually the response rates by school. So we have about 570 schools in the survey. It varies each year because, you know, schools can be just one school at a university or could be, you know, other universities report for the entire all the schools there. And the 60% of all schools have student response rates that are 90% or higher, which is fabulous. And another 21% have response rates that are over 70, but less than 90. So only 19% of schools have response rates that are less than 70%. And we do identify, actually we identify all the schools with less than 80% response rates and often contact them and try to figure out what we could do to help or, you know, share with them ideas for what they could do. We also traditionally until the pandemic do several site visits a year to schools that have a sudden drop in their response rates to see if we can figure out, you know, how we can help them move those response rates up. Next. Okay. Here are some of the best practices that we found. Steps to achieve high response rates. The most important one is if you have a graduation checklist, make sure to put the link to the SED on that graduation checklist. If you don't have a graduation checklist, if you can distribute the web survey link three to four months prior to graduation, you know, that would be great. And then finally, another option is to link the SED survey to your institution's exit survey, if you have one, that way you can sort of deduplicate the questions and reduce the burden on the students, make it faster for them to respond. And we will send you preliminary data files. You know, we can send them every quarter or however often you want so that you can merge that data with your own exit survey data on a timely basis. Next. Okay. And more steps to improve response rates, I think I'm going over. Monitoring student participation and providing graduation lists as soon as possible. Sometimes there are a few schools that wait until like September or October to send us their spring graduation lists. And so then when we see the non-respondents, there's really not a lot we can do. Some of them have moved far away. And, you know, for sure their, you know, snail mail address has changed. The phone had addresses changed. Maybe an email address is good, but it makes it much more difficult. So being timely with the grad lists really does help us. And we also now offer an API and application programming interface. Instead of sending copies of the certificate of completion emails that students get after they finish the SUD, you know, we send those to schools as well as to the students. You don't have to organize all of those or put them any place. This automatically pings us and we send you those names directly to your database. So it can hopefully reduce your burden. Next. So institutional participation is critical. I think I've already talked about the first several bullets, but I would like to hear from you about ways of making SED data of more use to institutions because we really do believe that if institutions use the data that they'll be more willing to participate. And second, are there tools that we could provide you to help you directly prompt non-responding graduates? Anyway, any ideas that you have as well as any questions I open to hearing from you. So, John. Great. Thank you so much, Payette. Oh, sorry. This was a last minute change. Please go ahead. I need one minute to talk about privacy and confidentiality. Please. Michael. Next. So we really do take privacy and confidentiality very serious because SED is an unusual survey in that it's a census, not a sample. And everyone earning a research doctorate in the U.S. is included. So that means this target population is highly visible. I mean, their CVs are easily online. So you could find somebody who, you know, got their PhD from the University of Minnesota in, you know, 1982, who's a female and whose field of study was in physics. And with that, you know, there aren't too many of those people. And so they are highly identifiable. And so that is why this data is protected. So these concerns require we review any data that we release goes, you know, through a disclosure review, and we take that very seriously. But NSF, go to the next slide, is also very concerned that people have new ways to access and use the data. So in a few minutes, Peter is going to be talking to you about some of those new methods. And we are trying to make it easier and easier for people to do this. So over to you, John. Great. Thank you, Pat. That was very informative. So we have a question from Deborah Charlesworth. Deborah, if you'd like to ask that live, go ahead and unmute your video and audio. Oh, wait a second for you to acclimate. Great. Hi, Deborah. Yeah, I was just wondering, I hadn't heard about the API yet. And I work with my colleague who collects the SCD data, but I'm sort of the technology person in our graduate school. Does that integrate with the banner databases? Do you know? Banners are pretty common database for universities. I think you're muted, Patricia. Right. It is. I am not sure about that, but I can get back to you with that answer. Great. I'd like that. Thank you. Good question here. So I had a question, which I think derived from your website here, over the life of the SCD, the survey has consistently attained response rates exceeding 90% from doctoral recipients, as you had mentioned. While the study is voluntary, we found that most of the graduates find reasons for completing the survey compelling. What advice can you give institutions to compel students to complete the survey? For example, what key items, a handful of items that are really compelling to the student to actually make them interested to participate or what's atop several reasons? In the communications like brochures we do for students, one of the big things is the use of the data. We point them to some of the products coming out. And the fact that decisions about future graduate programs are going to be made on SCD data. And it's the way of getting their voice and experience heard. And as I said before, a lot of people, the fact that it is the National Science Foundation that's funding this, I mean, I think they do see the scientific integrity of the data collection and so are confident that their data is going to be protected. Peter, do you have any ideas on that? Yeah, I think in this day and age, one thing you might want to talk about is diversity and inequality. Because what one of the things that the SCD does is it's the only reliable source for the number of minorities that are receiving doctorates across the nation, research doctorates across the nation in different fields by field. And so this is, if we have their responses, it really helps universities understand the availability of doctorates by race and ethnicity and citizenship and helps with planning. And disability status too. Yeah, yeah. Great, great. Yeah, I think there are many valid reasons both at the student level for why they would want their field to be enhanced, their institution to be enhanced by the analytics from this and certainly the institutions in terms of future planning, I think it's invaluable information. So a lot of that I think is up to the front lines of people to, you know, people to effectively communicate that and pass on the information from the survey-earned doctorates about these compelling reasons. So we have a question from Roxanne Trees. Roxanne, if you'd like to ask your question live, go ahead and unmute your audio and video. And if not, I'll be happy to read the questions aloud. Okay, so I'll go ahead and read Roxanne's question. We require students to complete the SED prior to graduation, but I'm waiting for the day that a student challenges this requirement. Any advice? Could a student only complete part of the survey? Yes, a student can skip any questions they want. That's on the opening screen. There's an informed consent that they can read through that says skip any question that they want. So that's certainly a possibility. We actually have had people that have completed the survey, but then afterwards regret having done it and they call and ask us their information. And we will do that. You know, that only happens once in a while. That's now the common occurrence. So and, you know, this varies by institution, institution, what their regulations are about like assessment activities. A lot of times, like in the college catalog, which is sort of the contract with the students, there is mention of participating in assessment activities or depending on how evaluations, whatever, which does give them some leeway. You know, I've never really heard of anything really becoming a legal challenge. I know that every once in a while, it, you know, it does go to the university general council and they tweak the language that maybe is used on the dissertation checklist. But so far, you know, there are a couple universities who want students to opt in, which is it's a whole lot of work for them. And, you know, it does definitely reduce the response rates. But again, I think it is just examining the language and we can send you if you what we think is good language to use. That's very helpful. It's interesting too that I think we've gone from, you know, years ago, it used to be much more considered a compliance issue. And now, as you mentioned earlier in reflecting the change of the title, it's more to compel people if they're informed and they know the reasons for why the survey is done and how that can improve their situation in their field or whatever at their institution. People are much more likely to participate. And so, you know, I think gone are the days or most of the schools say, you know, thou shalt, you must and but, you know, just say, here's the part of the normal process that we do and, you know, here's the link to it. And then people have additional guided information if, you know, they have questions or not quite sure about it. But I think these days compliance is less of an issues is more about compelling people to, you know, convince them, you know, it's the right thing to do. And I don't think at many institutions it's an issue. Janice Robinson had just one brief comment here at Brigham Young University. And we made the SED a required item to be cleared before we approve the graduation we approve for graduation. It works great. The email notice that it is completed comes to our staff. And so it's easy to clear. So Janice, thanks for your comment there. So I guess we should probably jump into the Peter's portion of the presentation. We're going till two o'clock here and we'll entertain some more questions along the way here or as well. Thank you. Thank you. Oh, Peter, if you wanted to share a movie, the icon next to the screen share, I think is it looks like a YouTube icon in the middle. I think that's actually if it's a YouTube video, you could use that. It's not a YouTube video. Unfortunately, I embedded it in the program. All right, we'll cross our fingers if you want to. Yeah, we'll cross our fingers. And if it doesn't work, I'll just describe it and we can send links to those that are interested to the tool. So thank you. So my name is Peter. And now I work with Pat for a long time on different surveys. And what I'm talking to you today about is, you know, sort of how the SCD data are used. And they're used by various entities, so researchers, federal and state agencies, as well as our universities, I would say our universities are probably the biggest users. But they're really used to evaluate the pipeline of doctors coming into the program into the workforce. They're used to evaluate to do program evaluation to make decisions about whether the financial resources going into different fields are appropriate and or the mechanisms that people use to fund their graduate students are sufficient. They are used to inform government policy as well. This is really focused on institutional uses. And we actually went out and talked to several of our respondents over the last year, talking about how do they use the SCD data. And we got the three, well, the two main uses are for program planning and institutional reporting. So generic institutional reporting about, you know, what are the demographics of our graduating doctorate holders, what are their postgraduate plans, what were their funding sources, how long did it take for them to to finish their degree, things like that, the level of debt. In terms of planning and peer analysis, though, you know, the SCD has a lot of really great potential. A lot of people, as Pat mentioned, request their own data. And so you can use that data to do all sorts of nice things. I'm actually going to go down one slide briefly to show you like this is a graph that one of the our respondents used in their annual report that looks at the funding and the change in funding for graduate students or for doctoral students between 2015 and 2019. You know, this is for their data, but it could easily be expanded to, you know, a national data set. So you could say, what is our university, what is the funding, what are the funding mechanisms we use compared to our peers. And so they do, respondents do are all of you and and other respondents do use the data to benchmark their programs by field and Carnegie classification. The other thing that sometimes is used is to look at, well, who's doing interdisciplinary research, how many people are doing interdisciplinary research, and what fields are they in. And then how does that compare to how other schools and other doctoral recipients report whether or not they do interdisciplinary research. And a final use is actually using these data as potentially sources for your exit surveys or potentially if you wanted to change your taxonomies over time. We're doing, we're constantly working on taxonomies and I apologize for the phone in the background. So looking at how NSF or as Pat said, we just started to use the SIP code, which is the Department of Ed's SIP classifications for field field of study. All right. So how do you, how do people access the data? And I'll say that there are many ways to access the data. Perhaps the best starting point is to go to the regularly produced reports. So there are a series of annual reports that are produced by NSF as well as the National Science Board. You can also request your institutions data and then there are several data tools that I'm going to show you briefly that you can use to look across institutions. And then finally, the last way is to actually ask for a data request or analysis through the project officer or through our website for specific data if you can't get it through these other sources. So the primary mechanism I would point you to is, we call it the digest, but it's an annual report. If you go to this website that's up at the top, NSF.gov statistics SED, it'll take you to this, this slide and there are two main sections. There's the reports and the data tables. The report is a, you know, it's a summary of the key topics that the analysts at NCSES have decided are the most interesting in this particular year. So it varies year by year. It has texts associated with all of the graphics. The figures are interactive so you can move your cursor over them and it'll tell you the data points. You can also on many of the figures take out a line if you're only interested in U.S. citizens as an example. And it was, and the initial slide talks about U.S. citizens and foreign nationals. It covers a lot of different things. Typically it's meant to be an overview to touch on the main ideas that are of interest to the analysts at NCSES. There's also 77 data tables that are produced annually. They cover most of the data that we collect, financial support, educational history of doctoral recipients, fields and demographic characteristics of doctoral recipients, postgraduate commitments and plans. That's one that is often of interest to universities and other stakeholders. Peter, you only have about five minutes left. Okay, thank you. So then we also produce a series of annual tables, supplemental tables that are available by request because they do not suppress any of the data in the tables. But they show by detailed field, sex, race, ethnicity, counts of doctoral recipients, as well as the baccalaureate origins of those doctoral recipients. And actually, the baccalaureate origins is one of the most requested tables. I would just like to point out two congratulately mandated reports that, again, are excellent sources of data about the state of science and engineering and about sort of the equity issues. And while these are cross survey reports, the SED is utilized in both of these every year. And so that's part of what we do is we support the authors of these reports as well. Every institution that has at least 25 doctorates over the last five years will get a institutional profile. This is sent to the coordinator as well as the dean of the graduate school. And as of last year, the VP of research. If you've not seen this report, it's a really interesting report. It shows your school and it does a peer analysis where the peers are the other institutions in your Carnegie classification. It's a six or eight page depending on the size of your school, the number of doctorate recipients you have documents. And it shows, it basically does a breakdown of many of the items that we collect and compares you to your Carnegie classifications and for many it's also for the national average. In terms of institutional data files, you can request historic data for your institution from 1920 to the most recently published year of data. This is something that you can do straight from the website. Pat also talked about getting your preliminary data. You can get that and you can link it to your exit surveys as an example as a way to add context, add additional data to your exit surveys. All these data are provided through secure transcription and with Excel and ASCII formats as well as SAS and SPSS code to read in and apply variables. For all of these data use agreements are required, but there are two tools that you can use. One, this is the NCSS data tool. You can get to by going to ncssdata.nsf.gov. Again, this goes across all their surveys, but this isn't just an example showing 2019, the distribution of men and women across race ethnicity. But you can see on the left hand side the variables that are available here. Unfortunately, it's a fairly small set of variables. So part of the reason for doing that is really because of the potential for disclosure for this population. So most of the things that you might be interested in looking at aren't available here. Mostly it's just demographics and counts. So NCSS has agreed to work on a new file called the RDS, a new system, and it's really intended to help you do analysis of a much broader set of variables. No institution level data can be done here, but you can do a broader set of analyses. It's based on a sample. And so it provides weighted estimates, a variety of statistics. It's very intuitive and fast. I was going to show you a video, but I'm not going to do it. That sort of walks you through this. It is in beta testing. And so it is something that if you're interested in doing, please let Pat or I know, or the survey know, we'll send you the instructions for getting on to the beta testing group. We're really excited about it. And so we hope you actually take us up on this. And then, oh yeah, so customer data request. So this is the last thing. So if you can't get the data that you need through these data tools or through the reports that are out there, you can write up a request and things that are typically done through these types of requests other than getting your own data include like 30 year trend analyses or there's something called the institutional yield, which is, you know, you're looking at bachelor's degrees, bachelor institutions to see of these people. So you've seen lists probably of which bachelor level schools produce the most doctor recipients, right? So but that really privileges the large schools. So one of the things that NCSS has done is they've come up with a way of using the iPad's data to say sort of what percentage of graduates of a school go on to a doctorate degree. That's the type of thing that really can't be done through the data that's available. But we could help you do that. So that's the overview. If you have any questions. Great. Thank you, Peter. So I just wanted to add a comment here at least in the time when I was handling a survey of our doctorates seemed to me in our institution, there were two lines of reporting one for the front lines person who handles, you know, contacting the students to survey and so forth and sending the graduation lists. And then there was another level West Virginia University, they had a decentralized office of graduate affairs. And those folks actually had access to the institution reporting and data and the communications back and forth from the folks that survey for doctorates. And I did not necessarily see those communications, but I think it would be a good idea for the front line front lines people, whether you're in the libraries or the graduate school to talk to the folks who could be in your graduate school or in office of institutional analysis and planning, who receive those communications and receive the reporting for that. So that they were all on the same page and aware, particularly the higher up administrative folks in your grad school, or planning office, that these resources are available. And sometimes a quick email, you know, will pave the way for that conversation and, you know, can open up, you know, a lot of other things avenues as well. Yeah, at that point, if you if you haven't seen the institutional profile for your institution, you might want to contact the graduate school dean or you could contact us directly. And I think we could probably send it directly to you. But Pat, do you think that's true? I'm not 100% because they do sign something or yeah said not to be released beyond the institution. But we could. And now our database, we don't just track the coordinator and the graduate dean, there are multiple people involved in this. And so those multiple people are in our database. And I'm sure that we could probably send it to any of any of those people. So if you want to just ask us and we'll figure your something out. Great, great. Do you all have a link publicly available link to that video? Or is it because it's in beta testing, you have to just request by email or whatever your process is. We don't have a link right now, but we're thinking about putting one up. Okay, that's cool. Yeah, let us know. And we'll be happy to, you know, even post it from our website. I think that'll be very valuable resource for folks. We can send a link to people for the their actual artist so they can go in and play with it. Great, great. Okay, it's very helpful. Just looking through the Q&A tab here, Sally Evans had a follow up to Janice Robinson's comment. Sally says, Janice, we do the same at George Mason University. I get the notification that the student has completed the survey, and I will not process their submission materials until I receive it. So interesting various levels of compliance, carrots and sticks and so forth. But sometimes it's a trick is just a trick the student into thinking that it's really required when it's technically not and then given reasons for, you know, compelling reasons why if they beg the question further. Janice, yes, we have a PowerPoint actually posted in our presentation. This PowerPoint presentation is in our presentation's web page. So you can take a look at that immediately. Can you give some practical examples of how institutions have used their SED data to analyze degree equity? Yes, actually, the custom requests that we receive, two-thirds of them really address equity issues. They're asking for data on women, minorities, you know, citizens, non-citizens, and often by field and either by like sort of fine fields and often over a long period of time to see how those distributions have changed. So I think also a lot of affirmative action officers use SED data to gauge the sort of population of people eligible to become assistant professors in a given area, you know, if there are 100 graduates in a year, PhDs in a certain field, what percentage of them are white underrepresented minorities or women so they can see what their recruiting pool really should look like and compare it to what they actually got. Yes, great. So that comes full circle from, you know, passing the students on and then they graduate to the recruiting cycle of the new incoming fold and, you know, so you can see how this all ties together. I think that's a very useful and practical example. Stacey Wallace at the University of Florida says we require the survey burn doctor. It's for a degree award. However, students can choose to fill in on what they want, which would be fine. But then we are notified to complete the missing information report. And then she follows up. It is not our policy to release student information without their consent, hence the 99% response rate rather than 100%. The other institutions provide this data on behalf of their students. Yes. And typically those missing information rosters, we only send, we only request them for people who are total non respondents for people who just didn't answer a question, skipped over it. We consider that question to be refused and we won't ask for it. It used to be years back that they did. There was a that we asked it for everybody, but that's no longer the case. It is for the total non-responsive. You know, we could never locate or they never gave us anything. Okay, great. I have another question here. Does FERPA apply to the SED? Family Education. Yeah, no, I know what it is. Yeah, because the data is also sponsored by the U.S. Department of Education. It is allowed. I mean, there's no FERPA. You know, FERPA definitely allows SED that institutions can release data directly to the SED. Okay, very good. Oh, here's a good one. What is an SED microdata and how can one obtain it? Well, the only way you can get microdata is through a restricted use license. Right now, like all that, we're developing the data analysis tools that will hopefully let people do the analysis they want. And, you know, they'll get more sophisticated over time, but releasing actual microdata files is not part of the planning, just because there would be too big of a risk of disclosure. I see. And so specifically microdata then that drills down at the fine-tooth level of, I guess it doesn't, the generalized data tables don't include those nuances or how is it differentiated? Well, the ARDAS does. I mean, it's based on microdata. Microdata is loaded in there and then you can manipulate it there. And so you can produce your table. Someday you can probably be able to produce regression and trend analysis. That's not there yet. But you can never actually see the record, you know, that has, you know, not even Peter Inaudi that would have my information, University of Michigan graduate, you know, 1989. And that it's just, again, since this population is so identifiable, it would be very hard to actually make a public use microdata set unless it only contained a very limited number of variables. And that's very useful. Yeah. So that's mainly an internal use kind of analytic. Okay. Very good. So our time is up, gang. I would like very much to thank Patricia Green and Peter Inaudi for joining us. It's been very informative. And let's keep up this dialogue. Years ago, we've had representatives from the survey of earned doctorates and it's been almost a decade. So we're delighted to have you join us. And we're happy to help participate in creating this national data set. Yeah. Well, thank you so much. Thank you for having us. You people are very important to us. So, you know, if you need something, just let us know. We shall. Thank you. All right. Take care all. Everyone join us in five minutes for the next set of breakout sessions. And thanks again. Thank you.