 Hello, everyone. Welcome to our webinar, Performance Assessment and College Admissions – How Students Show What They Know and Can Do. The webinar is being held as part of the Reimagining College Access Initiative, a partnership with Education Council and Education First, which believes that high-quality performance assessment, if organized in an easily reviewable form, could be used as an additional source of information about students' achievements and potential for post-secondary success in college admissions, as well as for placement decisions or advising. We are excited to see so much interest in the use of the performance assessment. We have over 300 people registered, from admissions officers to high school counselors to teachers, principals, nonprofit college access staff, researchers, and more. Thank you so much for joining us. Well, we give a few minutes to make sure everyone gets logged in and online. We'd love to hear from you all who have already logged in, and we'd love for you to just kind of experiment with the chat function to let us know who you are, your name, your organization, region you're from, state, or even if you attended the webinar we held last time. You can engage in the discussion in the chat throughout the webinar, just not right now. Right now we're playing with it. We'll be listing resources in the chat as well. If you have questions, please submit them using the Q&A functions. I'd also like to let you know that the webinar is being recorded. A video recording along with a summary of the webinar content will be emailed to you in a few days. The slides are currently available in the link in the chat box. For those of you who are new to RCA, this is the second in a series of three webinars on reimagining college access and success. This webinar is intended to build on our first webinar, Beyond Standardized Test, Using Performance Assessment and College Admissions, which introduced the use of performance assessment and higher education for equity and highlighted the evidence of using performance assessment for college access and success within CUNY. Today's webinar will provide you with the how to's and the lessons learned on using performance assessment in the college application and admissions process. We will conclude our three part webinar series with a webinar on July 21 that will address how to align K-12 performance assessment systems for use in higher education decision making to advance college access and success. I'm just going to look at the chat real quick. Happy to see Dan Gordon, David Ruff, Chris White, Lisa from Ithaca College, and a lot of folks, Duke from New England, so just want to kind of give a shout out to all our New England partners at Great Schools Partnership and the different higher ed organizations we've worked with up there. Just want to say a real appreciation for all of you joining. My name is Monica Martinez and I'm the director of strategic partnerships where one of the initiatives I support is reimagining college initiatives on behalf of the Learning Policy Institute. RCA began in 2017 when LPI with Education Council brought together a group of individuals and organizations simply to explore interest in and the value of using K-12 performance assessment and higher education admissions placement and advising to improve the quality and equity of these decisions by providing better information. Generally speaking, performance assessment refers to the spectrum of opportunities for students to demonstrate and receive feedback on their knowledge skills and abilities in meaningful contexts. Basically show what they know and can do, hence the title of our webinar today. High quality performance assessments are designed to surface students' facility with core modes of inquiry in the disciplines such as scientific investigation, mathematical modeling, literary analysis, or social scientific inquiry. RCA has been evolving over the last couple of years with the network of interested individuals and partners from both the K-12 and higher education sector. There were task force that were formed to shape the initiative and they made recommendations that would address anticipated challenges of using performance assessment for admissions. Today's webinar will address a few of the original RCA task force recommendations. Help admissions officers understand the value of performance assessments and how they can uniquely define and reflect accomplishments and potential contributions of an applicant. Help support a deeper admissions review process that can be used to integrate information from performance assessments and create or adapt the higher education admissions application system so all students can submit their performance assessments to a higher education institution. We are proud to be working with a broad-based network of K-12 and higher education policy and practice leaders, including NACAC and the American Collegiate Registrar's Admissions Officers, State Department Leaders from New Hampshire and Colorado, school networks like Linked Learning and Visions and New York Consortium. We're also working with the Mastery Transcripts Consortium who is supporting a network of public and private schools who are using digital high school transcript that shows in detail evidence of what students have learned and accomplished. The Coalition for College, which created an application with a digital locker that students can store their work and other application materials to apply to 150 colleges committed to access. And making Caring Common a project of Harvard's Graduate School of Education and a coalition that's focused on reshaping the college admissions process to promote greater ethical engagement. That was a lot that I said, but basically the point is there's momentum towards a holistic admissions process and using diverse admissions materials to reimagine college access and success. And we are excited to be part of this momentum and support success in the work from K-12 and higher education. Another one of the recommendations of the task force was for RCA to work through local and regional partners to pilot the use of performance assessment and higher education admissions. In states or regions where there was state policy supporting the use of performance assessment and where there's a set of diverse higher education institutions interested in holistic admissions. This brought us to New England for this reason RCA was launched with five higher education institutions that are listed on the slide with Common App as a partner given they had a platform through slide room where students could submit performance assessments as part of their application. MIT engaged with this learning group, even though they were an early trailblazer in this work incorporating performance assessment into our application process through slide room for almost seven years now. The five colleges that were part of the RCA pilot were provided with support from RCA in developing their descriptions of performance assessment for the Common App, preparing communications for high school seniors and counselors, helping consider how to incorporate performance assessment into the admissions review workflow and supporting their collective learning through monthly meetings among the piloting sites. So that is the context that brings us here today, and I'm really proud to introduce Stu Schmill, who's the Dean of Admissions and Student Financial Services at the Massachusetts Institute of Technology, lovingly called MIT. Stu will discuss why MIT began using Maker portfolios a form of performance assessment and how they integrated performance assessment and application and admissions review process. Our next speaker is Judy Wheaton. She's the Director of Admissions at Wheaton College. She will share with us Wheaton's experience in participating in the RCA pilot to include student work and their admissions application submissions review. Our final speaker, Scott Anderson, he's the Senior Director at Common App. Scott will address efforts by the Common App and its partners slide room to expand the admissions application system or platform for students to be able to submit performance assessment to be fully representative of all parts about their work to higher education. Common App has been a partner with RCA for over a year and without them we could not have piloted performance assessment with these five higher education institutions. We'll then open it up with questions and answers, a discussion among the panelists about 20, 25 minutes into the webinar. We'll conclude with me sharing some collective lessons learned from the piloting sites. And as a reminder, if you have any questions, please click the Q&A button at the bottom of your screen. If you'd like to engage in discussion, you may click the chat button and type in the chat box at the lower right hand side of the screen. It'd be great if you identified your institutional affiliation when possible. Next meal is the Dean of Admissions and Student Financial Services at MIT. Today he will share how and why MIT has evolved to a holistic admissions process seven years ago that incorporates options for students to include full-blown portfolios. Great. Thank you, Monica. I appreciate the introduction and appreciate the chance to be here with everyone. A college admissions application is designed to allow admissions officers the chance to review and evaluate the characteristics that are important to them. And one of the things that is central to the MIT experience is the hands-on learning that students participate in. And it was fairly clear to us some time ago that our application didn't really allow our admissions officers to evaluate students in that context. So if hands-on learning is important, we want to understand students that have had the chance to participate in that to allow them to show us their talents there. And the traditional college admission application really isn't designed all that well to do that. And that's where portfolios have come in. So we have traditionally had portfolios that we've been reviewing on students for quite a number of years, things like music and performing arts, art portfolios that students might submit. And those are important and those remain. But one of the things that we recognize is that there were a whole set of students who were doing projects and project-based work either in their high schools or outside of their high schools, who didn't really have a platform in our application to be able to demonstrate what they've done. So using the slide room platform that we've used for our music and performing arts and art architecture submissions, portfolio submissions in the past, we developed a maker portfolio. Portfolio at which students could tell us about these technical projects that they've done. When we first started, we essentially had the portfolio up there and it was a very open-ended question. Basically, we just asked students to tell us about their project and they could do it using text. You know, they could write narratives or they could submit photos or videos, but we weren't very directive. It was basically just tell us about it. And what we found was, after a year or two of reviewing these submissions, is that students weren't always telling us about the things that we wanted to learn about their projects. It was a little bit frustrating sometimes for our admissions officers to want to know something about the students' motivation, some of the challenges that they've faced, and not to hear about it from them. So what we moved to after a couple of cycles of this, we made a change to the portfolio submission and we started asking some more direct questions. So you can see here we ask students, you know, first of all, what kinds of things you make and we have that as a drop-down menu and I'll explain why we do that in a moment. Then we ask students a little bit more of a narrative about what they're making, but then we ask them a few other questions because we're trying to get at the context in which these students have done their projects. And that context, as you'll see in a few moments, has made all the difference for us in the value that we're getting out of these portfolio submissions. So let me explain a little bit about the process. So students will fill out this form and what we've done on our end is recruited a number. It's somewhere between a dozen and 20, so some about between 15 and 20 faculty members and alumni who are reviewing these portfolio submissions. And that's important for us because one of the things that we are finding or found was that our admissions officers weren't always equipped well enough to understand the, to be able to evaluate these submissions just didn't have the training, the technical training. So we've recruited a number of faculty members and alumni and instructors who have the technical expertise to be able to evaluate the submissions. I have to say it is enormously labor intensive and recruiting external external, you know, having faculty members do this is it's a heavy lift, both for us and for them. But one of the reasons we ask this drop down menu of what kinds of things do you make is so we can actually direct the portfolio to the faculty member that has the particular level of expertise, you know, in the different subject area. So if it's a robotics program or software or if it's a textile work or a chemistry project or some other kind of project, we can direct them to the experts in the field. So what we also do is spend a fair bit of time with the reviewers, calibrating them on what we're looking for and what we want them to comment on that's going to add to the admissions application review. And I want to go through I have a couple of examples of things that we get from the reviews that I think illustrate why they're so valuable for us. So. Okay, so here's the first first slide. This is one particular students portfolio. And you can see that the projects that they've decided to talk about in their submission came from some of their classwork. There were classwork submissions, either a direct assignment or some extra credit assignments that they were particularly proud of, and that we could therefore review. And you can see how our this is a quote from one of our faculty reviewers that added to the admissions review ultimately is, you know, they're looking for signs of motivation, grit, resourcefulness, problem solving, self teaching ability, things that really are going to signal that these students are going to be very successful students at MIT. Let me give you a couple more examples. And this is just a couple of more excerpts from reviews that we got. And these are the kinds of things that are coming out in in the reviews and these are the things that we're learning about our applicants that I don't think we had a real place for them in other parts of the application. And, you know, these are students who are, you know, good students, but they really shine when they're doing project work. And, and that's the kind of stuff they're going to be doing once they're in college. So our ability to evaluate them is really important. You know, things that we value this collaborative process, again, resourcefulness willingness to take risks. Even their communication skills, you know, their ability to explain what they're doing. And here you can see, again, it's a lot about context. So I think that's just an overview of our process. Maybe I'll leave it there. And then if there are questions, we can come back to that at the end. So Monica, back to you. Thank you. That's great, Stu. It's almost like it's just a trailer, right, for a movie, because you guys have been doing this for so long and you have so many lessons to teach us. But you did get a comment on that chat that somebody said they really like the question you ask about what's the most meaningful thing you have made. And I really appreciate your focus on evidence of a learning and expanding admissions process. We're now going to move over to Judy Purdy, who is the director of admissions at Wheaton College in Massachusetts. Judy has been working at Wheaton for 34 years, and every year she continues to improve on their admissions process. Today, Judy will share how and why Wheaton decided to use a form of performance assessment in their admissions process. Judy. Thank you, Monica. Welcome, everyone. And so what I want to be able to do is tell you a little bit about what I'm about to talk about. Just the context and background about who Wheaton is, our history list portfolios, and our partnership and goals, the process and the value. There's a lot to this, so I'm going to go through some of it very quickly. Just wanted to have you have a sense of who we are. So that's Wheaton College in a nutshell. And then historically, applicants to Wheaton have been able to submit portfolios for faculty review in various departments. So when RCA reached out to us, it was a very easy decision for us to say, yes, we want to do this. We knew that we would get to know our applicants better. It would allow students a little bit more of the traditional provide context to their applications. And we did one other little thing with this. It's not only did we want to learn about the students during the application and some of the things that they do, we very much use this as a yield opportunity. We are one of those colleges that I would say I feel very lucky and very blessed to work with the faculty and staff that we work with here at Wheaton. Our faculty never hesitate to work with us and to be able to want to reach out to students. So when we first started the process part of the prompt was working with everybody. It was an optional Wheaton admission portfolio because we've already been working with Slide Room. So we already had students who were submitting portfolios about music, theater, dance, visual arts, writing, and we were always sending those to faculty members who would assess them. And the faculty would always reach out to those students as well. We also use some of that once students decided to enroll or as part of the yield where we would write to them and let them know how some of what they were doing would contribute to the kind of life they would have at Wheaton. And faculty used it as an opportunity to say this student, very similar to what our coaches were doing, would make a huge difference in my department if we can admit that student. So when we were looking at it we had a working draft and so we have now changed it. What we learned the most about the whole process is that this we wanted to make sure that students understood the difference between a traditional portfolio and what now we were asking for with this whole new approach with RCA. So a lot of it was, again, we wanted them to show passion for subject area. We were now trying to create additional work. We were trying to give the students the opportunities who had already through their high schools and through some of the schools, especially that RCA was initially telling us about that it done some of these projects. It was seen on their transcripts that just said an internship or their senior class project, but we didn't know a whole lot about it. And they didn't have a real way of being able to show it. We didn't want them to have to create something brand new just for admission to make even more work as much as to be able to take something they'd already done. Where they could show us their leadership skills or their expertise that were developed through an internship or job, commitment to the community through a community service portfolio, unique skills, something else that they did through a short video. But again, it was mostly about creativity, collaboration and passion, what they really wanted to do. What we found last year is that we didn't ask the right questions. We didn't prompt. We were brand new to this. We knew how to do portfolios. We didn't really know how to ask this kind of question. So a lot of times the submissions we got, we weren't even really sure what they were. We weren't even really sure what they were trying to tell us or what we were going to learn about them. And we didn't always know what their role was in this work. Through a lot of meetings with our colleagues in through RCA, they helped us to really think about how to talk differently and communicate the information that we wanted them to submit. So we changed our portfolio site so that in slide room they could see the difference in what they were doing in the traditional. So what we didn't want is somebody who's putting together a music portfolio or an art portfolio of just lots of different work. That will stay the same. We didn't want them to change differently. We wanted it to be the students who had created a particular project. So we were more project based or more community oriented based or something brand new that they had done through school through their classroom. So that was a little bit of the difference of what we were trying to create in here. So they were now called supplemental materials. They focused on qualities and skills that they could convey other than just simply listing them. So examples of submission, types, requesting brief student reflection, these are the things that we made the biggest changes for. And we believe that what we will see this year will be even more impressive than what we received last year. And a lot of it again was us learning how to ask the correct and appropriate questions and give better direction. So we had the reviews were, they were 19 to 20 that we did during one small thing. There were two to three person committee review. This was done just with leadership. It was me and a couple of the associate directors. So we started to do that and we because it was brand new, we wanted to first figure it out ourselves. Moving forward, we're going to have the entire counseling staff to review these submissions during the application review. Most of them only take a few minutes to review, but they are only used to help the students. They are not used to decide who would be admitted and who would not. It's an extra texture. It's a little bit more context, us getting to know the students a little bit better. And again, using it for yield and using it for how we could help them once they enroll at Wheaton to achieve the goals that they wanted to be able to do. With the review with the staff, what we discovered as we were going through this, unless we were also looking at their full application, we didn't have enough of a feeling about who the students were, which is why we're bringing it back to the point of looking at it as you are reviewing the application. And we will make sure that we're all doing it consistently and all doing it the same way. So we'll have many opportunities to come together as a staff. I'm going to show you a few examples. The first one was a 12-page research paper. It was a submission. It was very strong because it was a research. I'm not going to read through this, but you can see from looking at it that it was pretty much a scientific type of thing. But it was well done. We enjoyed reading it. We learned a lot about it. They told us what it was, et cetera. The second example was, again, you can see that they created something that was called a Thousand Splendid Suns. It was after they read the novel and the teachers asked them to do something as a creative partner-based project. So again, it was a little bit from reading it that they created something. And this is something that the students submitted to us. It was easy for us to evaluate it because they had basically told us what it was they were doing. The third one was an exceptionally produced five-minute video. And it was somebody who was telling us about what they were doing. They partnered with a rotary club. They had the privilege of going to study abroad to meet some of these amazing women. So again, they told us a lot about this. Obviously, I'm going through this all very quickly because you have the opportunity to go and look at all of this later. So this is the team I worked with. This is the Wanda Sorrell, who's our Senior Associate Director. And Jay LaPierre, Associate Director of Admission. We were the ones, and Jay and Wanda have done a lot of the work as we are now changing our website and changing the verbiage and the wardage that we are doing in the Common App. So with our team, and then they will be taking on some of the leadership as we then train the rest of our staff. So thank you very much. And I will now hand this back to Monica. Thank you so much for this, Judy. And I think what Judy and Stu both shows, they're constantly at their institutions iterating on the process. A few questions have come in about, are these supplemental? And right now for MIT and for Wheaton, they are supplemental submissions. And Judy has really helped us learn a lot about the use of language too. So, you know, I've seen a couple of questions come in about the word portfolio. And as you guys are familiar with, a lot of our institutions, architectural design companies, they ask for a portfolio of work. And it's very specific, like what Stu was talking to a certain skill. But we're talking about a broader set of student work that can go into a portfolio that shows students strengths and how they can apply their knowledge and their skills. So we'll turn back to that a little bit more, but I want to just provide that clarification since I saw a couple of different questions come in. But right now we're going to turn to our next speaker, who is Scott Anderson, and he is from the Common App, where he serves as a senior director for access and education. Scott supports partnerships with educators, researchers and designers to create services, programs and resources that enable student success. Scott is the lead for the RCA partnership and has been instrumental in this piloting work. So thank you so much, Scott and Common App. I'll turn it over to you. Great, Monica. Thank you so much for having me here today and thank you to all of the attendees for joining us. Just trying to, there we go. Oops, advanced the slides. I want to talk a little bit about Common App and who we are because I think it's really important to understand why RCA approached us to be a potential partner to advance the work that they are doing promoting performance based assessments and the college admission process. Those of you who know Common App and webinar of educators, I'm sure many of you have heard of us probably know us for the tool that we have which is an application to apply to college. But what many people don't know is that we are actually a not-for-profit membership association that is mission driven and our mission is the pursuit of access, equity and integrity in the college admission process. The membership of the Common App represents the full diversity of higher ed, four-year higher ed in the United States. We started in 1975 with 15 small liberal arts colleges. And we over the years have grown into an organization over 900 member institutions, public and private, single sex and coed, urban, rural, large, small, religious, secular, you name it, you can find those institutions in our membership. Collectively, those member institutions direct to the Common App over a million applicants each year. The vast majority of those are first year applicants, although we also have a transfer application and that population represents about 10% of the pool. The applicants very much like the membership itself is extremely diverse and perhaps one of the most important statistics that we like to point out is about a third of the applicants who use the Common App are the first in their families to go to college. That percentage is a little bit higher for the transfer population than it is for the first year population, but together it represents about a third. And collectively those students are submitting over five and a half million applications each year. I put these numbers out there for context. What is important is not the numbers themselves as large as they are, but rather the reach that they represent among the college going population. And that I think that along with the diversity and number of members Common App has is what led RCA to reach out to us to begin with. One other aspect of Common App that is really important to point out and speaks to the mission of the organization is that as of January 1, 2019, former first lady Michelle Obama's reach higher initiative is a campaign of the Common Application. They joined us because they recognize that membership or rather membership but a presence within the Common App would be a way for them to advance their own mission of building confidence and inspiring students to think about college and to recognize that there's a the college and post secondary study in general can be something that is attainable for them. So if you think about the confidence that Common App that reach higher inspires and students Common App was able to take that confidence and channel it into the tangible tangible action of applying to college. Slide room has been mentioned a few times on this message, or rather on this on this webinar, and wanted to tell you a little bit about the work that Common App does with slide room. We've been partners with them since 2013. So for several years now, we partnered with them as a means of helping our member colleges gather from their applicants supplemental information that went beyond just writing and certainly grades and test scores and other things that you would find in an application. We supported a portfolio we wanted to make sure that we were able to help our members who wanted that the slide room partnership is important to our member colleges because it allows for a seamless integration of the students Common App and the slide room accounts and I'll show you what that looks like in a moment. It also that partnership supports a lot of flexibility for our member colleges who can receive the portfolios that students are submitting through the Common App as as data, they can get them through our automatic process that they receive on a daily basis, and and a tremendous amount of flexibility, which allow us and not just to get the information but then also facilitate the review. One other thing that's really important is that Common App has a fee waiver built within it and any student who indicates eligibility for the Common App fee waiver. That fee waiver also carries over to slide room as well so the students don't need to pay a fee to use slide room individual institutions can choose to pass that fee along to the students if they wish that but the vast majority do not. Judy talked a little bit about the language that we can has used and how it's changing within the Common App. And I wanted to show you what it looks like from a student perspective so you have a sense of that. Here you see last year's or I should say the current year language that we use and there is this explanation and then a yes or no questions you intend to submit an option of an admission portfolio and the student answers yes over on the left. A student will see opening up for that particular college, the instructions for a portfolio. And I should say here that each of our member colleges has the ability not just to gather the information that's collected within the Common App proper but also within their supplemental information something that we call the member screen and the portfolio would be part of that member screen. If a student were to click on instructions, they would be shown a login page or screen rather where they could enter their slide room credentials if a sliding account already exists for that student or they could create their slide and recount there. And then they would click login or sign up and they'd be directed over to slide room. In fact the on the bottom here and I think this is going to be put in the chat as well you will see that the link where rather a URL, where you can see from both the college perspective and also the student perspective, what that integration process looks like I won't go through the steps of that here with you it's just important for you to know that it's a seamless process. So RCA approached Common App. Last year, recognizing that given the reach that we have among institutions of higher education and the reach that we have among college going students that we would be potentially a good partner to help advance the work that they were trying to do with incorporating performance based assessments into the admission process. It made a lot of sense for us it was building on existing integration integrations and familiar tools that that students and that colleges themselves were using. But more important than that it really resonated with us because of the mission alignment so going back to how I started here, we are a mission driven organization and the main tenants of our mission or access equity and integrity. And so we are always asking ourselves how we can use our reach to help members collect meaningful information that assists them engaging potential for student success. As we think today at this moment in particular about how the sands are shifting underneath the feet of students and in terms of whether tests are going to be required or what kind of information that they are going to be able to provide to colleges to help give appropriate context for their academic work and their lives, both in and outside of the classroom. So we saw RCA and the work that they were doing as an opportunity to help our members, if they were ready and interested in doing so, gather that information from students and to do it within the framework of a common app which is a tool that everyone is familiar with. So that is some basic information about how Common App has worked with RCA and Monica, I will pass it back to you and we'll be happy to answer any questions that might come up. Yeah, no thank you thank all of you and we're going to move into our discussion section. So we're going to put everyone on screen, a lot of really good questions coming in and we're not going to be able to get to all of them. And while we transition Scott I'm just going to ask you a real quick follow up question from from you know you guys have 900 members so what are the skills and competency areas colleges look for from students and how do you get that information. Oh, you're on mute Scott, but I bet it was great answer. It was yes, you know presenter spoke on mute you can all check that off of your bingo cards now for resume. Yeah, what I was saying is you know we do have 900 members and if you ask them you get 900 different answers. I think one of the benefits of membership in in the Common App is the ability to marry shared information that students present through the main application with the institutional specific information that is really important to the colleges themselves. You know the question if if if I before I joined the staff the Common App as a counselor and I would often tell my students you know if you want to know what colleges care about, look at what they ask on their admission applications. Certainly is is true of the maker portfolio at MIT and and and what Judy has pointed out for what you know the sort of the range of things that that that would be really interesting to we find those exact same things at Common App member colleges across the spectrum. Yeah, I really appreciate that. Thank you. And it's very similar to the conversation I had with Stu as well. Building on what Scott just talked about and both Stu and Judy you both touched on this a bit. What, what did you learn most about students when you started accepting the maker portfolio and student work. Maybe I can start and then Judy after I would say the thing that we appreciated most about this portfolio work is it allowed us a fuller picture as to the students we were evaluating. You know the college admissions process is it's imperfect in that it's a, we're trying to learn about students through this medium of you know looking at transcripts and essays and evaluating what they've done and letters of recommendation I mean we're getting as full a picture as we can but it's still really incomplete. And I know, I mean most students will tell you that they are not the, you know that they're, they're not their college admissions application they're more than that and that oftentimes students will feel like they can't really it's hard for them to get across what their talents are and and what makes them special and I think the portfolios is one more avenue for students to be able to express that. So I think that is something that we've really appreciated about it. Judy, would you like to build on that. So I think what we understood is a little bit more about a little bit more with the texture and who else they are and what their passion is. We didn't get enough of that just through the application itself or even so once we started to see portfolios we could feel it and we could sense of the things that they really had strong interest in. And you could see that it made more sense to also see why they're in their transcript. It looked like they had a little bit more emphasis on the arts or a little bit more emphasis on writing or it helped to explain the classes that they were taking. And maybe they didn't like their essay about that but we were able to see it. So I think we just got to see a little bit more of the holistic view of who a student was. It also explained at times what a counselor was talking about when they would say that a student excels in some particular area and the student never told us that they sang on stage or they were part of theater. Because they didn't feel like they had the right place to do it in the application but then you would read all about it from a counselor or teachers. So I think we just began to figure it out and learn a little bit more about the whole student. Appreciate that. And so what are the challenges about accepting performance assessment for admissions from a process point of view. The question came up on our last webinar we had CUNY you know which is one of the largest systems in the United States for higher education, but people are very concerned about how do you go to scale with this right. So what are some of the challenges about accepting performance assessment for admissions from a process point of view. And Judy since you're on the phone let's start with you and then we'll go over to Stu. And if Scott has anything to add to from common app. I would say is that we do pretty holistic view anyway so it might add a few more minutes to the process, but not anything extensive. And actually what it does is it helps to clarify it we don't end up with wondering so much. And we do a lot of communicating back and forth with students but I think that for us I see it all as a positive. We are not one of those schools that is trying to read 60 applications in an hour we are we're trying to read 35 to 40 applications in an hour. So, I mean, in a day not an hour I'm sorry. So to be able to add this extra time and texture and not every student is doing it. And I think that it all it can do is enhance it for us and make us better at understanding the students and making it help us, especially for the students who are on the fence and we are on the fence and just thinking, what is it special about them this will help to sway some of the conversations as we're deciding whether to admit or waitlist deny etc. And we never use it as anything but in the positive. Thank you. So do you want to add to that and then we have a couple more questions we want to move to. Yeah, we, I guess because the nature of our portfolio which is a little bit more specific towards prod technical projects a little have to be technical but basically hands on kinds of projects. Outside reviewers that are reviewing them in the same way we have music faculty that review our music portfolios we have our engineering faculty or science faculty reviewing some of the submissions that students and, and so it's a lot of work to coordinate all that it's a lot of work for for those faculty members. It's a lot of work to coordinate and calibrate the faculty members to make sure they understand what they're looking for. And therefore it's not that much more work for our admission staff because we have, except for the one person who works with the faculty group. One of the things that we found is that for these kinds of projects are admissions officers were not really able to fairly assess the work because they didn't have the expertise in those areas and what in what an admissions officer thought was really neat project might have been a relatively standard one and something that one of our admissions officers thought was kind of boring. We would have faculty members jumping up and down saying oh my God this is the most amazing thing I've ever seen. And so that's, I think that will make it hard to scale and that needing to get some kind of expert review. I'm not sure that that is always necessary in every case but at least in some of these. Figuring out that review, I think will be important to allow this to scale more broadly. Yeah. Yeah, and we appreciate that we get to learn from you and part of LPI's, you know mission is to conduct research and we're going to continue to garner and gather as many of these lessons as possible to help folks understand the systems and structures that have to be in place. I'm really excited to see that like we have a lot of questions about equity that have been coming in. So I'm going to ask kind of a two part question so one's going to kind of be the upside around equity, the potential upside, as well as some of the challenges around equity. So the first part of the equity question is how do you feel performance assessment can support admissions policies and practices around issues of access and equity, and then we'll do the flip side after this and we just have about five more minutes or so. I'll chime in on that. You know, I didn't go into depth when I was speaking about our mission but the concept of equity is incredibly important. We, you know, it's interesting. The Common App is an online form and but there's a PDF version of it and if you look at the front page of that today's PDF and you compare it to the front page of the first application from 1975, they are almost identical. And a lot has changed in those years. And it really, I think is incumbent upon all of us who work in this in this profession helping young students get to college. And especially those of us who are orchestrating things, if you will, in the admission office and in organizations like the Common App to try to figure out how to help students see themselves in the application. And the flip side of that is how to make sure we prevent students from not seeing themselves in the application. In other words, we don't want students to encounter aspects of the application process that said signals that they're not welcome here. It really is all about being welcoming. And at the Common App, we're going through a process right now where we're really trying to put an equity lens on the entire application itself to see who is included, who is excluded, whether it be actively or inadvertently, to make sure that the tool that they're interacting with as they're trying to convey who they are as whole people to colleges like the ones represented here to make sure that they've got that ability. Great. Thank you so much, Scott. So Stuart or Judy, how do you feel performance assessment can support admissions policies and practices around issues of equity and access, Judy? Yeah, so what I would say is that it's really incumbent to on how savvy the student is, whether especially for first generation, if they're not getting help from their counselors or their teachers. So with us trying to promote and send out messages to, especially on our website, encouraging and explaining to students what these mean. If a student has been very interested in dance or really in the visual arts, I think it's a little easier for them to understand how to go to the slide room and to present that. For us it's more of this other one where they've done the projects and that they've done these senior internships or projects that they're not really sure how to showcase those. And so sometimes for us, again, that's why we spent so much time working on our language to help explain to students this is the opportunity for them because if you're at a school where it's not an inner city school, you've got a lot of connections and a lot of contact. Your families know how to help you with this. It's easy for them. It's for the students who really don't understand how to explain something that was really incredibly meaningful that they did and to be able to present that. So I think the more we can do Monica to help especially on our website and also through the language on Common App. That's incumbent upon us to make sure that those students have the access or at least know enough to then email us or call us and ask how they could present this. Thank you. Yeah, I really appreciate the message both you and Scott are saying basically how do we signal that this this is what we want we want to know more about you so thank you both. Stu, how about you wrap up this question and we'll probably move on to our final. Yep, just completely agree with both what Scott and Judy said there. And in fact it's one of the reasons why we added more contextual questions to the portfolio to try to draw some of this out, because we don't want to simply be rewarding wealth and privilege and resources that students have we want to be rewarding the talent that students have. And yet you have to understand the context to get at the talent right and, and it's not just the resources that pile on to that you want to strip that away. But instead, it's still true at least what we're finding is that students with more resources are more likely to be doing these projects and I think some of that is just for obvious reasons that they have more opportunities. Some of it also is directly related to what Judy said around our ability to communicate. I think we have an issue with low income students. First generation students and even women, we find women are less likely to submit make portfolios, even if they are have projects that they have done. And that would fit this very well. There's just a higher barrier and we're trying to figure out what that is. So I just want to, you know without going on too long because I know we're going to need to wrap up to just acknowledge this is an issue that we're thinking deeply about, and we haven't solved it. I think there's potential, but there's more work that we need to do there. That's a great, that's a great sentence to end on Stu. Thank you very much. And it really is at the heart and soul of what reimagined college access and success wants to be about is to ensure there's equity. So we're going to have to wrap up the Q&A and I'm just going to move into some closing comments given we just have a few more minutes. So you can see by by this slide that just came on your screen. You know we've certainly learned lessons from folks like Judy and Stu and the other higher institutions that participated. And really it's like you know what you're you have to know what you're looking for to develop the right ask. And we can go into detail about what performance assessment is and right now these are some forms of student work. So we have to know what you're looking for to develop the right ask. And then what you're hearing now about admissions counselors and everyone else they have to be proactive in cultivating relationships with K-12 schools and their counselors that use performance assessment to strengthen the pipeline. Within these schools performance assessments are just themselves a learning tool that builds students abilities to apply knowledge and complex problems, while also helping students develop co cognitive skills, such as collaborative collaboration grit resilience. And we also have to consider admissions as only the beginning. You know I had a lot of conversations with Judy where she talked about how she was going to use student work in the yield and how to connect students to other people. So what are other implications. And then what you've heard from both of them is you have to anticipate iteration that you're not going to get at the right the first time you're going to continue to develop structures and systems for this. And we're learning with these folks we know this work is nascent performance assessment has been in K-12 schools for some time and this opens an access for other schools to be able to use performance assessment and develop the kind of teaching strategies for critical thinking as well as other skills. So I'm going to move to the conclusion and as you can see we're just starting to understand and develop these structures and systems and processes to be used. In addition, you know evidence is now starting to emerge about the potential use of using high quality performance assessment and admissions. Next month RCA will publish a study assessing college readiness through authentic student work how City University of New York and the New York Performance Standards Consortium are collaborating towards equity by Michelle Fine and Karina Priyomka. They're both from the CUNY Graduate Center and essentially the pilot program that use performance assessment for students who do not meet the cut score on the college entrance exams are successful in their first year of college outperforming their peers in the first semester grades and credits earned and persisting after a year at higher rates than other students. An embargo draft of this report can be found at the link in this chat. We have to find a way to ensure we have a pipeline of schools where students produce high quality performance assessment. If you'd like to learn more about or begin to pilot performance assessment, please do not hesitate to reach out to us. We're developing a tool to help higher institutions consider how to use performance assessment information in ways that are both meaningful and realistic by helping them connect purpose to performance assessment and artifacts. I want to thank you for coming on this last side. You can see that we're going to have another webinar. We hope you will join that. You can also see where we have our different resources. We will publish a summary of this webinar in addition to a video recording. The slides are already on the on our website, but the next webinar is actually going to start focusing on some of the K 12 questions that came through. How do you best capture students work so colleges can see it and use it. How are colleges getting this information about using performance assessment out to parents and students. How can high schools calibrate with higher education so that we know our students are leaving with the necessary skills. Thank you so much for taking your time today to join us.