 This panel is on aligning supply and demand. There's obviously a lot of credentials out there. So we're going to talk about trust and verification and what credentials are what. I think my panel is ready to come. We're going to have the moderator. It's going to be Dr. Donifu Elston, who's a VP of strategy of Complete College America. We're going to have Nathan Anderson, Senior Director for Jobs for the Future, Amy Heitzman, Dr. Amy Heitzman, who's a Deputy CEO and Chief Learning Officer at Upsia, and Kyle Trigg, who's CEO and co-founder of Moonshot. And they're going to come up. Let's welcome them, and they'll take it over. Good afternoon. I get the pleasure of moderating this panel of dynamic individuals. And in just a few moments, I'll allow them to introduce themselves. And we'll talk a little bit about some of the amazing things that are taking place in this larger space around credentialing and within institutions and within the technical environment as well. My name is Donifu Elston. Work at Complete College America, where I serve as Vice President for Strategy. And Complete College America, for those of you that are not familiar, work with state systems across the country. We really kind of started off our work focusing on this larger guided pathways movement. And so right now, about 35 plus states, number of consortia, all working together to ensure our primary mission. And that mission is a very simple one. We want to ensure that more individuals across this country receive either degrees, credentials of purpose and value. And so that looks very different depending on the populations that you're speaking about or the institutional type. But that's our large mission. So it provides quite a bit of space and I think quite a bit of connection to this current work. The one thing that I will share with you is that this work for me is extremely important because, of course, when I think about how people make decisions about majors, it's a very personal one for me. I grew up in Southern California. So I was into water sports. I was good in science. And so with all that being said, I should be a what? A marine biologist, exactly. I was going to be the African-American Jacques Cousteau until I fortunately had a chance as an undergrad to go on an internship with the National Oceanic and Atmospheric Administration. And in my mind, it was so exciting. I was thinking about being in the Gulf of Mexico. And when in my mind, I just was thinking about sun and surf and all these other things. And I will tell you, that experience sucked so bad. And no disrespect to any marine biologist. That may be in the room. But what I realized very early on is that we don't even have a conversation with students on the front end about what it is that you can do. What is it that you want to be? What is your purpose? And how does it align with workforce and labor market demands? And so as we think about this guided pathways movement, we are now realizing that an important step in that process is to put and ensure that the front end of it begins with purpose first. And so we're very fortunate to be working with Strata Education Network on this idea of assisting more students in identifying they're making that big major decision with purpose in mind. So thinking about real-time labor market data and understanding that career counseling and advisement happens early and often. And that it should include conversations around values and skills assessment, et cetera. And so this is the big idea that we're working on. And I think there's quite a bit of alignment as you'll hear from some of my colleagues that I'll introduce or allow them to introduce themselves shortly. And so that's the exciting work that's taken place. We're working with five states and a multiple number of institutions to really identify how do you operationalize that process and embed it in this larger guided pathways movement. And so with that, I'm going to go ahead and jump into a conversation with my colleagues that are here. And we're going to go ahead and just talk a little bit about this work around credentialing and the economy. And so what I'd like to do is have each of my colleagues starting on the far end just do an introduction of who you are and how you think this work and your particular work is aligning with the spirit of the credentialing summit. Definitely. Yeah. Thank you for the intro and thank you all for coming and allowing the opportunity for us to speak today. My name is Kyle. I'm the co-founder and CEO at Moonshot Insights. We perform predictive candidate evaluations for employers. So we evaluate candidates to predict their job performance, their growth potential, the longevity at a company, and culture fit. So our unique perspective really brings to bear data that we have on internal companies, their employees, employers, to identify predictors of success, which then can be mapped back against candidates who are coming and looking for their career opportunities. So that's really some of the insight that I have here is from a data perspective. I'm a data nerd and I love to look at patterns with vast sets of different data to figure out predictors of success are. So then we can communicate back to candidates the guided pathways for them to succeed and find the highest leverage potential position for them. But at the same time, we speak on the employer side as well, understanding the business outcomes that they're looking to drive, the culture and work environments that are conducive for productivity, pushing their products to market, and being able to bridge that gap. So interesting perspectives we have are really informing employer demand. So we have positions that we evaluate against and oftentimes we can find patterns and predictors of success that employers may not have thought of and obviously we're not included in their traditional resume screening and an interviewing process. And as Michael Saylor brought up earlier, there's opportunity for a standardized credential system for pre-hiring to understand fully that process for candidates and that progression into the workforce. So that's some perspective we have as far as connecting the dots for candidates on the supply side and also informing for employers where their demand actually lies and how we can bridge that gap between those two folks. So. Kyle, thank you. I find that fascinating and definitely be talking. Yeah. As we have already. Absolutely. Good afternoon, everyone. My name is Amy Heitzman. I'm the Deputy CEO and Chief Learning Officer for UPCEA, 104 years young. Our members, thank you, my friends, are the standard bearers for non-credit and professional learning on campuses. And so to this end, our members are on the forefront of what we often call alternative credentials. While many of our members are involved in traditional degrees, such as bachelor's associates and graduate level degrees, most of our membership is really involved in the kinds of things that we now refer to as micro-credentials, nano-credentials, badges, stackables, or as my colleague Jim Fong would call snackables. We'll get to that. Yes, yes. To this end, our organization has created a series of deliverables. And many of these, if they're not in the app today, they will be on some sort of site posted later for you to download and review. We have two bookend pieces. They're called Hallmarks of Excellence. They're inspirational, aspirational rather, institutional benchmarks on online learning and professional continuing education. We've also done great work with Credential Engine. Hi, Scott, you're here, I know. I sit on the Higher Education Advisory Board to make sure that our organization understands the value of a credential registry. That's important in terms of the transparency work that we've been talking about. We also have a Center for Research and Strategy within Upsia and we've done several pieces to date with Pearson, with Credly, with Penn State World Campus to recognize the landscape of alternative credentials. What are these? What are some commonalities? What do we call them? What are these techniques and how are they leveraged in the workplace? But more specifically, how are they communicated authentically and transparently so that employers know what competencies are denoted by said credentials? We also recently convened, and I think I see some friends in the audience who were with us last October, November rather, we convened a group to discuss the future of alternative credentials. What is this landscape? Who's involved? What from the employer side, the practitioner side, the student side needs to be involved in this conversation to make sure that while bachelor's degree might continue to be the coin of the realm, that won't always be the case. And how are we making sure that this whole portfolio of learning and credentialed learning, whether it's on a college campus or externally, is recognized, validated, and then leveraged for the betterment of the student? As demographies are changing, and we'll talk probably more about that throughout the next day or so, our members, CNA members, are working to bring alternative credentials, this whole universe into play in a much more formal, much more authentic, much more trusted, verifiable way, as well as the recognition of all learning, whether it happens on a college campus or not, but to the forefront. And it's essential for society, it's essential for workforce development, but it's also essential for the members that I represent. We are leaving a lot on the table if we are not recognizing that students need more than just traditional degrees. It's a lifetime of learning, and we need to be prepared for that. Thank you. Hi everyone, I'm Nate Anderson from Jobs of the Future. Also a recovering would-be marine biologist. I'm a harsh reality that it's not all dolphin training and discovering new species. So I work primarily at Jobs of Future JFF, work primarily on issues related to equity and economic mobility for low-income populations in the US. And we as an organization have committed to a mission that really tries to address these issues of what are the career pathways and career pathway support programs that we can put in place that allow institutions and the individuals that they serve to ultimately achieve the kind of career goals that they've set for themselves. And when you look at the data on what happens with low-income populations in this country, it's pretty stark in terms of their ability to achieve those goals, to escape the kind of cycle of poverty, and to gain the learning that they need and to have that learning recognized in the employment marketplace. So personally I manage sort of an intersection of three different areas at JFF that I think are really relevant to this conversation. One is our demand side engagement in analytics. So what is the labor market telling us? What are employment engagement strategies that are effective? What are institutions doing in the field to really get employers to the table? And what are employers doing to get institutions to the table? It's the future work side of the organization. So really looking at how things are changing and how we can prepare students for those changes. And that will involve significant changes to the way education and workforce training is delivered in this country to be able to account for that. And then third, it's the credentialing strategy group. So really looking at what alternative credentials are there to the four year degree and the two year, four and two year degree that are accessible and high value to low income individuals and that have that stackability, that ability, that portability, the ability to move people from job to job. And at the heart of that is really the skills and competencies question, which if we could get rid of the credential and go directly there, we'd love to because that would be a much better way to sort of signal readiness and to see the kind of pathways that people do take and can take. So that's my connection to the work. So Lily, as I think about your introductions, one of the first things that comes to mind for me is this disconnect that tends to happen. I mean, prior to joining Complete College America, I spent 20 years on colleges and universities on campuses. And so, institutions like Georgia State that everyone hears about with regard to retention and student success. But even in that environment that's gotten a lot of national attention, I feel like there's been this disconnect between what institutions think they should be doing around credentialing and what employers are thinking. And so, you know, I'm very interested because right now there's this large proliferation of conversations about credentialing. But I think that one of the important first steps has to be a level of connectedness and trust that takes place between students, institutions and employers. And so, if I can start with you, Amy, and then maybe bring you in a little bit more, Nate, on this. Can you talk to me a little bit about the trust component in this? Because it's very scary on both ends, especially as we know this is the direction we have to go as a country. It's terrifying, frankly. Not just scary, it's terrifying. And I think you've hit on that triumvirate. It's the employer, it's the student, it's the institution. And we talk a little bit about how those all relate. And I think this is a fascinating part of our conversation. How do you build trust? Ever the optimist, I do see a lot of things that institutions are doing well, keeping in mind that when employers need changes rapidly and institutions also change quickly. No, exactly, exactly, precisely. So we have a bit of a conundrum therein. But I have seen in the landscape, over the last three to five years, the number of non-credit certificate programs at the institutional level has increased by about 70%. Now that's a very scary number because we then have a proliferation of credentials that are out there in the world. And thank God for folks like Credential Engine who are helping us to quantify and do some nomenclature and realize what exactly does that mean, this credential, that certification, that micro badge, that nano. So thank goodness we have that. But that proliferation also tells me that institutions are responding to student demand as well as labor market changes. Many of these are in online or on-demand formats which recognize that we have technological demands of an increasingly changing workforce. It also tells me that our largest population in terms of college-going entities right now are Gen Z and Millennials. And if it's not in Google, they're not gonna touch it. So we have to be very, very careful about that. We're also seeing a proliferation of CBE programs. How many of you in the audience are either part of one, know what that means? You know I was saying when I say CBE, competency-based education. And so not just credit-for-prior learning, which is phenomenal and extensive and shall always continue, this is a true beginning of a program from a CBE entity, meaning that it's based on competencies and what you can demonstrate as a learner as opposed to seat time. Talked a little bit about the difference between what an employer looks at when he or she sees an application or a transcript. Course titles are one thing and credit hours are one thing but if we're not speaking in the language of the student or the employer, we don't give the student a way to narrate what he or she knows and can do, then we're in trouble. So those are some of the things that I'm seeing in terms of the institutional perspective of helping to build trust and have more cachet in the marketplace. If we do have a moment, we can also talk a bit about maybe later the concept of stepping out and starting in points. We don't make it easy or certainly not penalty free for students to step in, do something in terms of a credential, step back out and step back in. And there are lots of examples that I'm happy to chat about sort of offline of folks that have figured out how to make this not scary. That's a technical term. But I'm also seeing across the landscape really bold partnerships. Now, while this isn't for everyone, as you might have sort of the Wheelock College being merged with Boston University School of Ed, sort of the small schools figuring out how to survive in the landscape and build trust, you've got folks like Georgia Tech and AT&T in Northwestern's Professional Studies School partnering with IBM. So big, bold, audacious partnerships are things that I'm seeing across the landscape in ways that I've never seen before. I'll pause there, but thank you for letting me lead off with that. Absolutely. Yeah. So I had the honor to serve on a work group for connecting credentials with Evelyn that was focused on this issue of trust in the credentialing marketplace. And one of the things that really, that I really took away from that conversation was a lot of people from across the sort of education and work continuum there from everything was sort of national intermediaries to institutions, to people that were non-traditional institutions were doing really creative work to employers themselves, to credentialers, was that we don't talk enough about the dynamics of trust. We sort of treat it as a concept that everyone agrees on and is sort of a critical part of whether or not a credentialing marketplace works, but we don't talk about the details of trust itself. And so that team dug into that some, and some of the things we pulled away from it that I thought were really interesting. So one is it's, I mean, the bottom line is that essentially trust functions like a currency, you know, it's a thing, sorry, credentials function like a currency. They're not, they don't have an inherent value in it of themselves. They only have value to the degree that people believe that they have value. And so trust is actually the cornerstone of the value of the credential. But the weird thing about trust is that it's both logical and emotional. So it's logical in the sense of you can take the data that says that you should trust something and there's, there can be reams of that that come along, the assessments that were done, the sort of value that it has, the individuals coming prepared to do the work that you want them to do, but it's also what your gut says or what your friend told you or you have a bad experience you had before with someone who had the credential or they changed the credential and you don't know how they changed it so you decided you're not gonna trust it anymore. Like all of that is irrational, but still totally human. I mean, that's how we make decisions. So when we think about effective strategies, campaigns to change the way particularly employers and institutions trust or don't trust credentials, we have to come at it from both directions. So you can't just have the data. You gotta have the personal, the human campaign as well to change the hearts and minds of people that you're trying to get to come to the table. So I think that's a really important thing. Another thing is that it's multi-dimensional, trust is multi-dimensional. So it's not one thing that makes us trust something. It's a whole bunch of different things. It's all the qualities of that credential coming together that matter. So again, when we think about trying to change the dynamic here, you have to come at it from all those different levels. So it can be the content of that credential itself and sort of ensuring that that's high value, but it's also the portability of that credential. It's also the cost of that credential. It's also the other skills that individuals are bringing around that credential and are they ready to work for those reasons? So you just have to come at it from a bunch of different levels. And then the third is that it's dynamic, meaning that it ebbs and flows. So trust is not something that is, you trust something and then you continue to trust something or you distrust something and then you continue to distrust it, although that's more likely. If you distrust something, it's hard to get someone back. You have to continually build trust. So it's not a one-off. You don't create the credential, build the relationship with the employer, for example, and then ask them to sort of hire your graduates and it looks like it's working and then you walk away. If something happens there to affect that trust that you're not aware of, then you've lost that relationship, you've lost that opportunity for your students. So I think to me that was just really an important aha moment for me in terms of how complicated the idea of trust is and how important it is that we treat it as a complicated idea that needs complicated solutions. And so when you think about coming out in the field and sort of driving trust, coming out of all those different levels. So when we think about trust, it's kind of, I think about setting the conditions for change and then there's the actual practical examples of the change itself. And so Kyle, I'd like to bring you in on this and I hope you don't mind me sharing with audience. We had a conversation backstage about this student experience or this learner experience and in the middle there's all the things that have been shared, stackable credentials and prior learning and traditional education for students. And then when you think about workforce and credentials and the things that are needed in the economy, there are two things that don't happen on the front end and back end of that experience. And so I shared a little bit about Complete College America's work with Purpose First and what we realized is we weren't having that appropriate conversation on the front end. So from the time students are admitted to a campus, what should the admissions application look like? Should it be designed in a way that helps individuals understand here's a major and this is what you can do with that particular major. And so those are things that we're doing and these are conversations that we're having with a number of organizations that haven't always been at the table but have been doing this work. Organizations like NACE and NCDA in the career space and NASPA and Student Affairs and ACRO in the admissions, registrar's, strategic enrollment management. So if you're thinking about book ending this work, there's that front end of what people experience that gives them some insights. And then Kyle, there's the back end. And the back end is really about how do you align talent, the talent supply that comes out with the employers that have a heavy need for new employees and individuals in this space. And so I'd be interested if you could share a little bit about how Moonshot Insights is aligning that talent supply on the back end with employers. Yeah, I'd love to. So just to walk through our process too, when we do the evaluations for employers, it's with the applicant pool. So it's when the resume submission comes through, the next step in the process is typically 20 to 25 minute evaluation that we do. We gather all the inputs that we need to predict against specific work environments for that company and that position. And it really streamlines across to the interview. So that's very important to understand where we are in that process. So we get a lot of applicants in that applicant pool and the inherent nature of hiring is such that the majority of those folks actually don't end up getting hired. So many positions receive as many as 250 applications for that one position. And then of the remaining 245, 240, the question is, how then do I better position myself to go and get that job? So the question there is really understanding that process and also the outcomes and predictors of success that we know through the data in our work with employers of where they would necessarily need to position themselves to successfully move from that 240 who were passed up to better position themselves to be one of those folks who are hired. So I thought it was important to be able to paint that picture a little more. And from our perspective, again, from the employer demand and identifying the predictors of success is really going back and communicating with those candidates and taking them in a guided way using a platform of resources and online courses that are available. And we serve as the underlying structure to quantify that learning in those credentials. And I think there's a question over there. Yeah, kind of a way it feels to have. Yes, and that's a question we field often and something that we're more than happy to field. So one of the opportunities, again, with new forms of technology and data as a transitional point for how we see the workforce is there's opportunities to do better than we did with the previous sets of conditions that we had. So this is a working challenge to be able to use this as an opportunity to do that. And the way we've approached it is studying the dynamics of work environments. So successful work environments include diversity, inclusive perspectives from a variety of different stakeholders. And again, what we promote is the best productive work environment for employers. So being able to factor in algorithms to factor in diversity in those biases and get rid of that and allow for more stakeholders at the table to allow for greater productivity and being able to get rid of those biases. And again, at the crux of a lot of the challenges that we're facing is the resume. So we find six seconds is the typical time a hiring manager spends on a resume. So I'll ask you the question, what information can you pull from six seconds of looking at a piece of paper? It's really what their name is, where they went to university, what degree that they have. And there are just enormous amounts of inherent biases and heuristics that are applied to that. So a lot of employers we work with, we actually de-identify the name, where they went to university and just let the data speak for itself. And again, in that data and those algorithms, we really prioritize inclusive and diverse work environments because that again drives the most amount of success. And again, it's a working challenge as we move forward in a really astute observation is how do you make sure this doesn't perpetuate prior conditions that are disadvantaged in folks. And something we've been very thoughtful about and hopefully more people in our space are thoughtful about that as well. But as we're at this transitional point, it's really an opportunity for us to interject and make it how we know it should be. Oh, I'm sorry, please, I see you over there. I guess it was around 15 years ago, they created the Collegiate Learning Assessment, the CLA. And the idea of it was that it would generate a number. Almost like an SAT or an ACT score. And that this would go on resumes. And I talked to the director of it at the time and I said, so now after all their education, we're gonna put them down to a two digit separated by decimal number. Because in the end, the worry is, that's all that the employers will look at. So now what I worry, and I was gonna ask the same question of you, so I'm looking for this, just like if I go to Amazon or Home Depot or anything else with the aid of the cookies, I'm gonna see only what they've programmed through their algorithms, what they want me to see. And it was interesting at the developers conference the other day that Apple said the new Safari was gonna shut that down or at least limit it. So the question is, in the algorithms, just to maybe get some other perspectives on this too, how do we ensure that just like whether you're on Indeed or Monster or anything else, that someone doesn't keep on getting filtered out of the equation? Absolutely, absolutely, absolutely. So I'll let you take that question, but I wanna bring in our other two panelists on this as well. So you have some thoughts about that? Yeah, I think I just have anything to add as well. So again, it's not bowling down to one specific number. And a lot of, and going through the college admission process and understanding SAT and ACT scores, that's not at all perpetuation and analogy for the workforce. Because we quite frankly, that's not a proper approach to assign a score at 30 ACT, which then eliminates you from consideration of the vast majority of higher education institutions or other institutions as well. So again, being able to understand a candidate is not assigning a score and discriminating against them. It's really understanding them in their individual traits and where they can best succeed in a potential position. And really the fortunate opportunity we have is to understand candidates fully and guide them in a career progression and course progression and learning progression to quantify their credentials so that they can be in the best position possible. So we use it as an opportunity to be the advocate for candidates and again, help them position themselves and give them very actionable direction to be able to do that and quantify their learning and their work. So it's not at all a boiled down number. There's so much that goes into it, right? And we don't claim to be the end all be all for any employer making a hiring decision. It gives you greater insight into who they are. So some background in the evaluation, we do a psychometric evaluation which involves our chief product officer headed up brain machine interface and psychology labs out of UW Madison. So a lot of that allows for a spectrum of personality types and archetypes that perform well in specific instances and situations. And as we talked about with becoming a marine biologist or whatever position, we have very little filtering and understanding of what positions we could be most successful. So any more information that we can allow for candidates and guide them in a directed way to quantify their results and be able to put them in the best position for the workforce is something that we're an advocate for. Not at all an advocate for boiling down single people for an empirical number that keeps them out of the process. It's a lot more nuanced than that and it's something we appreciate. And again, a challenge working with data, a challenge and opportunity as we see it. So it's an ongoing conversation and I'd love to take it offline as well to discuss how we can improve and how others can improve. And it's a holistic approach with all stakeholders in the space to make sure the best outcomes with this transitional point, we reach the best outcomes through new technologies. That's one of the biggest challenges with any integration of new technologies. How do we leverage it and how do we create a framework through which positive outcomes are reached? So thank you for bringing up the point. You know, I'm actually glad that both of you brought up that question because it was actually on our list of things to kind of talk about because the reality is what we're trying to do is not limit the pipeline. We're trying to increase it. And so there are quite a few demographic trends that we have to acknowledge. We have to recognize that our goal is to create a more equitable environment. And so we're seeing more students of color, more low income students, more first generation students, adult learners, et cetera. And so the typical college going student or credential going student is not the same way that we've always viewed it. And so the question that I have for you, Amy, is talk to us a little bit about some of those demographic trends and how our institutions and organizations adapting in supporting these changes. Absolutely, thank you so much for that. And to the two of you for bringing those questions, I appreciate that. I can address slightly from my lens and then I'll jump back into this question for you, Donnie, who I appreciate it. It's really about truth and lending, if you think about it. It's making sure that we're transparent about these transactions because at the end of the day, it is a transaction, whether it's from student and college, whether it's from graduate and employer, it is about making sure that you're having a conversation which you're both speaking the same language. And so I think to one of the points we were talking about earlier today, one of the big things that I see happening is, and this is not as a friend would say, rocket surgery, it's happening right now, is that being careful and clear about outcomes, whether it's choosing a major, whether it's a stackable credential, whether it's a certification, being clear about what those outcomes are is incredibly important. It also gives students an opportunity to make thoughtful, intentional choices earlier on in their trajectory, whether they're 22 or 52. And that's exactly where my audience comes from. We've talked a little bit about the role of faculty, perhaps, in not just academic advising, but career trajectory advising. This is huge. Who best to know the area of practice than a faculty member who has spent his or her life delving into the ins and outs of a particular nuanced area, who better to let individuals know what's out there for me if I look into marine biology or whatnot. But back to your question, Danifu, about demographics. We've talked a little bit about the changing face of today's students, and I realize that that's a very laden term. But it really is something that not every institution is fully grasping. And one of the things that I have shared earlier with my colleagues is that institutions that recognize the ebb in our traditional population, we literally don't have as many students graduating from high school right now as we did a few years ago. That's a tricky problem. We have an ebb and flow of our population. If we are not recognizing that as institutions and we are also not addressing the mass amounts of adults who have some college but no degree, who are interested in career changing, who are ready for their third, fourth, fifth, or sixth career, it's happening, I tell you, my poor 16-year-old is ready to be done with school. Oh, God bless her. I don't know how, we're talking about that at home, sweetie. I don't know, it'll be done. No, it's not, it won't. But recognizing that it's that continuum, that stepping in the stepping out is extraordinary. I think I mentioned CBE programs, credit for prior learning, things like Hopkins that are doing flexible, graduate, online, high quality, often hybrid credentials, that's huge. I think Lewis is gonna talk tomorrow about digital credentials. Have any of you seen institutions that allow for a degree to be combined with a certificate and have a badge slapped on top? As I tell the 16-year-old at home, you can be a biology major. Just make sure you have a Spanish minor and an accounting badge. And of course, she's not very excited about any of that. However, this is a different story. It is ironic, is it not? But with her mom, anyway. But these are the kinds of things that I'm seeing across the landscape. It's not just the degree or even the certificate. It's the narrative and the freedom and flexibility to allow our students to stack and snack and substitute and customize in a way that makes sense, whether it's the industry telling them, this is what you need, or it's them as their own locus of control saying, here's where I wanna go. I wanna do that. And I'm gonna reverse engineer it because I think I'll get there if I do these things in an apple. So that's what I'm saying in terms of change with the demography. Nate, JFF has been in this space. And so you've, I'm sure that a lot of this resonates directly with you. And I know you gave us a kind of a high level overview, but can you talk to us a little bit more about some of the work jobs for the future has been doing in this space around supply and demand? And you're happy to jump in on some of the topics that we've started to delve into as well. Yeah, so many things I wanna say. So this issue of bias, I think resonates well beyond the algorithms that are used to determine whether or not our candidate is considered. And I think it's something that, so JFF is about to launch something we're calling the center for the UNTAC workforce in the next week or so, next two weeks. And that's really focused on, we have this tremendously strong economy right now in theory, but when you start to dive beneath the numbers and look at not your U3 unemployment rates, which are the three, you know, whatever the 3.8% or whatever it's at. Now, I begin to look at your U6 unemployment rates. You begin to look at your long-term unemployed. You begin to look at people who were formerly incarcerated. You begin to look at black men. You begin to look at populations within the greater sort of workforce or potential workforce of America. You look at people who are making less than $15 an hour. That's 40% of the workforce. It's 80 million people that fall in this category. And this is a tremendously strong economy we have right now, right? So what happens when the economy goes south? What happens to those individuals? What happens when educational systems, public educational systems begin to fall behind because as the chart showed earlier, policy lags the changing economy. And when policy lags, educational institutions lag because they're reliant on policy. What happens to the students that today attend those institutions and use those institutions to access their first job and to access their career pathway? So we have tremendous concerns about what the future of work means. And if we're facing a tiered system in this country where low-income people will not have access to the pathways, the institutions they need, the employers they need at the table. And so the reason we're creating this center is to try to draw a national conversation around that that brings people to the table from all sides of the equation, not just the education side, not just the workforce side, not just the student side, but we really need to bring employers there too to talk about that. I will say in terms of strategies, like where I'd wanna see growth and where I think there's promise. So one is on the work-based learning side. So the idea that you can take the workplace and bring it into the classroom or the classroom into the workplace. And apprenticeship is a hot term but it's really much bigger than that. What's great about that as a strategy is that in addition to being very content-specific so that students are really able to master what they're supposed to do in a workplace if you're actually learning in a workplace. In addition to being content-specific, it's personality-driven and it brings personality and character of that individual to the employer in a way that they would never get through a resume sorting system. So if you have anything, even if it's an internship, job shadowing, even bringing an employer into a classroom to meet with students and there happens to be a student in the back of the room who asks a really good question. Like that's making it personal. And to me that is if we could have every institution in this country engaging in some form of work-based learning, I think that would be a huge step in the right direction to sort of bridging that divide. So that's one solution. I could go on and on. I don't wanna monopolize the time but that's a really important one for me. So if I could press on that for just a little bit. I love everything that you're saying. One of the things that I find in my members are wrestling with this. Whose responsibility is it for workers tonight? That's for the capital W. To continuously be retooling. I think there was something in Inside Higher Ed just the other day about this concept that you go to school for four years right out of high school. You get this degree. You're done. Again, that's the 16-year-old. You're done. You're ready for whatever your meaningful career will have at you for the next 20, 40, 50, 60 years. That's not the place that we live right now. Nor will it be into the future. And so what is that relationship? Is it corporate-based learning? Is it institutions? Are workers responsible for their own learning? I would love to think so but we've gotta get them some supports. So maybe working through that is one of the things that helps with that bridge is what is the relationship? Are workers out there on their own? And to what degree could they be on their own and claim some of these things for themselves? But what is the response from industry and from a higher education institution? What supports are we giving to help this be a continuous pipeline as opposed to a one and done? And I've heard it across all three of you and we've talked about this alignment. But a lot of our conversation tends to be heavily weighted. What should workforce do? What should educational environments do? I'd be very remiss if I did not think and ask a question about the student experience. I mean, this after all is all about lives. It's about families and citizens within our country. And so Kyle, I'd like to start off with you because I think that you've been talking a little bit about these new forms of evaluation. And so what does that look like and how does that change the experience for a student from this traditional path of moving into workforce? Yeah, and to paint the picture too is to understand what we've been subjected to in the past, right, is making guesses based on what major we're gonna take based on some hobbies that we have and combining some elements together. And with having greater insight for employers, there's greater insight on the other end for candidates as well. To have informed decision-making based on true indicators of who you are and where you could succeed and then guided courses and positions and pathways for them to be able to succeed and understand their role more fully. I think that transparency is really important for the candidates, understanding how they perform relative to their peers, where they're deficient, how they can actionally do better, right? It's having an assessment that doesn't have any context on how you can improve is pretty empty in itself. So that's something that we strongly oppose is giving and painting a picture for candidates of where they're deficient and how they can then retool themselves and create the skills and necessary attributes to be able to better position themselves. And that's again, what where our mission lies is being able to quantify them through that journey to put them in the position where they're most successful. So when we have first day freshman orientation and we've all had it, right? Science majors over here, business over here, veterinarians over here, right? It's kind of like, well, my friends going in this direction, this is a huge life decision that I'm not thinking too much about. So more often than not, I'm sure we're all guilty of it. We might've followed our friend over to a group that we had no association with. We had no understanding of that profession and career. And maybe 20 years down the timeline, we're still there. So being able to give transparency for students and candidates and let them understand fully who they are, be retrospective of where they wanna be, and then painting the picture of what that process looks like. So I think that's an opportunity we have through data and being able to then map it against workforce outcomes to then be able to drive them in those directions. So I think that's an incredible powerful vehicle to drive forth a better understanding of individuals and students to perform it. Yeah, I think we have a question for you. Yes, sir. Building your algorithms. I think of the words of Mark David Mule-Iron saying that when you either get forward or lose a job, it's because of the jerk factor. Either you're a jerk or your boss is a jerk. However, it speaks to social and emotional intelligence and the non-quantitative factors that are here. How do those, which seem to be more of those things that when we talk about jobs for the future, those are some of the things that we look, talk about grit resilience and other social aspects. How do your algorithms or in looking at this go beyond technical or other skill sets and look at the social-emotional skill sets that are required there? How do you vet that throughout an organization to assure that you're not gonna hire a jerk? Yeah, for sure. Or promote a jerk. And understanding fully soft skills is a huge predictor of success. So overlooking that entirely is something that is not in the best interests of candidates or employers. So two bookends of the components is a psychometric analysis which fully paints the picture of an archetype. What personality drivers you have, what dispositions you have as a person. And then another element that we've included are qualitative responses. So the typical behavioral and situational questions that you have an interview in your first interview of tell me about a time when or other things that allow for behavioral situational positioning of candidates, we can then map that to a lot of the quantitative data. A small subsection is really the hard skills as they are today. We put a lot more value in what trajectory and growth potential you have given your soft skills. So those two elements are enormously important in being able to fully paint that picture is fully robust psychometric analysis and then qualitative responses that can sift through and understand previous behavior and situations where you've been successful in the past. I couldn't agree more. One of the things that I'm seeing across the landscape is that skills and tasks can be taught. They're often discreet. They often ebb and flow. But some of the things that we've here too for called soft skills are probably the very foundation of us to be able to be employed non-jerk humans. Collaboration, communication, critical and creative thinking. These are the very backbones of liberal arts education. One of the things that institutions need to figure out is how to make that appropriate and accessible for all humans as well as build bridges toward industry to allow there to be specific and discreet skill sets that can be tapped on demand. If we position higher ed to only be available for on demand just in time learning, we have failed our population because in five minutes those skills will be obsolete. And then we have another group that needs to be coming forward to be trained, if you will. Yep, we have a paper coming out next week with Pearson called, demand-driven education. And there's a quote from that that I love. One of the areas we look at is the skill question. And we did an analysis of Pearson's 2030 skills report but also Cognizant and McKenzie and the Oxford original automation paper and all the stuff kind of looking at break down there. And the quote is, as the future of work unfolds what makes us human is what makes us employable. Which I think is how we need to think about this question of soft skills being the core of the educational experience, whatever we want to call them. And the technical skills being what is surrounding it. And that's a totally different dynamic for many institutions in this country to think in those terms, not to mention the policy doesn't reward them to do that either. So some things need to change to make that difference. I want to bring up, I want to put something on the table that has not been mentioned at all and yet is certainly implicit in the discussion of the entire afternoon. And we're looking at a society that is deeply divided with locked in stratification and locked out opportunity. And we're talking about what it takes to give people who've been locked out opportunity. But finally, this is a democracy. People make decisions as citizens and nothing in this skills conversation has yet even mentioned that much less explored it. So I wanted to put that on the table and see how you are thinking about it because you're all thoughtful people and I'm sure you are. Anybody want? I can throw it a little bit and not sort of open up proverbial can of worms. But there are a lot of things that say, for example, policy change through the Higher Ed Reauthorization Act could do to address some of those pieces. CBE as being fodder for financial aid, adjusting workflow, course loads, those sorts of things. Let's hope that those are future conversations that we can be having collectively between data and industry and employers and institutions to help make some of those policies allow there to be more accessible opportunities for some of the audiences that you're mentioning and specifically those I think you're working with on a very daily basis. So I'll pause there so we don't dive into policy landscape. So I'm looking at the time and this is what I'll do. I wanna make sure we kind of popcorn a couple of audience questions. But before we jump into that, I wanna leave with one final piece, which is I'm assuming that some of these things are taking place and they're working and there's some great strategies. If each of you could just quickly give us, what's one key takeaway of something that we should be doing to move the needle on this work broadly, that would be helpful. And it doesn't have to go in any particular order, but I'd like to hear from everyone quickly and then in our remaining time, we'll take questions from the audience. I can start on the end. Yeah, Amy and I clad read on this backstage. See it's one of the C words, collaboration. Collaboration, here we go. It was key for this insight that we're gonna have. So one of the key takeaways really is ownership from the learner and the candidate's perspective of their credentials. So to be able to centralize and understand their competencies where they lie and be able to control that and direct that in the future. So one of the biggest takeaways I've seen that we address backstage again is ownership with the learner and with the student and being able to have informed direction and being able to give them structures to better understand how to make decisions and how to follow through with those decisions to reach the outcomes that they wanna reach. Right, so sort of student-owned, student-centric, student is locus of control in terms of his or her own learning record. I'm looking at my friend Pat Leonard from Creadly. But in particular, having the capacity for those records of learning be written in the language of the student as a narrator as opposed to in the language of the institution, which as I say this, I realize my members would be shocked and terrified to say this. However, I have one more thing, because in the words of my friend, Dave Jared, inside track, if you ask Amy Heitzman for one thing, she'll give you at least two or three. I have the second item of what we need to take away from this. You mentioned this very, very astutely. Has anyone seen the book Robot Proof by Joseph Aowan, the president of Northeastern? Everyone get a copy, read it tonight. It talks about the very nature of the work in the future, not from a tactical or specific way, but what keeps us employed, non-jerk, what keeps us relevant and available and open to new ideas and opportunities is the thing that makes us human. It is the critical and creative thinking skills. It is the humanities. It is all those kinds of conversations that only we can do that robots, at least right now, can't. So I encourage you to take a look at that. Thank you. So I'll add all great points, I agree with everything, and there are a lot of answers to this question, so I'll just pick one. I would love to see a more flexible educational institution, whatever, in the public sphere in particular. I think right now, particularly on the credit side, we have real problems with being fast and responsive and being able to change what doesn't work and to be able to measure what doesn't work. When we work in the field, the demand of an education paper is a good example, a lot of the institutions that we found through that process that were really having a lot of success with placement, they were non-traditional institutions. Things like, personally, I work with Per Scolese and General Assembly on some IT pipelines there. They produce, they can turn their programs around in a matter of weeks when they want to make significant changes to them. And so I'd love to see starting with the policy conversation all the way up to the federal level with higher ed and with the other TANF and SNAPE and T&E and the other programs that are out there, but ways in which they can create a more flexible system and incentives to innovate within that system. Opportunities for programs like, designs like CBE, Companies to Education to be adopted at scale and financed at scale, and then an accountability system that really looks to how responsive institutions are being to the outcomes of their students. And I'm not saying to the employer, to the labor market, but to the outcomes, what happens to their students. So that's where I'd love to see that, that if there was a kind of wand and I can make a massive change, that would be one I'd love. Good, good. So we've got a couple of hands. I see one in the far back and then we've got one right here. So maybe we'll get the one right here in the front until the microphone gets to the back. Hi, thank you for a great panel. I'm interested in the idea of ownership of the learner and student as the locus of control. Can you talk about that with respect to first generation students whose parents haven't gone to school and don't have the college knowledge to do ownership of learning. Can you address that concept? I can speak to it obliquely because it is something that truly needs attention and to be addressed. You're completely right. I think your colleague at the table also mentioned those structures of preexisting notions of access and knowing that when someone sends you a calendar item, you hit yes. You say please and thank you. I'm not being pejorative, but these are lessons that we as a society teach each other, we teach our children. These are things that not everyone comes to college ready to do to learn to know and that relationship, that savviness is a critical part of success. One of the things that I mentioned to a group of presidents a few weeks ago is what if the transcript was in blockchain? Yes, they had the same response that you do right now. Silence. It's incredibly exciting and it's also terrifying because then those supports and structures that aren't there right now limiting as they are aren't there right now. And so these are the conversations that we desperately need to be having in tandem in partnership to make sure that when we say access and we say locus of control from students we really know what we're talking about. So what happens when those training wheels and those parameters come off? How are we teaching the next generation and that can be everyone from 22 to 52? How are we teaching ourselves how to be in charge of our own learning and where's the benefit and what does it mean and how are we shaping that? So I would love to have been able to answer that with here's what we need to do but I believe fully that we do need to do where all that's headed. Sure, so very quickly on that. So an area that there's not enough conversation. So look to the sort of early bits around blockchain and the potential within the higher ed space and what that would mean. It turns out data is collected and owned and shared. It's a very exciting idea but it actually has really significant equity concerns. If you just put the data in a blockchain and give people access to it with no structure around that then we aren't really exploring the issue of what do people do with that data or what do other people do to try to get that data. So I think that it's very important that as those of you in this sort of education and tech worlds are having these conversations that you intentionally include a piece around there around unintended consequences, around equity because if you don't do that this will be just another grand experiment that ultimately can have really negative consequences for people in this country. And there are institutions that are doing some of this work well. So ensuring that the voices of campuses that this work is embedded in their mission and telling those stories and how it can be replicated. I mean I think about historically black colleges and other types of institutions that have long kind of focused on how do they take some of these previously underserved populations and ensure this is part of the work throughout the educational experience. I saw that hand far in the back. I think you were the second person. Hello. I'm just not loud enough. That's a new problem for me. I'm Armur Dalewalia from The Evolution. I think this is, Amy's gonna have a great answer for this but I'd also love to hear what the other two have. A lot of the work that's happening in terms of sort of the things you've been talking about innovating the approaches and access points to education, finding new ways of certifying knowledge, creating new opportunities for folks to get just-in-time learning to broaden their technical skill sets across a range of soft skills is happening in the continuing ed space already. But doesn't seem to making that transition over to the main campus and the work the main campus is doing. So what can, whether it's like a pull from the main campus from their CE division or a push from CE to main campus, what needs to change to create a philosophy that supports the adult learner and the non-traditional student on a main campus as they start to shift their sort of focus away from 18 to 22 year olds? And small question, little thing. Right, right, right. And we probably got maybe one person can respond to that and I think that might be our final question. Okay. I'm happy to try that in Amrit. I'll get you later. One of the things that I've seen which isn't exactly answering your question but it's an example that I think might illustrate it a bit. I'm saying across institutions there are opportunities for PCE units, professional continuing or online units to become the OPM within their institution for other entities that are not yet in the online space or not yet in the alternative credential space. Syracuse is doing it, University of Washington is doing it. Those are some great examples to check out because they are serving those non-traditional units or serving traditional units in terms of modality and access. So I'm gonna pause there. Maybe my colleagues have a chance to weigh in as well. I mean, so there's a lot in your question so I'll just take it more narrowly from my world but I think that there are a lot of, so community colleges have been doing non-traditional work for years and they have learned over time to the degree that they're able to be flexible within their constraints to change their delivery model to support non-traditional students. And so that's everything from, and I wanna say this isn't just like a distance learning solution that they've taken on because in fact there are a number of students that don't do well in a distance learning environment so don't just throw that out there. But it's everything from changing their office hours and availability to when things are available on campus. It's the way that they do intake to look to the barriers that individuals may have that they haven't identified or wouldn't otherwise identify to their faculty. It's having people who specialize in understanding supportive services and where there are resources in the community and public funding streams that are available to support people. It's the way that content is delivered within the classroom, the pedagogy itself needs to change. It's understanding the priorities that your students have. A lot of non-traditional learners are in a rush to get out. They wanted to get through their training program and then into the workplace. Don't operate under that assumption, definitely test it, but that is a common thing. So those are the sort of examples that come to mind for me. We could have continued this conversation for another 30 minutes or longer. If you can please join me and give my panelists a round of applause. And please, just a final note, a couple of us backstage, we're talking about the opportunities for additional partnerships and conversations. And so you have our contact information in the program and the app. And I believe we'll be adding a few additional resources that may be helpful to you. So we'd love to continue this conversation. Thank you very much.