 We have Ryan Davis, President of the Corporation for a Skilled Workforce, who's going to come out. But first, Louis Torres, who is the Vice President of Strategy, Research and Advancement at the American Council on Education, is going to do a quick presentation, and then they're going to have a conversation by our virtual fireside. So let's welcome Louis now, please. Am I on? I'm on. So these are the true believers. It's like 3.15 on a Friday, and you guys are actually staying. So I asked for a little indulgence from Sailor, and I wanted to show a few slides. And I gather my fireside chat was supposed to be kind of like takeaways from the conference, reflections on the two days. And so I hope it serves the purpose. I've been of a mind to focus on being human in my observations about the summit. And I wanted to just take us on a little trip on a start with a credential, end with credentials. But I will just talk about learning a little bit in between. And a reminder that the words that I keep hearing, and I'll do a little talk here, and then Ryan Davis is going to come out and we'll have a chat. But technology, disruption, delivery, assessment, competency, these are all artifacts of learning. They're not learning itself. And I want to start with a story about what I argue is, oh gosh, yes, that's interesting. I argue that the GED is the first badge of the modern era. It was the first assessment that was agnostic to where you learned something. And its journey actually began in 1892. With the commission of 10 run by Charles Elliott that ended up creating the Carnegie unit at colleges. The first indicator that they used to try to use their leverage with institutions was called the high school unit. A college to get access to the retirement funds for faculty had to use admissions requirements that included 14 high school units, which had been defined by the thing. That conversation about the high school unit blended into the ongoing debate between the folks that thought that high school should be college prep, vocational, or general education, particularly the progressivists. As far as I can tell, not unlike one of our last speakers, that conversation is still going on. So the journey of a credential, a badge, a way of measuring learning, this one's been going on for over 100 years. So it's a very sobering and I hope informative thing about the journey that all of this conversation is that we're going on, is about to go. The other thing is too that the GED, depending on how you view it, is either viewed as an engine of progress or an engine of inequality. And as we heard over the course of a couple of days, this is already emerging for more granular credentials. Who gets it? And are you going to double down on people that already get credentials? And is that going to expand inequality? So this is learning. Learning is a process, not a product. It's a transformation and it's agentic. It's not something that happens to somebody, it's something they do. They can be helped, but they do. But most importantly, the transformation isn't just about content knowledge. It's knowledge, beliefs, values. It's a holistic view of the human being. So not only is learning these things, it's not necessarily content skills. Learning is also social. It's a demand-driven identity-forming social act. Research inside the classroom, outside the classroom demonstrates that the knowledge development and developing expertise is linked to the community that you're in and the identity that you're forming. And one of the questions I have about the granularity we go to is this embeddedness in community, in knowledge communities, is directly linked to how much of what you learned in that context can transfer. And so when we get to a granular level of knowledge, how far can we go before we have to question the transferability to other settings, in particular with badges? The way I think of the demand for why this is happening, institutional economists, there are a strand of economics that focuses on the way an economy is designed. And they've been doing a lot of writing about knowledge flows and learning by doing. And this basic argument here is that we're creating more knowledge in real time than we ever did before. We've heard about that already. IBM says knowledge will double in the next decade, things like that. The way to think of it is it's synthetic knowledge. When you think of green technology, green technology is really three technologies. It's nanotechnology, material science, and biological science. And the new knowledge that gets created in creating green technology from that. A lot of folks are studying that and saying that that knowledge is all happening outside of formal settings and it's at college level or better. And with that level of knowledge is happening in outside of formal settings, there's an increase in the intensity of learning by doing. That means learning, we would call it applied learning, but happening in many settings. And what these folks argue is that this is why we're seeing the drive towards measuring learning at more granular levels. Because the increments we were using in formal settings no longer apply given the learning intensity that's happening in non-formal ones. This is important for me because it's the vector along which this whole conversation is taking place. You know, what you measure, how fast you can measure it, how relevant is a badge for how long. And when knowledge is changing that quickly, what is it that people want? And this goes back to the late 80s. I use this slide all the time because I find very few surveys that ask this simple question. How do you feel as a human being that you'll be competent in the world? What do you need to know? And this is a National Institute for Literacy study. Most surveys go back and look at surveys in the last 15 years. They all ask specific questions about what form of education would you like? What form of post-secondary institution would you like to go to? This broadness was simply, what do you need to know to feel competent in the world? And not surprisingly, they end up talking very instrumentally, but I think not instrumentally in a bad way. It ends up being about how you orient yourself in the world for independent action. The four categories are access, voice, independent action, and a bridge to the future. Most importantly, for the purposes of learning by doing, is that they emphasize in their roles as parents, citizens, students, and workers so that they can do good for their families, their communities, and their nation. It's a very whole life view of what they consider competence to be. And I'll come to badging at the end of this and whether or not we think that the badging and credentialing movement is taking us in this direction. If the way you feel competent is by somehow understanding those four domains of your life and knowledge in learning is changing so quickly, how do you manifest the learning identity that integrates all those different domains in your life? I call this optimizing human capital. So what is it that you're learning as a worker that actually feeds a learning identity that allows you to distribute it to your family or to your citizen or your student? The example I use of this is if you're a single mom with two kids trying to manage two IEPs, what do you learn about how to leverage a system that might apply and transfer to the other domains of your life? The interesting thing about that transfer is that the cognitive researchers that study this with TANF recipients, part of the problem is you don't see it as knowledge that is transferable. So you have human capital that you're unable to access because you don't recognize it. And the way you develop a learning identity that's a dynamic flow between these different domains in your life is through something John Sealy Brown calls practical imagination. You build practical imagination, experience by experience, relationship by relationship and community by community, and you integrate it into the learning identity there. So if what folks are looking for from us is to help them integrate a learning identity, and then this is my only slide on AI because I think the AI conversation actually only serves one purpose. It actually, by our interaction with technology, it's not pointing out the skills we need to have, although it does do that, but its most valuable thing is that it's going to help us understand more deeply what it is to be human in light of technology. And when I think of how you integrate that learning identity, what I believe the next space that badges can help us in is actually helping us move along the continuum that Hannah Arendt gave us in the human condition. Labor was her subsistence work. It's what you need to do to survive. Work was what we would likely call jobs, like the things we do to make things in the world. And action was how we work together to make each other better and society better. And I actually believe that her framing of it was largely political, but I think the work that's left for us to do in light of technology is inherently relational, and it will require that we become remarkably good at integrating all these domains of human capital. And the two questions I'll leave you with before I ask Ryan to come out are our badges. The GED, in its most cutting-edge form to return to the first badge, its diagnostic tools linked to real-world peer connections, something called GED Flash and GED Live. The diagnostics about how you did on the assessment combined with these two peer-to-peer platforms are driving some of the most increases in knowledge that they see among GED test takers. A community is being formed there. It's early, but this notion of how you link the learning process, the social asset, the identity-forming aspects of learning are badging helping us do that. This paper is from the late 1970s. It's written by a Marxist economist who basically argued that credentialing was just going to increase inequality and it was going to get worse and worse. And the largest reason is that people who hold credentials politically are unsented and don't hold those credentials out of whatever occupations they're in. And I don't think we're there with the current credentialing system yet. However, we're beginning to see this doubling down. You have a bachelor's degree, you're more likely to get other add-on credentials that would increase income inequality. I'll ask Ryan to come out. And hopefully that was at least the beginning of being worth staying at 315. Outstanding, our fireplace is working. So we definitely know that we have an audience here of the diehards that are staying all the way till the end on a Friday. So I want to thank you all for staying for our chat here and appreciate Lewis making the remarks. A lot of pieces that really kind of tie together some of the things that we've been talking about for the last couple of days. So I wanted to start off by talking... Yesterday, a lot of things were technology-focused, but a lot of these new technologies that are coming on really seem to be a convergence of multiple technologies. So we talked about mechatronics, which is really a convergence of electrical, mechanical engineering with some information technology. So your concept about learning by doing, how do you see that concept fitting into the workforce demands of this kind of convergence technology economy? So one of the slides I use a lot in my presentations is a picture of the Boeing 787 Dreamliner. And human resource scholars, so academics who study human resources, estimate that the learning that happened, the learning by doing that happened in the building of that Dreamliner was the equivalent of an associates degree. It happened totally in workplaces. And I'm not arguing for workplace learning as much, it's simply pointing out that that level of learning is happening out in the workplace. And that's because in that instance, the Dreamliner is a mix of everything you just said. Right. Everything. And that's happening more and more and more in particular in developed economies. That kind of value creation is what's driving economic growth and job creation. Great. So within that economic growth and value creation, equity has been a theme that's really come across and a lot of questions come up from the audience about the impact of some of the ideas that have come up here with pieces. So we're going to address inequities. We're going to address equity issues if we're intentional about what that is. Do you have some insights about some of those structural pieces or systemic pieces from your research and experience that we should all take in mind as we're going out and advancing the credentialing economy? So the Chetty, Raj Chetty data, I mean, it's pretty clear that even as we're trying to experiment with different models, a lot of the old variables still apply. Parents' income, mother's education, where you went to school can impact you. And sometimes there's more value in some other institutions, and there are speaking specifically of the post-secondary education sector. With alternative credentials, the question becomes what is it connecting to is the platform already likely to reduce inequity? And are you not having the top of the credentialing market having more credentials added to it, sub-credentials, that are expanding, pulling the top away from everybody else? So what's the quality of the middle and the low? And is the top moving so fast away that you're going to cause inequality anyway? And the record's still out on that, actually. So are there anything that you can think of from a policy perspective, whether it's at the local level, state level, or national level, that we can be doing to ensure that that top is not pulling away, that we are lifting up people and trying to reduce some of the equity issues? The one thing that I can say that's healthy about the public policy conversation about workplace learning, in particular, manifest it in apprenticeship now, is that the institutions, I'll give an example of Northeastern for two reasons. One is their history of co-op, but also an experiment we're doing with them in the EQUIP project is that the rise of learning by doing is already hitting formal post-secondary education. And what I say to deans and provost when I talk to them is, what is your education model look like if 30% of all of your credits of the 120 are offered through experiential learning? Because that's the only way you keep pace with the rise in learning intensity. And that requires public policy change around the way we recognize and pay for credit hours. And I'm not so much making the competency-based education argument here. I'm simply saying that if the institutional economists are correct and this rise in learning by doing both exists and is accelerating, then experiential learning becomes one of the few ways you actually can optimize human capital in your country. In formal settings, how do you get more experiential learning, even if you didn't change the 120 hours of the credit hour, how do you make experiential learning more a piece of that? And so that's a great segue to the next question, is then that really has an impact on the business model of higher education. Because most institutions of higher education really are basically getting paid by the credit hour. And so when you're a part of what you're providing is in that experiential learning piece that you're recognized, the business model has to change a little bit. And so I think as opposed to the question about what that business model looks like, it's kind of up to every institution to carve that out. But how do leaders go about changing that business model? Particularly in higher ed where the tradition and the structures really sometimes dig their heels on and change, how can that change take place? So this is, of course, the hot topic with all the philanthropies, right? Change management, if I hear institutional transformation or change management comes from Gates or Lumina or Kresge or, but really it's Gates or Lumina, which that's a healthy conversation. I think, you know, it's funny, I used to be in the think tank world and now I've been at ACE for about six years. And one of the things I've noticed is that when you think about just to use these two two broad artifacts, the administrative side, the business side of one of the institution and the academic side, we're only now, there will be still cultural issues and mission-driven issues about how higher ed needs to evolve to meet these demands and there always have been. I'm struck that, for example, on the academic side of the house, we're still learning what outcomes mean and then how to manage to those outcomes, right? I went to, I visited UC Riverside and they have calculated, they can now, for degree program and course, they can calculate at a granular level, you know, if they change 50% of a course, what the ROI is to the degree program. That's a very unusual set of both accounting norms and technology tools out for a college campus. What I'm trying to say is that I think that one of the issues is new ways of thinking about, new tools that actually help you see the business model differently. And then, so that way you at least know what you're changing when you change it. We're still at the beginning end of that. I just say it, just to pitch my paper shamelessly. I wrote a paper called Evolving Higher Education Business Models, which focuses on this topic exactly. It's the confluence of our ability to understand learning outcomes, which is getting better. The financial systems in particular on college campuses, the enterprise resource planning systems, the learning management systems, the education systems are finally getting to a point of integration where you actually can begin to map learning outcomes to investments on the financial side. It's still in its infancy, though. So it's not that change may not need to happen. It's recognizing that are the tools and processes in place beyond cultural issues, and those are very reasonable conversations to have. You also need the tools to be able to do it. Right. Yeah, and so before I took this position, I was the Dean of the Community College, and the connection between the outcomes, the outputs that we were trying to drive and the investments of where the money was going out of the college, often were not connected at all. I mean, it was, well, it's copy and paste last year's budget and make changes as we go, instead of strategically making those investments to get to the ends of where we wanted to be able to do that. And just to add complexity to it, which has been with us for as long as we try to slide on the one that what is learning and the social nature, that's a very complex thing. And so, you know, given that the complex nature of what you're trying to do, getting to connect the outputs with what you're doing is just hard. Right. Right, so in addition to the fact that we may need to be better at it, it actually is just complex, and that's okay. So from your view at AC, you get to look out on this broad educational landscape, the good, the bad, and the ugly out there in the landscape. Are there things that you would recommend to the audience of kind of glimmering towers out there that we should be looking at, or the shining stars of pieces that are really going to kind of, that are ahead of the curve in this transformation and are really going to take us into the future? I mean, I'll name the ones everyone would name, right? Rio Salado College, I would look there. I would look at Valencia. I would look at the folks of Northeastern experiments with the quip, but also NYU. NYU is doing this phenomenal, they've extended their alumni network into an actual knowledge and learning community. You have to think of it as, if folks will recall the eDating sites, eHarmony and Match.com, one of the things that they realized about five years ago was they needed to add a high touch component to what they were doing because the communities were starting to form, but they wanted a physical embodiment to the relationship. And what NYU Northeastern is also doing this, what they're doing is they're saying, let's actually make these real learning communities. We don't have to have much control over them in their case. In part, it's interesting, because part of their case, I would say they're selective and their alumni can be self-organizing in knowledge communities, but they don't do it further down the selectivity scale. But those are examples in different ways of folks that are trying to understand learning differently and have learning continue differently as part of, of course, for both Northeastern and NYU, it fits in with their existing business model. It's a plus-up. Now, you pay a little bit more to supersize your alumni network to get into the learning community. But that's not necessarily a bad thing. Yeah. And being a service academy graduate, that's one thing that particularly in the Armed Forces was really there. So Air Force Academy grads, West Point grads definitely create those communities and pieces between helping each other through the places and identifying where there are knowledge gaps and where to be able to do those things. So as you get into, that's within a contained silo basically a particular branch of the service, as opposed to something that's diffused out there. So I guess now we'll turn it over to the audience for any questions for Louis, any questions for me if you happen to have a question of that or just any closing comments as we're wrapping up today. Hi, Carol. This question might have gone to the... Is this on? Okay. It might have gone to the last panel but I decided to hold it because they were clearly racing to wrap up but Louis, the remarks that you made about the complexity of learning and you put up in front of us a whole lot of books that go deep into the learning as a process and I would argue and I wonder how you would respond to this that these things we've been variously describing as soft skills or desired competencies or cross-cutting competencies, things like critical thinking, things like communication, things like intercultural fluency, etc. are not developed even in one course much less in a miniature piece of a course. I mean we may be able to take a student out of, let's say, a course and say you now have this competency but you didn't get it from this course. So I found it problematic to keep calling these micro-badges for sub-course competencies when in fact what we're talking about are capacities that are developed over time and I just wonder whether I have missed something here or you see it differently or what. It's not that I missed it. The example I have for you will actually freak you out knowing you and me. So one of the examples that I'm working on a paper called Learning by Doing, not that that would be a surprise to anyone at this point but there's an organization in San Francisco called the Flaming Lotus Arts Club and it's largely for underserved folks who are lost their way but they teach them performance art. I use that example because what's missing in your example is community and context. They learn soft skills and intercultural fluency over time by the practice of what they do. And it's not that I disagree with you. It's that learning is happening in places and they become quite fluent. This is a model that's been around for some time and they don't really graduate people, people move on. They become quite successful in Silicon Valley doing other things and in part because they've learned these skills in that space our challenge becomes what's the appropriate level of badging to understand if someone has that. And in the digital world I try to use there's a lot of learning about the way digital communities in relation someone mentioned it earlier in one of the panels today about how they're developing reputational things. GitHub is a good example in the coding community how you develop a reputation because of the community you build in. And this is my question. For the badging community it's how are badges situated enough in specific community and context that two things happen. First you acknowledge what the level is at that moment but because you haven't left the community you're allowed to nurture that thing still. And it I don't know that the badging community has to answer that by itself but it's just an artifact of learning that you have to address to know if someone's competent. Is that helpful Carol at all? I knew it. I didn't disappoint you. Devin Deb. We have two Debs. Okay. So I think the point you're making Louis comes back to assessment and this is going to sound really academic but it's not. John Dyer made a point earlier also about how you come into this mechanics class and the mechanic teacher dresses a certain way and keeps the lab clean and etc. But if you don't have an assessment for these competencies then not only do you not have a record of that level of proficiency of that competency but what's probably even more damaging is that the learner doesn't have a way of understanding what they've learned and being more aware of it. So the example that he was giving is simpler because in a lot of these occupational skills there are actually very detailed checklists where there are specific things that you need to remember to do and that's part of how you're assessed and in this example also I think that there could be a robust rubric that has criteria that can be interpreted in this context so that when you make an assertion in the badge that rubric is providing a very detailed description of what was being assessed. So it sounds academic but it's really a practical application of a process you know what I mean? I get it, right? Since the first time I ever wrote about CBE I wrote a short blog post actually the first thing I ever wrote years ago and I basically in that paper was a disruptive look at competency-based education I said this is really this movement is a boom for the assessment industry and it is and we're still trying to figure it out but I want to be provocative again because the reason I used a lot of examples in this latest paper of I called them learning economies for a reason I adapted the term that these institutional economists use because their economies in which learning drove competence that was never measured yet demonstrated success. The two examples I use are the Seattle grunge music scene and the rap music scene in New York that started in the Bronx in the 1980s they're both featured in a book at the Smithsonian called Places of Invention so they're well researched and essentially both locations created new technologies, new job categories, demonstrations of proficiency and competence that led people to move through them without credentialing ever taking place and that's because the community and the context and the relationships continue to understand what competence meant and I don't offer that as an alternative Deb because I wouldn't do that but it is possible and what the evolutionary economists say is that in learning by doing sometimes competence is only in the living demonstration there is no other way to do it and it's portable what I would say to you is that the grunge music industry went from nothing to a three billion dollar industry and a whole bunch of people and it started with a bunch of people many of them that didn't have high school diplomas and if you ever go to the experience music project in Seattle my colleague you traced a history of the early grunge music scene where people that were failed basists ended up running sub pop records and they learned it all by doing and again I want to be careful here I'm not being kind of like a no school triumphalist I'm simply saying for the badging movement if you have the community is large enough right the rap music scene did the same thing the rap music scene actually developed entirely new technologies people that didn't have high school diplomas the sound boards that actually make the different sounds that they use and all that that was invented on the street first in fact in burnt out factories in the Bronx again it's not an alternative it's not hyra it's going to blow up but it's a way that you know learning happened and common street competence is demonstrated that is totally apart from what you just said and it doesn't mean that what you just said doesn't make it more portable but it means it's possible for an entire ecosystem to come without the assessment industry saying that you know this you know this you know this so I just one up to my provocative measure Deb Seymour I was just going to say that you can tell that Deb Everhart and I have co-authored several papers because clearly our minds work the same way I just wanted to make a follow up observation regarding the exchange you and Deb just had with us I think in recent years one of the criticisms that we've noted about higher education in the formal sense has been that we've developed all these proxies for knowing what knowledge a student has acquired while they've been attending an institution of higher education so we've got college rankings we've got the traditional transcript earlier today somebody asked a question how many transcripts do you have and somebody had eight somebody had nine in some cases representing an individual MOOC that had taken so the transcript has become a proxy and I think one of the things that we need to be careful about in this ecosystem of micro-credentialing badging and other kinds of micro-credentials is that we don't start developing proxies for what it is that people actually have learned in order to represent the learning that we have not measured or assessed they actually have and with response to your answer to Deb I mean I agree with that if the community acknowledges that the person has the knowledge through their demonstration of it then the formal assessment may not be necessary but in the absence of that just using a proxy is not necessarily a solution micro-credentialing could follow that same path that's what we all have to watch out for the totally true I totally agreed to so one of the books I had that I love and it's so dog-eared is the social life information which the original version came out in the late 80s but folks should go read Celia Brown and Paul DeGuid put out a new version of it in late 2014 read the last chapter it will describe for you the exact debate we're having here and for them it's just that they were pressing it as we you know anyone knows they are but it's still so meaningful they posited the challenge like with traditional higher ed it's because the package is so broad you may have learning that's happening in there's time maybe wasted but maybe in the broadness there's time that's not wasted but we don't measure very well and this is you know first you get more granular in measuring below the course level what do you miss the way they describe it in the last chapter of the book is a bachelor's degree is like an omnibus bill in the congress you're getting a bunch of good stuff done and probably a bunch of other stuff that isn't necessarily as useful but you wouldn't have got the other stuff done if you did and it's still it's so interesting because and it makes sense that the badging movement is about smaller and smaller increments of learning because there's a utility in us knowing that because then you know if you're looking at it from a formative assessment perspective you know where people are missing what they need to know right and how to build on it as far as summative and terminal things what are you missing if you add up 15 things that are all smaller and actually it's actually less than the whole right I don't know but it's a great observation there so we probably have time for one more question I'll say to the gentleman at the right to get his hand up first hi sorry I do want to ask you because I know that it builds on a lot of what we just talked about that I read recently that ace has just partnered with credly and I wanted to ask you about that partnership and why that was particularly strategic for ace I think the reason it's there were two reasons first, credly one is a practical business operations reason credly had a technology that we did not that allowed us to automate some of our traditional credit evaluation process the other that's far more interesting is it's going to allow us to badge competencies using the credly platform and we are exploring badging competencies either for communicating to member institutions but more importantly for us to employers that's a totally new space for us that we would use the expertise we developed over the years to begin to badge competency for communicating directly with the labor market we haven't done that traditionally well great well thank you very much loose appreciate you joining us today and providing a phenomenal insights and thanks to the audience for hanging around until 4pm on a Friday