 I'm very excited about this panel on competency-based education. I'm not going to do a big introduction because they're going to introduce themselves. And we also have the great Paul Fain from Inside Higher Ed to moderate. So please welcome this great panel, and Paul, take it away. Hello, all. Can you hear me? Great. So I'm Paul Fain. I'm the news editor with Inside Higher Ed. We have a great panel here to discuss competency-based education. So briefly to describe our interest in this topic, Inside Higher Ed and particularly me, I've been writing about competency-based at a lot the last few years, and for three real main reasons. Tremendous interest among colleges to create new competency-based programs. The last I heard, some learned in the neighborhood of 6 to 700 institutions are in the process of trying to create a competency-based program. We have pioneers in the space like Thomas Edison, Western Governors, been doing this a long time. Some upstarts, making a lot of noise, College for America from Southern New Hampshire where Kate used to work, Capella University, community colleges. And then in recent years, the University of Texas system, University of Michigan, Purdue. So a real wide range of interest among higher ed, and that, of course, peaks our interest. One of the reasons it seems to us that this is having a moment is the belief that competency-based education, when done well, can better connect to jobs, can help employers have more confidence about what students are bringing to the workforce. And in a lot of cases, we hear employers actually work with some of these programs to help set the curriculum and to work on the competencies that go into these programs. And then finally, you know, I think perhaps most importantly, competency-based education raises really fundamental questions about higher education. The question about the credit hour and seat-based learning versus time, you know, you can see that if this continues to progress, it might really change all of higher education, which is I think one of the most exciting pieces for us because higher education changes fairly slowly. But that seems to have changed since the recession to some degree. So enough from me. I'm going to just briefly introduce the panelists and they're going to tell us a little bit about their work in the field. To my left is Kate Kazan, the managing partner with Volta Learning Group, formerly with College for America at Southern New Hampshire. To her left is Steven Phillips, the assistant director for the Center for the Assessment of Learning at Thomas Edison State University. And finally, last but not least, Mary Ellen Wiltraut with MIT. She's a digital learning scientist for MITX in biology. So I think Kate's going to kick it off. Good morning, everybody. I always am aware that there's a certain irony involved in giving lectures about competency based learning. I just feel it. And in fact, the name competency based learning has caused no end of trouble, as we heard a little bit yesterday. So I'm thinking that if we do this again, we should call it something like Betty in a little less baggage, you know, a little more intriguing, but competency based education it is. And I'm going to talk a little bit about what I think competency based education includes while recognizing that, you know, hasn't flowers blooming and all like that is actually a really good thing. Nonetheless, I think because of the phenomenon that Paul mentioned, namely the sort of, I hesitate to call it a fad, but certainly a mushrooming of interest in CBE. Some things are being called CBE that to my mind probably are better characterized other ways. It's sort of like how do you remember when everything started being natural, you know, and now we have new, healthier cane sugar, you know, so I think it's a similar phenomenon. Anyway. So to my mind, there are three elements of competency based education that make it distinct. And I'll talk a little bit more about what each of these is. But my perspective is first that the flexibility of time is critical. And the assessment aspect is critical. And the doing, and that's probably the part that I think gets the least amount of attention and is to my mind the most important. So this was my attempt to do a Venn diagram that came out more like, you know, the Olympic rings on a bad day, but nonetheless, I think you can see how these circles overlap so that if you have a sort of flexibly paced or open, self paced kind of learning experience and there's assessment, right, you get credit by exam basically, right? If you have sort of a flexible pace and something that is focused on the doing rather than what's sometimes called simply knowing, which is kind of funny. And then you get maybe something like self enrichment or a continuing ed kind of class that doesn't have an assessment. And if you get the doing, focus, and assessment, you get performance assessment, right? So you can imagine something like a phlebotomy assessment, which could be time bound, but nonetheless is focusing on performance. And then when you get all three of them, you get the gold star, namely CBE. So my beautiful graphic is not showing right now, but I think we have the wrong slides, but it's okay. I want you to imagine the Dolly Clock, which is pretty famous, which, you know, is kind of an awesome counterpart point to this quote from the former president of Phoenix and actually a continuing board member at Lumina. And various people have made some version of this crack, you know, namely, you know, if you're measuring C time rather than learning, you're focusing on the wrong end of the student. And I think it still holds true. And assessment, obviously, we all know it when we see it, but it's worth remembering some of the different types of assessment. You know, when I took my test to get my learner's permit, I would say I could tell you how far down the alley you were allowed to park. But I couldn't tell you very much about what it was like to drive, because I really had no clue. When I took my driver's test, I still had no idea how to drive, but at least I was licensed to be dangerous. And I think this illustrates pretty well what we mean by performance assessment. Now, this is the part I want to zero in on for a moment. You probably can't read this, can you? So we can have a little contest about what it says, which could be amusing, but time-consuming. So instead, I want you to imagine a stereotypical hiring situation, and the old white guy is the hiring manager, and the people are clustered around him, and the hiring manager is saying, let's hire this candidate, why she has a B+, an introduction to sociology. Okay, a little bit of chuckle, maybe, I don't know, but okay. But why is this so ludicrous? Well, it's ludicrous for a couple of reasons, right? But one of them is the grade you get in a course tells you absolutely nothing about what skills you might have that are transferable, right? And introduction to sociology probably tells you that this person did pretty well in that course, at least compared to other people in that course, right? But it doesn't convey anything about can she do research, can she interview people, and so forth. So I think this is where the doing actually becomes so critical, and as Paul mentioned, this is part of why there is such interest in the employment-informing capacities of CBE, even though many people then say, oh, wait, it's just for vocational education, right? As though education to help people get jobs was just. But nonetheless, I think the focus on doing, in other words, rather than simply knowing, is the thing that I think gives CBE its power. So I've been using the word competency rather freely. And again, here's a slide you can't possibly read. And I just want to suggest what we, at College for America, a competency-based college that is part of Southern New Hampshire University that Chris Clerkin and I helped found five years ago. And by the way, Paul, only in the world of higher education can you have done something five years ago and be an upstart, if I am. That's true. But I think there are three things about these competency statements, and we have a real honest goodness CFA student here, which is exciting beyond belief. So one of them is that it doesn't say how long or how you get the information, you get the competency, right? That's the time flexible idea, right? And you get a little confused sometimes when we start to say, think of CBE as only in an academic environment because then you do care about the environment in which people developed competencies, right? But if you think about competencies as statements that stand apart from how someone developed them, right? You begin to see their power, right? Crossing from an academic or learning environment into an employment situation. They are at their best measurable, right? Duh, that's pretty basic assessment, but also observable. So sometimes I see competencies that say student understand such and such. I think, well, okay, you know, your guess is as good as mine, right? What does that mean? How do you know, right? And again, that goes to good assessment principles, but it also goes to the doing I talked about, the actually using competencies to do something. At College for America, we express competency statements as can do statements, which I will admit I ripped off from ActFill, which is the does language assessment and they express all of their proficiency statements as can do statements. And I liked both that suggested that students could do something, right? It's got a kind of optimistic quality. But again, it puts the focus not only on the can part, but also on the do part. And if you could read this, what you would notice is that it includes things like can negotiate with others to resolve conflicts and settled disputes, as well as more kind of conventionally academic things like can convey information by creating charts and graphs. And one of the things I'll talk about at the end is, which will come soon, is why this kind of fear that somehow what are sometimes called 21st century skills or soft skills, which really seems diminutive, is that they're somehow not measurable or that they're like some deep web that can't be accessed. And I think competency-based education gives you a way to think about how to assess the full panoply of competencies. So because my perspective of on competency-based education, and it's called Perspectives on Competency-Based Education, so I thought it was okay to have one, is that if you wanna know what people can do and how they can use what they know to accomplish something, it's really helpful to ask them to do something. So at CFA, we designed the academic model around projects, which are, I would say, kind of the poor man simulation. They're scenario-based, a lot of information to integrate an actual real thing to do. And they're evaluated by experts, sometimes practitioners and or faculty using rubrics. But the part that goes back to time flexible, that first component I mentioned, is that there are multiple opportunities, in fact, infinite opportunities to get feedback, to submit, revise, resubmit, and so on. So it really does take time pretty much entirely out of the equation, right? And I sometimes hear people say, but isn't time really important? I mean, after all, if somebody takes five years to write one memo and somebody else can do it in five minutes, isn't that a good thing to know about an employee or a candidate? To which I would say, if being able to do it relatively quickly is actually what you're trying to assess, then of course time is legitimately part of the assessment, right? But if it's irrelevant, then it should be irrelevant, right? So it used to break my heart when I was a regular old English professor, and at the end of the year or the semester, I would hand back students papers in which I had written great, great tones of feedback, and of course they never picked them up, right? They wanted to know what their grade was. And it was the end of the learning experience as opposed to the continuation of it, which I thought was kind of sad. And some examples or projects are things that you might see in, you know, let's say a business class, like creating a marketing plan, or let's say designing a website, but we also have things like curating a virtual art exhibit, and you can imagine that type of project as opposed to the kind of exams I did when I was in college in which there'd be a picture of a famous painting and I identified it, right? I mean, it really kind of activates students' learning in order to imagine themselves helping someone else look at art or think about art. So one of the reasons why I'm so jazzed about competency-based education and about project-based learning in particular is that as I mentioned, it's such a wonderful way to assess the kinds of skills that employers care desperately about and that they're not getting from most graduates, right? Even really good graduates from really good colleges will often come into the workplace without what the employers regard as really critical skills. It could be writing and problem-solving, you know, it could be knowing how to talk to someone who has a different role or place in the hierarchy than you do. And every single year, AACNU does these surveys of employers and every single year the employers say, these are the things we really want, and here's what's not happening, right? It's this persistent gap we talked about yesterday and I think is really contributing to this generalized dissatisfaction with higher ed, fear or not fear. I would suggest that it comes about in part because our colleges and universities are set up around disciplines, right? So if you're a history professor, right, you may want your students to write well or want to try to help them, right? But your main job is the history content, right? And people say, oh, wait, wait, but we have interdisciplinary courses. It seems to me it doesn't get at the real issue which is how can we surface the kinds of competencies that every single college says on its website are critical, right? But that are somehow invisible in the general approach to teaching and learning. Again, I think that doing, I keep talking about it as though it were sort of a Zen quality or something, but I think when I read competencies that are really about, they're basically student learning outcomes in which the student is being asked to either regurgitate to put it unkindly or to reflect back or spit back or even just put in a test of some sort, something that really doesn't help them know, right? Think about our poor sociology applicant, right? They don't know what skills they have and they do have skills, right? So I think this is why Project Centered CBE has such promise. And if we want to call it Betty from now on, we can do that, but it'll be just among us, so. Thank you. Thanks. Steve, you wanna? Oh, welcome. Good morning, everyone. I'll try to get through these slides pretty quickly so we can get to those questions. I'm just gonna give you guys Thomas Edison's perspective on competency-based education. There we go. All right. So yeah, so that's me. You can follow me on Twitter. Sometimes I say insightful things. Thomas Edison State University. We were founded in 1972 part of that college completion movement trying to get folks to earn degrees. Our average student age is 37. We do a lot of prior learning assessment and we have a really good persistence rate with students that go through, as Mark mentioned earlier. We're competency-based education network members and we've been working on CBE for a long while which I'll get into. Yeah, we can skip through this one. All right, so Thomas Edison was founded as a competency-based education institution. If you look back in our college catalog from 1972, it actually says competency-based education. But that term has changed over the years and so has our institution. We earned a couple of FIPSI grants and got some money from the New Jersey State Legislature that allowed us to build the first online courses in the 1980s. And since then, we've really focused more on online courses than we have on the competency-based education that made us great. But we've found over the last couple years that there has been a need to return to our roots and return to competency-based education with the sequestration and the freezing of the tuition assistance for military. We lost a lot of students that just never came back. We saw more and more institutions moving into that online market so the University of Phoenix, among others. Cale's learning counts has sort of replicated to a certain extent and expanded what we do with prior learning assessment and new direct assessment schools like College for America are coming in and doing essentially what we do with prior learning assessment, but better and more effectively. Not only that, as Kate mentioned, students and employers struggle to understand what they learned in a course. If you get a B plus in accounting, what does that actually mean? And we need our students and the employers to understand what that means. And not only that, students were coming in to do prior learning assessments and maybe they didn't have three credits of knowledge. Maybe they only knew the practice. Maybe they were a manager for 20 years, but they didn't know management theories. We couldn't give them credit in our current system. So what we decided to do was to take a look at an individual degree program. In this case, the Associates of Arts and Liberal Studies because we wanted to tackle our general education. One of the main criticisms of competency-based education in its new formation is that it's really vocational training. It's workforce training. It doesn't have those soft skills or those power skills, as Kate mentioned. So we wanted to prove that it could be done. So we took a look at what are the degree requirements? What are the general education requirements? And then we took a look at what are the courses that satisfy them? So in this case, I don't know if you guys can see, but for ethical leadership is one of our requirements that all students have to take at least three credits of ethics. And there's a half a dozen courses that satisfy that. So what are the overarching themes and ideas in each of those courses? What are we asking students to do across that curriculum and can we design statements around that? We also took a look at the DQP and the AAC and U-value rubrics to help feed in and give us kind of a more global picture of what degree qualifications looked like. And then we designed our competency statements. So it's broken down for this degree into, this is only a screenshot at the top, but it's three groups, 16 dominions and 57 statements. They're each equated to some sort of credit hour equivalency, but they're not all the same depending on the size of the competency. And we've at this point worked with the Sailor Academy to design a curriculum around each one of those competency statements. And we're working on the assessments ourselves. So this is the model we came up with. Every student that enrolls in our CBE program will be given a list of all of those competency statements, the assessments that we're planning to have them go through and then the rubrics that we're evaluating them with. They'll take a look at those and essentially submit a portfolio assessment on what they think they've already demonstrated mastery on. So they'll take a look at their workforce experience, courses that they've taken, independent study that they've done and try to fit that in to those 57 competencies that you saw earlier. They'll submit that to us and we'll have a team review that evidence and then schedule an interview with the student. In that interview, we'll use that, the information that we gathered from their portfolio and probe deeper and try to figure out, does the student experience rise to the level of mastery? And then after that, we'll give them a learning profile saying here are the competencies that you've demonstrated mastery in, here are the ones that you tried and you have some experience in, but it didn't quite rise to the level of mastery and then here are the ones that you really need to go through the module for. As I said, after the diagnostic interview, they'll enroll in a six month term and in that term, they'll have access to all of the competency modules. They'll be able to go and do as many of them as they want. Since all of the materials that we're using in the modules are openly licensed, they'll be encouraged to annotate and to suggest modifications to the modules so that we can continue to improve and make them better and more student focused. They'll also give students a little bit of ownership over their curriculum if they've had a role in changing it and trying to make it better. At any point after the registration, as I said, they'll be able to move forward right to that summative assessment and take that assessment. So if they went through the interview process and we didn't give them credit and like, well, you know, I really do think that I can do this. They'll have the opportunity to do that and to prove it to us. And then finally, they'll be supported by a new faculty model. Traditionally in our courses, we have what we call mentors because all students at Thomas Edison are self-directed adult learners. So they're more facilitators, they're mentors, they're there with the student. In this model, we'll have coaching mentors and we'll have grading mentors. The coaching mentors will be assigned to an individual student and they'll be with that student for their entire academic career. The grading mentors will be assigned to individual competencies based on their expertise and they'll be able to provide substantive feedback for students upon submission. And then finally, students will be given a extended transcript. We're looking at the work that UMUC is doing, the Stanford's doing, the Acro is doing. And we're in the early stages of designing something that is our own. So students will have this competency-based transcript where they'll be able to show employers exactly what they did to satisfy the competencies. As each one of these links here will go directly to the product that the student did to satisfy mastery. So that's our model and I'm excited to talk more about it. Great, thanks Steve. Mary-El. Great. So I'm not gonna give you the slides. I'm gonna try to tell my story of how I ended up on this panel. And so my background is that I have a PhD in biology and I was, after getting my PhD, went into teaching. And I ended up coming back to MIT to work on online courses when the announcements of MOOCs happened. And so I worked on the first introduction of biology course with the Department of Biology and Eric Lander. And with that, we learned there's a big learning curve to the space. So we wanted to, so I talked to the department and they created a position for me for overseeing all the online projects, all the future MOOC projects for the Department of Biology. So that's what I do now. But it also entails working on projects for the residential courses with the MIT students and we create digital learning materials for them to help improve their experience on campus. And sometimes, for all our MOOC material we do use it on campus but we also have some of the materials first created for the residential experience that then end up going out to the public through MOOCs. So I work with two postdocs that are full-time and then some grad students and undergrads help out from time to time. The unique thing about our models that we are domain experts and subject experts but we also try to be the expert in technology, the expert in digital learning and the expert in the learning sciences. I mean, my job entails a lot of project management but we also do things in video. The one postdoc I had animated the videos ourselves so we did most of the course creation all within our own group. So it's different than the central models that most universities use where there's individuals doing each part of a course. And so we have that first introduction biology course and March 2013 it was released and we ran it a few times instructor paste with deadlines and we had a lot of activity and certificates through the edX platform were free at first and then they were free and for fee and then towards the end of 2015 they became four fee only and it was around that time we wanted to do something to sort of refresh the course anyway. So if we were gonna have a four fee certificate we want to help make that certificate more valuable and meaningful to the learners. So with that we had several motivations for trying to improve the certification of the course. One is that we wanted to just have the learning be more flexible for the learners so that it would be open. The course would be open all the materials, the problem sets exams for self-paced learning and for free so anyone can go in and use the materials at any point but also we wanted to have the experience be fitting the best for the learning sciences so that students had access to instant feedback. And then for the certification side we wanted to create something that would be more rigorous certification and in hopes that employers or academic programs would recognize this certificate as being rigorous and being something meaningful that they would understand that the learner has gone through something that means something that they really have mastered the content for introduction and biology material at MIT. And what we have now is that the course is open self-paced for any time. The certification is a competency exam and that competency exam is offered a few times a year in a one week window and it's timed. And we also, our motivations were to have the best coverage of the learning objectives for the course so we have about 200 learning objectives so it's very finite but we tried to cover that when we designed the exam to be the best coverage and then we also wanted the Bloom's taxonomy level of what we're assessing to match pretty well so we first evaluated an on-campus semester to support the material and what Bloom's level those are falling in and then we also evaluated the current course materials and then we created the exam to match the on-campus experience as best as possible and we wanted the problems with this final competency exam for an exam to be separate from the learning experience we could actually test biology in a more realistic way. Instead of testing just biochemistry, just genetics we could put it all together like real biology problems happen in the lab that you put together all different aspects of biology when you're solving something it wasn't, well this unit was on biochemistry so the test has to be biochemistry. Then we had one other driving feature just having a separate competency exam is the fact that it's pretty easy to cheat with online systems so we wanted to build something that would be better for the integrity of the honest students and try to keep things fair to them so we don't show any correctness we don't show any feedback on the competency exam and the platform is not enabling us to do that so we had to hack the system to do that at that time and we also have randomization of each problem represents a story problem that's randomized a randomized version of that problem so with all of that that's how I ended up here in this space but it's a small scale relative to a program scale type of thing. Great, so obviously folks are coming at this from some different places and I wonder if we could start a little bit on defining what competency-based education really is I feel like to get to the idea that this is a bit of a fad, I think it's a real one some of the programs we hear sound a little bit to me like they're just measuring learning outcomes which their creditor probably was requiring them to do anyhow so that's kind of the CBE light side of things and then on the more aggressive side you have direct assessment where do folks are familiar with direct assessment what that is, I think most of you are well this basically means if you know it and you can do it you can test out of it and move on there's completely untethered from the credit hour completely untethered from time and frankly freaks out a lot of people in traditional higher education and regulators as well but in that range it seems to me no grades, real mastery that seems to me to be kind of core of the definition of something different and emerging and I wonder in that if you agree with that and also when we talk about competency-based education it's often these adult students who can really cruise through things they've already learned, already know how to do what happens if you can't get past a competency I think that that side of it I mean this is not a gentleman C world that you're talking about here I mean you have to prove mastery so what happens in that scenario and what do you think about my kind of definition of what really is competency-based education who wants to take a first crack at that? I can see. So as I mentioned Thomas Edison used to be competency-based education and even now we have a degree program Jeff mentioned yesterday where sailor students can go through their courses and come to us for the challenge exam all the way up to a degree and some folks might consider that to be competency-based because it's self-paced there aren't no grades because it's prior learning assessment but for us what we're really trying to do is build that direct assessment modality because we think it adds additional value for the student in terms of being able to advance kind of at their own pace even if they only know a small piece of it they're able to be credentialed in that small piece. It also allows them to have more transparency about what they know because we're building this content and these assessments around discrete skills and abilities rather than saying you got credit in English comp. Right, right. Anyone else? Yeah I think the US Department of Education they have a definition of competency-based being that it's flexibility transitioning away from seat time into flexibility for the learners and that matches well and they even recommend online or blended programs that match as well with what we're doing and then I think really focusing on matching the assessments with what your objectives are with the content but also we do allow for our comp's to exam that you don't have to do anything in the course that if you just know it from work experience or if you know it from taking a course then another school is fine if you just take the exam and most of the learners that we're seeing who are taking the exam they wanna use it for a job application application to another program or to show a professional development within their own workplace. So they can do it just at any point if they feel they're ready to do it. So I mean I think I spoke at some length about my view of this, the only thing that you asked like a really important question Paul which is yes great if you can motor through but what if you need more time and it seems to me the key is the flexibility of the time not accelerated because one of the cruelties that seems to me of our system is that whether your semester is 15 weeks or eight weeks there's sort of this presumption that everybody starts at the same place and in lockstep so marches through to the bitter end and then it's a little bit like musical chairs that started well you didn't get it too bad you could have gone faster too bad. So because for College for America there are no grades but there's also no it's really self not self-paced it's flexibly paced so that if it takes you a whole bunch of times to master something it does and the whole idea is that the curriculum really supports the demonstration of competencies not the other way around right. So we often think of assessment as this thing that comes after the curriculum and says oh how much of this do they know and I think if you turn it around and say you start with what those competencies are and then think about you know how can you support students in developing them it looks very different. Just to add to that I think one of the unique things about competence based education is the coaching model goes along with it and the fact that you do have someone who's with the student throughout their entire academic career understands their goals, understands their limitations and can give them escalating feedback if they fail. So in a traditional course if you fail the final exam you fail the course and at least in our competency based education program you have up to three attempts and so if you fail the first one we'll give you additional support if you fail the second one we'll still give you even more support so I think that's an important component of that as well. Great. So some of you talked about the challenge of transcripting competencies. I mean it seems to me we hear a lot about exciting innovations in higher ed but it seems that the HR department is where some of those ideas go to die where folks in HR just look for kind of familiar alma maters and programs and what are some of the elements of your programs you really think are of value to employers and how do you convey them in a way that sticks? Anyone wanna take a crack at that? I mean in my case it's not a program it's getting people to see the value of an individual certificate and in that case it's really getting the word out there that our model of how we're doing the certification is different and the other part is we are in Kendall Square and we're in the biotech central area so we can just try to reach out to the people and the connections we have there to really explain to them how it's different and why it's different and if you want to know if your lab tech has this background or not and what level of biology they know they can take this exam and we also work with local programs for high school teachers and they use our materials because that's another group of people getting training because biology has changed a lot over time so knowing biology 20 years ago is very different from knowing biology now so a lot of people need updating and biology knowledge and going through the course or just taking the exam to show that you have that updated knowledge is valuable to some people. Anyone else? So College for America is a little bit unusual in that everyone who comes in comes in through an employer so but I can tell you that employers are actually thrilled about competency based education because that's the language they speak by and large but it's actually not so much the HR department it's sometimes the form that the HR department requires to have something reimbursed and I remember we were at a very large logistics and shipping company that I won't mention somewhere in the south and we had this conversation in which the high level people were really excited and somebody says but I don't understand it says the grade has to be at least a C and where will we put this in the form and there went you know now we actually worked a lot with employers and there are definitely workarounds but I find it's not necessarily the core learning model it's you know getting people reimbursed if their employer is paying for it or financial aid and that kind of thing that actually is where a lot of it goes did I? Sure. Yeah I mean I think the key is just to make sure that employers are part of the process that you know they've taken a look at your competencies that they value your competencies and that those relationships are set up ahead of time so that by the time that the students go through they know what they can do with it already. So a lot of interest in competency based ed but you know I have heard that some of the programs aren't as big as people had hoped they would be you know WGU has something like 80,000 students and the College for America, Capella and some others have really big robust programs in the several thousands of students but some of the other ones are really boutique and small and I wonder if part of that is kind of you know explaining what this is to people you know I write about this all the time and going under 1500 words is really hard to do it's really complicated so what are some of the barriers to scaling this up to getting to really being a meaningful part of the American higher education landscape does anyone want to take a crack at that, carry on. Yeah so even at MIT I gave a talk in May to our own people in digital learning and I had to start with the definition of competency based because it's not really discussed much using that term on our campus or around our campus and even within the digital learning space it's not discussed that much and so you know one of the few examples with this course and what we're doing of going that direction and so it is a lot of education and even within our own institutes in that case. Yeah I mean if you look at the marketing materials for CVE programs they don't say competency based education it's flex path, it's flexible option it's you know there's always flexibilities in there but we're not coming out and telling students like oh this is competency based education you know we all care about what it is because we're educators but I don't think it's made that jump into the mainstream. Should we though, I mean what do you think? Should, is it important that people think of it as competency based education? I think it's important from the standpoint of getting accreditors, regulators, lawmakers, and ed tech companies on board with it as long as we're all speaking the same language we can work towards the same definitions the same standards, the same systems. University of Wisconsin system has a high profile direct assessment competency based program I think it's pretty small though I have 1200, 1500 students right now but they have an online little ad for it and I think it's called flex path. It does a great job of explaining this in 30 seconds. I really, we're checking out actually. So let's briefly before we open up to questions talk about the regulatory side. Believe it or not we're in Washington this is an issue that is actually bipartisan. There's a bipartisan bill to create a demonstration project around competency based ed it seems that the past administration really liked competency based ed it seems like this one is not gonna at least stop it on the interest and maybe encourage it once they kind of staff up. But what would you like to see Washington do and the creditors do to both preserve quality, protect it but to also encourage the growth in this field? Any, anyone, I know this is not a simple question but I'll start with this one. I think the question has always been what do we as taxpayers wanna pay for and up until now it's been instruction. And I think with competency based education it starts a conversation around it should we be paying for instruction or should we be paying for outcomes? Should we be paying for what a student knows and can do? So when you change the paradigm to focus on that things like regular and substantive interaction don't seem as important because the reason why that exists is to make sure that students are getting instruction. Can you briefly explain that briefly? So regular and substantive interaction is a statute that differentiates distance learning from correspondence education and has to do with whether or not the faculty member is regularly interacting with the student. It has to be faculty instigated. And just briefly on that, so the office of Inspector General at the US Department of Education has been auditing Western Governors University which is obviously the big fish here on the regular and substantive issue. And if they find that they're not operating under the statutes they might label it as a correspondence course provider which would be very bad for WGU and would have would require them actually to pay back some of the federal aid they've received over the years. So this is a very hot delicate issue right now. And a lot of folks are wondering where the kind of regulatory side will go on competency based education. You know it's hard cause I'm torn honestly. I'm really interested in new models. I think they're serious issues with the current ones. And I'm aware that there's this rush to the bottom whenever there's a loophole. And so I do believe that the amount of money at stake is so huge that you do have to deal with fraud and abuse. You have to try to prevent it. At the same time, whenever I hear somebody say well, I don't know, you know, direct assessment, that sounds pretty fishy. And I think, oh an indirect assessment is okay with you? Like, you know that we never question the status quo. And I think with the level of scrutiny that it deserves to be. And we actually had a fabulous experience with our little shout out to me ask, great, great experience. But you know, it's a sort of different, that hasn't been the case in every regional end. There is again such a disconnect between, you know, the folks in the undersecretary's office and you know, the folks who actually are seeing, you know, does this fit the regulations or the folks who are in the IG's office that, you know, you can have a very positive movement that is stopped in its tax by people who are really essentially, you know, small minded if you don't mind my saying so. So, you know, I just think we have to go back, as you said, to what is the purpose of higher education? What's the purpose of, you know, the societal interest in it and support for it? And what are we as tax payers, you know, willing to demand, right? And it's not, you know, some of the things that are happening now. What is, you raise a good point about the double standard that it's just there. And you know, there's not a lot of interest in regular and substantive contact in 600 person lecture halls, but it just is what it is. So I think we'd like to try questions here. Anyone want to give us one? I think we have one over here. Got a microphone right, coming right there. You're Steve Brandt from Bay Path University. The point about do versus no, what your point in the beginning. So in the business world, do is either visual or proximity. It's hard to see do when it's virtual, right? So if, you know, when confronted with this situation, this is a particular way that we hope that you would behave in terms of assessing some kind of behavior. And the question is, how do you do some, how do you create competencies that cannot be assessed? That's hard to assess them virtually, or in words, or how, where does that go in terms of if it's done like in business, it's, you know, in many businesses, it's like face-to-face. Like if you're trying to teach people customer service, for example, it's about how they interact with the people who are in front of them, not necessarily, I do corporate learning. So that's sort of what I'm thinking about. Yeah, no, actually it's an interesting question because it gives me the opportunity to clarify what I mean by do. And what I mean by do could involve right, it could involve, you know, speak, sing, take blood, you know, write a reflective, I mean, there are any number of things. So what I mean by do though is that there is, in the Department of Labor's definition of competency, that you are using, you know, basically knowledge skills and abilities to perform a task. Right, so there's an implied quality to it that nonetheless may have a lot of knowledge behind it. So in your answer to, you know, how can you do that virtually, first of all, College of America is exclusively online. We had videos, we had audios, I mean, we now live in a pretty virtual world in many cases and all of my work virtually is virtual and people have very strong feelings. They think about whether I'm doing it or not doing it or doing it well. So I think if you focus on the sort of product in a sense, whether the product is an interview which you can tape and evaluate, you know, rather than on the imparting of knowledge, right, the teaching, I think to go back to your point, I think certain things open up a lot and, you know, so. When you have, you know, a flight attendant, a crew member who's, you know, if you wanna teach them, if a skill that they want or some behavior is, you know, exert some level of emotional intelligence when interacting with a in-civil where someone's not civil, sitting in the seat and everybody's got their cell phones out, if that's something you wanna teach because that's what the corporate experience needs to do, how do you do that when I guess there's ways? Well, I mean, we did that sort of thing all the time, right? And so part of what you need to do though, if it's both a teaching or I would prefer to say learning as well as an assessment experience, is that you have to be very clear about what are the characteristics of behavior that you're looking for or the doing that you're looking for. So, I mean, this is something the Army has done for years, right? I mean, it's when we think, it's really in the academic world, we tend to think a little bit because the paradigm of teacher-student face-to-face is so powerful that we forget there's so much to learn from how you train psychologists, how you train, I mean, you could watch a zillion tapes of people, you can model it, and the wonderful thing is when you throw in digital technology, you have all sorts of opportunities to watch it over and over again, to respond in certain ways to aspects of it, to slow it down, to blow it up. And so I think that there's a very robust set of opportunities here. I can add to that. In biology, we actually are adding more realistic experiences with online than what they are doing on campus with just a PDF of something or a paper exam because in biology now, when we created the first course, the first MOOC, we were trying to think, well, what can we do in the online space that we couldn't do in the one-hour paper exam? And we designed problem sets or exams where they can look at actual gene sequences and we had the right connections, the right software around that, we may have modified it some to be student-friendly, but they are looking at real genes. The stories are about mutations within genes that are associated with real diseases. We tried to create the problems to be a realistic biological experience. They also look at proteins and move the structures around in 3D space. You can't do that on a paper exam. And all of that is similar to what the research scientists use in the lab. And so we can do more in online. And there's also ways to test that with computer grading that it's hard for me and that's part of my job, but we have to know what can the platform do and what are we trying to test and then we need people like me to translate that too, like how do we test it in online? And there are tools where we can have students create data. We give a scenario, explain what's happening and say, okay, produce the data. They're not going to the lab to produce the data, but we can computer grade them producing the data from an experiment. Other questions out there? I guess you went in the middle. Not fully a question, but kind of a follow-up to that is as I think there's a misnomer that all competency-based education has to be 100% digital and online. It doesn't. You know, one of the programs that we're taking at our college and moving to a competency-based approach is welding. Most of the didactic piece can be done in an online environment, but we set up open lab spaces for people to come in and do virtual welding and then actually demonstrating the skills because they need AWS certification and they have to actually lay the welding beads to make sure that that is a certified weld and so forth. You can take a competency-based approach and if your program absolutely needs a face-to-face or a scenario-based situation like in law enforcement training where you're actually going to have to have them apply these skills in a mock scenario, the domestic or whatever, you can have competency-based taking approach where you're still removing the time, you're still giving flexibility, not kind of gig credit hours or anything like that, but I just think that so many people think competency-based means you absolutely have to have 100% online and that isn't the case. Yeah, just to take moderator's privilege here and tell you about one program I wrote about, Lipscomb University in Tennessee's, I believe 100% face-to-face competency-based program and actually watched their intake of new students behind a one-way mirror where they have them all day long do business scenarios to really set a baseline for where they are in their learning and it was intimidating, by the way, really intense experience. Anyone else wanna talk about the kind of face-to-face? I was actually gonna bring up Lipscomb, so I'll just cover that. Oh wait, just a couple more minutes here, any more questions out there? I see one over here. Thank you, hi, my name's Michelle Mills. I'm from the U.S. Department of Labor, Office of Apprenticeships. Just started though, just a couple of days ago. And I'm curious, we mentioned regulatory frameworks, so I'm curious the top two legislative reforms that would possibly help advance this initiative of movement. And the second question, how could we partner more effectively to promote apprenticeship opportunities and how is that evaluated in terms of curriculums, assessment, accreditation, et cetera? Thank you. Well for me, two regulations that I'd like to see change or go, regular and substantive for sure, that needs to be changed, especially if the focus is, again, more around outcomes. And the second one is with the experimental sites, there are certain financial aid waivers, for instance, allowing a subscription-based model rather than a term-based model, but I think just adds another element that we can all experiment with and play around with and add more value for the students. And to your point about apprenticeships, at Thomas Edison we have a department that goes out and reviews academic training programs, certifications, trainings, apprenticeships, similar to ACE and NCCRS, who we heard about earlier, and assesses them for credit. Certainly something like that could be done in a competency-based format as well. So at least in our case, the intake process, if you're coming to us with a certain apprenticeship, we would already know how that would speak to your requirements to the degree. Just briefly on the apprenticeship side, the President's Executive Order on Apprenticeship, I think it was last week I wrote about that, kind of opens the door, I think, to a competency-based approach in the learning component in that it allows, so to be federally registered as an apprenticeship, you have to go through an application process. I know you know this, but under the order, companies and education providers can kind of come up with their own definitions of what a registered apprenticeship would be, and the word competency gets bounded around a lot in that. But anyone else want to? Well, I just want to say, which is, first of all, I went like this when you said Department of Labor, because I worked there back in prehistory, and one of the things that was happening then is that the Department of Labor and the Department of Education were talking to each other all the time and really working jointly on school to work and what were then called skill standards and so on, and at least in the previous administration, I think that really, we all knew all about the Department of Education, and I think many people were less aware of what the Department of Labor was doing, so I think that that's a very productive, not making this sort of split between work, education, I think would be very productive, and I know you'll see to that personally because you have connections there now, so, okay. General thoughts on policy, just the idea of supporting a workforce that will have a lifelong learning model because that's required now, especially in science, it's changing year to year, so by time you get a four-year degree, everything's being taught in those courses is outdated unless the faculty are updating the courses each year, so the degree doesn't matter as much, but the on-demand, getting knowledge, if it is that a school put out a new course that is updated information allowing the learners to go use that and get it the moment they need it rather than the idea of having to have the degree only, that these on-demand learning experiences matter and getting recognition for that. Well, with that, we're out of time. I hope you join with me in giving a hand of applause to our excellent panel. Thank you. You're great. Thank you.