 I'm going to get right into it because we have a great panel here, and I'd like to leave times for Q&A with all of you, so I hope you won't be shy when we open it up. But I'm the moderator here. I'm Paul Fain. I'm a senior reporter at Inside Higher Ed, which is a digital news publication that covers Higher Ed. And the folks on this panel and some of you have kept us extremely busy of late. I would say the busiest of my career as a Higher Ed reporter. It's been an exciting year so far. So I'm going to just very briefly read the names and titles here so you know who these folks are. And then I'm going to let each of them talk very briefly about what they do and why they're here. So directly to my left is Sherry Lieder Kelly, vice president of the Council for Adult and Experiential Learning, which we call Cale. Then we have Mika Hoffman, who is the executive director of the Center for Educational Measurement at Excelsior College. And then we have Joseph Tebow, director of academics at Strader Line. Then Mark Singer, vice provost, Thomas Edison State College. And finally Devin Ritter, director of special projects at the Saylor Foundation. So I actually have written about each one of these institutions I think in the last month or two. So this is all very exciting for me. So just to very briefly kick this off, I was actually at a Cale conference a couple months ago, like Time Flies. And we were talking about prior learning assessment, competency-based education. And I said kind of jokingly to the panel, it was the night of the State of the Union, if you could get the president to address these issues, what would you ask him to say or do? Little did I know the President Obama in his State of the Union and the supplementary materials actually addressed these issues and encouraged higher ed and accreditors to speed to market these innovations, which was pretty shocking because it doesn't usually get that sort of airtime. And then Marco Rubio, in his response, in addition to the water thing, which I may do later, actually was even more explicit, I would say, in supporting prior learning assessment. So exciting time. So I'm going to start with Sherry if you talk about what you do. Sure. I'm Sherry, obviously. I'm vice president for learningcounts.org. And we are a nonprofit currently foundation-supported online service that helps adult learners align their experiential learning and knowledge, what they might have gained at work, maybe in a MOOC or volunteer work. We have them align that with college courses, currently a course match system, which is really kind of backwards in our view. But it's what higher education recognizes right now. They align their learning against the learning outcomes. And then we have an assessor who's a subject matter expert and a faculty member review the electronic portfolios and decide if college credits can be awarded. So students can come through. They learn how to align their learning, demonstrate their learning, and earn college credit, which helps them to significantly accelerate degree completion, get acknowledged for what they know, and to also save money. Turn it to Mika now. I'm Mika Hoffman, and I work at Excelsior College, particularly in the Center for Educational Measurement. We are kind of the prefab version of what Sherry does, which is that we build standardized assessments in areas where there are a lot of people learning that amount. Those standardized assessments are pegged to typical college courses, so interest psychology, or criminal justice, or bioethics. We have a number of these exams. And essentially what they do is they, it's the same as what Sherry does. You take an exam, you pass an exam. The exam content is pegged to learning outcomes for typical courses. And again, we translate performance on the exam into what would be expected if you had taken a course. That's worth college credit. We do, because we're a college, we transcript it directly, but you don't have to be enrolled in the college to take an exam. You can use that as credit anywhere that will take it. I was tuning out there for a second. Apologies. So next up, Mark. Oh, sorry, Joe. OK, I'll go. Sure. Joseph Thiba, the Director of Academics for Straterline. We offer the typical college courses for $99 a month plus a small or nominal start fee. And I'm happy to say that we're partners with everybody here. So do we have some very low-cost options from Sailor and others that can, student can take. It's a kind of more likened to a traditional online college approach, though it's self-paced, so students pay only for the time that they're using. And they can transfer that to either the American Council on Education Credit Network, or to one of our three dozen or so partner colleges. Mark, now it's your turn. All right. I'm Mark Singer. I'm with Thomas Edison State College. I'm the Vice Provost in charge of the Center for the Assessment of Learning. And what we do in that area is just try to figure out all the different ways we can give students credit for what they already know, whether it's through exams assessments, whether it's through portfolio development and assessment, or whether it's through an evaluation of the training programs or licenses or certificates that those students have earned outside, either in corporate training or military or something like that. And so we really see ourselves at Thomas Edison as kind of like the, within the college is like the Department of Flexibility. We're just trying to figure out all the different ways we can map what students know to our degree programs and to sort of be as creative as possible in evaluating what they do. And we've also been working with the Sailor Foundation for about a year. And as part of our whole flexibility thing, we're developing exams that will align to sailor courses. So that's a student who finishes a particular sailor course and we can then take our, a TCEP exam, that's the name of our program. If they pass our exam, they get credit for the sailor course. Devin? I'm Devin Ritter. I'm the Special Projects Administrator with the Sailor Foundation. So all of my colleagues here have kind of spoken to the fact that we have a working relationship with them. And we kind of see ourselves as being the learning piece that fits in with the flexibility of their assessing that learning. So through various means, we are taking our content and matching it up to the colleges and universities and organizations like KIL and organizations like Straighter Line, who can benefit students by allowing them to take what they've learned with Sailor and actually transform that into college credit and hopefully into a more affordable college degree, which is something hopefully we'll start talking about a little bit more. So yeah, our first theme is credit worthy assessments. If someone wants to disagree that the college credit is still the coin of the realm, please have that. But let's start with assessments and because I'm the moderator, I'll talk for a second because I'm allowed to do that. So as you all may have heard, in California, there's a bill that's been floated to create a pool of online courses to serve students of public colleges. And the way that the legislature has proposed making these credit worthy is through direct credit. So basically if you take, let's say, I'll just pick one randomly, a Straighter Line course, you would receive credit through the AC credit recommendations that you could then use, as I understand it, at the community colleges or Cal State or potentially University of California. And by the way, for context, the California community colleges turned away 600,000 students during their recent budget crisis. So we're talking about a lot of capacity. So on the other side, we just discovered yesterday that there's a bill in Florida to do a similar but different thing. And the details are still a little vague and we're still trying to figure them out, as I think some folks are here. But to encourage colleges to use a pool of courses from outside providers, and correct me if I'm wrong here, but where the credits would actually be through assessments that I believe the colleges would manage, which is more of a direct assessment. So could you all talk about those two methods and what you see as potential upsides of each method? Who wants to try first? Sherry? I'll try. Well actually I think that what we're seeing, especially in California, is that policymakers and taxpayers are just tired of business that is usual with higher education. And although the California model is actually one that a lot of states wanted to emulate maybe 20 years ago, it became just so expensive and onerous that now, and you're seeing a lot of governors do this, are trying to find a way that they can somehow enable students to be able to kind of bypass the entrenched part of higher education and come around and still get the credential that's going to be recognized by the entrenched part of higher education, which is kind of a challenge that we face at Learning Counts in that students come to us knowing what they know and what they can do and then we have to retrofit them back into the course match credit our model so that it can go onto a transcript and be recognized by all the colleges and universities. If we were able to move the other direction to simply competencies and competency assessments, that would actually serve the students better and I think that's kind of in line with a lot of workforce development people are really asking for at this point. Sure. Someone else want to jump in? Devin? Yeah, well personally, my view on those two models, I think the California approach is more efficient and actually ties in what more people are doing. One of the benefits of that approach is it allows for a student to take online courses such as sailor courses or straighter line courses or hopefully a lot more courses where students can be able to master the learning objectives that are needed in college course. But the second part of that is it allows for the assessment of that by various means. So hopefully they can take courses that could then be assessed via Excelsior or Thomas Edison or a straighter line exam or a kill portfolio and California will then say, yes, we will take that assessment because we've already had leaders in education prove and validate that you did that. The Florida model to me is great in that they're allowing students to save time and work on their own pace and learn online, but they're taking a burden upon themselves of having to do the assessment which is understandable in some means because academics like to have some of that freedom to test themselves. But I think hopefully and seeing, and Miga can speak to this probably, the incredible detailed approach they go to exam development, that Florida doesn't need to do that as well. They can trust that Excelsior can handle the weight of that or that Thomas Edison could handle the weight of that. And so you might want to speak more to that. I would like to speak to that. As an assessment nerd I have to say it may not be rocket science, Michael talked about rocket science, but I don't know that assessment science is quite that hard but it is a science. It isn't just I know the subject and so I know how to ask people questions to find out if they learned it. It's more complicated than that. And in my experience people who are experts in English literature or calculus or algebra aren't necessarily experts in how to assess what people know about all those subjects. So I get very concerned when I hear about well the colleges are just going to assess it themselves because the quality is going to be variable. Some of them will do a great job. Some of them are going to have people who really understand how to pull out the important things that the students need to know and they're going to be able to capture that, they're going to be able to do it reliably and that's going to be great. And then other places it's going to be random where the people are going to build bad tests and nobody's going to be there to check on them. Can I ask, is there any quality control when colleges do their own challenging exams or who's looking over their shoulder? I have no idea. Now it's possible that the accreditors do but in my experience accreditors aren't assessment experts either necessarily. Which isn't to say they should be. I don't think that's necessarily their job. The job of an accreditor in my opinion should be to make sure that the pieces are there to make sure that the institution is doing its job. But I think something like challenge exams tend to pass below the radar of accreditors because it's such a minor part of what goes on. If it now becomes a cornerstone or a mainstay of how education is going to work it's going to need a lot more attention. It's going to deserve a lot more attention and I'm not sure that accreditors or people out there know how to tell if it's being done right. And so to get back to Devin's point I think there are a number of organizations, not a large number, but there are a number of organizations that really know how to do assessment and a lot of the assessments that are done are at scale. And by at scale I mean our exams tend to be machine scored. So they're at scale in the sense that any number of people can take them and it doesn't cost anything particularly to score them. They're at scale in that sense. What LearningCounts.org does is at scale in the sense that they have very well thought out standards and procedures and rubrics that have to be gone through so that they have that assurance that even though it may be an individual level assessment the process can be trusted and the process is transparent and the process is something that experts have vetted and know is a decent process. So those are the experts. It's not an exclusive club. Anybody can join that club. You just have to know how to do it. And so what I would call for is we need to make sure that in California and Florida there ought to be standards how do you do assessments. If the schools choose to do it themselves that's fine. I think it's a waste. I think if there are assessments out there for intro statistics you don't need to school to do a challenge exam. They can take an intro statistics exam. There are plenty of them out there that are done by reputable organizations. Sure. Anyone else want to jump in on this? The only thing I would add is that in either scenario California or Florida in a sense it makes my job easier. Because what we're trying to do is allow students to earn credit for low very low cost. So if you are going to draw a line in the sand and says this assessment will get you that credit then we can work with experts to build a course to help students pass that assessment. Or vice versa we're going to say it's these learning objectives that make up this course. So you have to achieve these learning objectives and competencies in order to earn credit and the institution that does that just has to jump through a few hoops and become accredited either by the state or some other agency that also makes it easy because we have a recipe to work from. So if we could all get together, if all states got together I think they would come into agreement in some cases of what college algebra looks like. And that's I think is one of the problems currently is that it's an entirely fragmented marketplace where everybody has their own ideal version of college algebra which is incredibly inefficient and causes all sorts of redundancies. With the Florida model the only thing I would worry about is redundancy in what the student has to pay. So I have to pay to learn potentially or I can do it for free but for free still may need some additional resources. I may need to go spend time. I may need time and effort but then I also have to pay for this test which could be a high risk test. If I fail what happens if I'm out that money. Whereas in a course way you can learn towards mastery. You can stop and say I don't get this concept yet. How do I learn that? So just so folks are clear because I often am not when it comes to what they're actually talking about here because it is complicated stuff. We're talking about challenge exams that students take to prove their prior learning for credit. And couple of the most common ones are Excelsior's exams and the CLEP exams are offered by the college board. And we're talking $80, $90 a pop so a pretty affordable way to credit. But you all brought up a point I think would make for a fun lightning round here. When we're talking about standards where should the feds get involved and help set standards for that? Everyone loves that question. Anyone? I don't think they're the right people. I hear that often. But I think that gets to the question of encouraging innovation and preserving quality which often seems to be at tension with this. Do you agree with that? Yeah I think one thing that the feds could bring that individual schools couldn't is data collection and if they're collecting data appropriately and actually giving statistics on when meeting certain learning outcomes translates into having certain skills and success in certain areas then they can potentially better kind of reverse engineer what a good course is and what is a needed course. And they have a much larger sample to pull from but then they can count on the people who are actually experts in developing the courses and the assessments to actually fit those models and perhaps. But there's a lot of bureaucracy and there's a lot of people not getting the right information so there has to be a lot of teamwork. So let's switch gears and go to open educational resources so we are here. In the place that it has in this you know I talked to a student who used open educational resources to prepare for a CLEP test. He said he spent two weeks it was free obviously because it was open and he passed the test for $80. Where does fully open material play a role in being paired up with these assessments? And where does it not? I guess I'll start with the lighting round. 30 seconds or less. Well increasingly I think prior learning assessment has been around for almost 40 years and it has kind of a connotation of being really old and stodgy and conservative but as more and more people gain access to the web and web resources and see the foundation courses I think we're going to see younger and younger aged people, persons trying to take advantage of this. Like right now our average student is probably in their early 40s. But there's definitely going to be somebody at age 16 that's going to say I want to be able to align what I've learned on my own because I was just interested in it to something that's going to be meaningful to an employer or to my future at college. I think the OER, there's just so much potential and right now it's a matter of kind of helping us all to align and be able to raise awareness around all the opportunities for demonstrating that learning and getting credit for that learning beyond the constant authentication of who took the Zaylor course or not. That's a matter of demonstrating the learning. That's a good point. In the California model they're looking at about $140 for these courses that would be in this pool. So let's say Coursera has a course in there or Udacity. That's no longer a mook. It might be a mock. Just so people are clear that's not really fully open anymore. I wanted to follow up both on what Sherry said and then also something to have said in response to the last question. One thing I would say about OER we do align our exams with OER as fast as we can but it's changing so fast that we can't really keep up in the sense of do we know everything out there that's open that aligns with something in our exams. One of the reasons why we love working with Zaylor is because they talk to us and we go back and forth and if they're going to change something they let us know and so we have a relationship but most OER is just kind of random and it's really hard to say if somebody says hey if I went over here and took this OER can I pass your test. So the second piece of that is what Joe was saying earlier which is the learning outcomes are really crucial to all that. It's the triangle. You've got the learning outcomes here. The OER are addressing learning outcomes from this direction. Assessments are addressing them from this direction and as long as everybody is clear on the learning outcomes you should come out okay. So what that means is it's incumbent on providers of assessments to say here are the learning outcomes that are associated with this assessment. You pass this assessment this is what it says about what you know and then at the same time providers of OER can say here's what you're going to learn using these resources. This is what we're trying to get you to learn using these resources and that way the consumers, users whatever you want to call them the learners can say okay if I want to get credit for and I'll say statistics. If I want to get credit for statistics I can go to the exams content guide which is freely available. This is true of most of the large scale exams or you can look at the content guide and see what the learning outcomes are. So I can see what I need to know. I can use these OER resources and they should be telling me what the outcomes are so I can learn those. And then as long as all of that is clear and open and transparent then I think everything works out well. I think the problem is a lot of open resources are just hey I put this up because it's cool and nobody's doing the extra work that it takes to put up clear learning objectives which is fine open should be open and there shouldn't be extra regulations on it. Openness is part of the charm. But that's those learning objectives are what's going to enable people to do the matching. Getting back maybe a little bit to your question. I think that OER works very well with certain higher ed models. One that's really learning outcomes based on what you're really thinking about in granting a credential is something like well the person who's got this credential is able to do this, this and this and has these skills and has this knowledge it doesn't work very well with the more traditional model in which a college or university sees itself as somehow branding or leaving its stamp on its students. So you don't come out of Harvard if the majority of your courses were through OER a Harvard man in the way that you might have if you'd set through all those lectures. So that's something that's missing and I guess people can decide whether or not there's a value to that and I think for some people it's out of their reach or it's not practical, it's not what they want. What we do with OER is something that's more the learning outcomes focused model where our students are all adult students. That's really what our focus is and there are a few schools like us mostly in the northeast that are only open essentially to adult students and so we assume that they come in with a certain amount of knowledge that they've picked up from various places. However what we find over and over is that the knowledge that they've come to Thomas Edison with is not neatly packaged in three credit bite size pieces. They might know two credits worth of a subject or seven credits or they might know the practical side of it but not the theory side that overlays it and so what we're thinking about in regard to OER is saying to a student, alright well this is what you've brought us. Here's the gap between that and the courses that we offer, the curriculum that we offer which is still very much course based. So you can turn to OER to color in to fill in those missing gaps. Now it does change rapidly but we're thinking more and so the thing that we thought we knew was true of that particular move is no longer true but we're looking at it as more of a just in time model. So you're coming to us now and you're saying I've got this knowledge and so we will research what's out there and say well here's something that might fit. It's not itself the final product that we want you to have but it'll get you ready for an assessment which will then measure your skills and knowledge. So some of the sailor courses work very well in that regard for us. Can I just add one more thing to put in a plug for OER University? I don't know if people have heard of this but it's a consortium which includes Thomas Edison and Excelsior that's committed to providing credentials primarily based on OER where the members provide OER but then they also commit to trying to find ways to get credible assessments for that OER so that you can build up a degree that comes from your home institution but that's based on assessments which are in turn based on learning that was gained through OER. So that's a neat initiative to look at. And they've kind of mentioned this but I wanted to reinforce what I kind of believe to be one of the strengths of sailor courses and why our courses work so well with the assessment models that they have is one nature of openness in that the courses themselves are transparent and individuals who are looking to build assessments have a much clearer picture of what our courses teach to and also the fact that we've taken the approach of, we've designated the learning outcomes that we think are needed and even if we change the content we're not going to change the learning outcomes. We're going to find the best open content to continue to teach those learning outcomes so that even if our courses change and our courses do change because we have now a mission to be 100% open resources that we know that our students can still take that same course and still pass that same assessment that is really validating the learning outcomes that we're building to. So it kind of makes us more flexible and able to work with these individuals on a long term scale. So I think it's probably fair to say that a lot of the energy for all of this is the interest in adult students, but traditional age students too to be able to be tested for what they know and receive credit for it and to basically test out of courses or to learn at their own pace is driven in part by concerns about the cost of college as we've heard and the time to degree and training our workforce. It might surprise you all to hear though that some people are not that excited about all this and I encounter people who are very skeptical every day actually wrote a story today about Thomas Edison having a really interesting partnership with a community college in New Jersey, Warren County to create a degree track for veterans and it's a really cool story, not the story that I wrote but what they're doing but they basically allow veterans to qualify for credits from their training and experience up to 45 even if it pairs directly with courses. So I knew what would happen immediately people began criticizing this and one of the first critiques I saw the community college actually sent its president to boot camp on Paris Island for a week to kind of get a sense for what the training was like and what the training was like in terms of credit and the Marine Corps basic training under A.C.E credit guidelines is eight credits, eight college credits and they were impressed with the level of depth, the learning outcomes in the Marine Corps training. Some of our readers thought eight credits for the Marine Corps training, come on, pull ups, what's up with that? I don't think that's fair but I'm just saying what do you all say to critics who don't believe that non-credit learning should count for substantial credits? How many pull ups? Probably a lot. Well often that comes from ignorance, I mean I'm not trying to put any red flag words out there but a lot of it comes from ignorance and it comes from just not even really knowing what we're talking about or the science that goes behind what we're trying to do and it comes from wanting to maintain the status quo to be able to keep higher ed the way it is where you have the experts, the sage on the stage, you have the things that you have to do to pre-rex to be able to get into the course and all of the different gatekeepers along the way and it's concerning because that particular model which actually up until maybe the last year or so is the one that most of our policy makers in Washington and even in state government, they had all gone through the traditional way like they may have gone to Harvard, they may have had elite education and therefore they never once gave a thought to people who don't always go down the same pathway and so I think when you see that kind of criticism, when we are trying to help people to get validated and move forward and have pathways, I think it's really unfortunate. I think if I'm not mistaken, most of that training was reviewed not by us in this case but the American Council on Education and their members include 1600 geologists the big one too, the elites as well as everyone and the concept behind that maybe this is a group that already knows this but is that they've already learned through this training whether it's military or otherwise something that they would only have to duplicate in a classroom and I would say there are some people who would say that learning that you get through experience is really of more value than something you get in a classroom I'm not sure if that's the conversation we want to have now but because they really internalize it and they really know it so to have that person come out of that training and then sit in a classroom and go through the textbook version of what they just did seems pretty silly and so all this program and all ACE is doing is just acknowledging that for them they do it by mapping this training to learning outcomes and other standards that they find in college courses it's a pretty rigorous process I would say Part of the problem and the stigma and the lack of adoption is schools and individual teachers not really understanding the process behind ACE credit recommendation and CCRS credit recommendation and not knowing the fact that it's their colleagues it's actual college professors and subject matter experts who are validating this learning are just doing it on a wider scale so that they don't have to do it again and if you look at regional accreditation and some of the things that go into a school being regionally accredited it has nothing to do with the actual quality of the individual courses and what's being taught in those individual courses is saying that the institution is being trusted to teach good courses but they're not necessarily always speaking to their courses whereas with ACE and CCRS they're actually looking at the individual learning that is taking place and making a valid judgment on whether or not any schools should accept that for credit and I think once that becomes a little bit more widespread and I know our colleagues here at Tina are doing a really good job of kind of spreading that message but I think that's an important thing that needs to be reinforced is that who is kind of doing that making that recommendation and why other people should trust it. Would you actually explain briefly what Tina's shop does so folks here? I knew this would happen I think all of us probably can in some aspect but I'll speak specifically to three courses that we put through review so we kind of reached out to Tina and CCRS to see what would need to be done with our courses to have them reviewed and recommended for college credit and we spent a good amount of time kind of beefing up those courses and building strong assessments for those courses and ensuring that students would be learning what needs to be learned in the equivalent of that college course and being assessed appropriately for it and when it came time for review, CCRS came to the actual Saylor foundations and brought with them professors of these courses to kind of go through our entire course and our assessment with a fine tooth comb, asked us questions about why we did certain things and looked at all the exam questions and asked us about how they were assessed so that they could understand that material that we're teaching to and the assessment that is being used to validate that learning is just as strong as end course in the classroom that they gave so that they could feel safe saying yes this course is worth three college credits and luckily we've had partners like Thomas Edison and Excelsior who understand that process and now allow our students to take that course from us and transfer that credit to those institutions and say significant amount of money. So I want to open up to questions in a second but I'm going to ask another really easy question. I think a lot of these innovations challenge the credit hour and I think a lot of people view the credit hour as a barrier to going further with self-paced learning. What do you all think? Do you think that it's time to revisit the credit hour and its application to higher education? So there was an easy one? I was being sarcastic. So we field a lot of calls at ShareLine where students ask what's the average time it takes for a student to finish a course and in general we say around 40 days. That's kind of indicative of any of our courses many will take longer, some will take shorter, but it all really depends on the student and the student's prior experience, their aptitude to learning, their aptitude for online learning, the quality of the resources that we're providing their ability to use to leverage some of the support resources that we're providing. It really is all dependent on the learner these days, but the semester credit hour is based on this kind of number picked out of the air that's hundreds of years old and it says you have 16 hours of classroom time and that's what three of those equals a semester and you add all those up and you get 120 and you get a degree. So it's arbitrary in a sense but there's not a good alternative currently and to be honest, if it were predicated on some other thing, learning objectives, they being, I think if we were to come to agreement and put people on this table and say come up with an alternative would be based on the semester hour. What's the number of competencies or the quantify what's learned in a semester hour and then let's turn that into the new currency and I don't have another alternative. Yeah, I would just say that it's not the credit hour that's the issue as much for me as the fact that the credit hours are designed to measure courses. You know, as Joe's saying if you could tie those credit hours to competencies or the kinds of things that you want somebody to know when they've completed a degree, that's fine. But right now our degrees are based on the fact that somebody got a passing grade in 40 courses and I don't know that that just knowing that is enough to tell me that that person has any skills except that they know how to cram for tests and things like that. And to me, that's where the issue is with the credit hour. Each of those is worth three credits and it's not really building toward anything. I think that's really where the drawback is because anything you replace credit hours with is going to also be arbitrary in some way. You know, it's just going to be all the little building blocks like the sort of Lego pieces that add up to the degree. You know, you have to count it somehow I suppose. Sure. Sherry, did you want to? Yes, I think one of the things that's been a reinforcement of the credit hour is Title IV funding. You looked at the numbers on the screen around student loan, student indebtedness when they graduate. Many of our, what we do here, because it's based upon assessment, it isn't eligible for Title IV funding. So in order to kind of help to really truly innovate and move to more competency based show us what you know, show us what you can do, kinds of degrees where employers can really actually see what they're hiring beyond the brand name of the institution, then it's going to continue to be like this. We need to have, and I know we have a lot of think tanks that are considering this, how would we be able to award financial aid for students who are going to be taking an exam, or students who are going to be going through learning counts for an assessment. I saw a very interesting outcome of legislation in Tennessee where they've gone to performance based funding for all of their public institutions and community colleges, the Board of Regents system and the UT system. And because they've gone to performance based funding, they actually get additional funding for students who are making progress, who surpassed milestones. And by getting credit through examinations through learning counts, by actually acknowledging prior learning assessment, faculty members who really weren't supportive of that before are becoming supportive. Deans and administrators are getting very interested in that because they could take a funding dip if they aren't actually moving people on to degrees faster and being more productive in terms of the output. So that's another, I think the whole incentive system is weird right now for credit hours. I'm sure it's kind of briefly touched on this speak up your microphone. But one of the other important aspects, I think, is from the employer end. Part of the reason why I think there's still credit hour and degree base is because employers continue to take a degree at face value and they look at a resume and see, oh, this person graduated from this college, that must mean something if they kind of try to flip that on its head themselves and start saying no, I need to see a more detailed report of what these individuals have actually learned that you're sending to me, then that can force it to be broken down into some more competencies and re-evaluate the way from structure to degree and also be able to break down courses where there's overlap so that if you're taking one course that or two courses that cover half the material the same, then perhaps don't view that as six credit hours, view that as one and a half courses worth of information learned and kind of be able to detail those learning outcomes. That could be more useful but I think a lot of that effort needs to be done on the employer end of saying these are exactly the type of employees that I want and universities and colleges have been around for a while to serve the economy and the employment force so they kind of need to have back and forth conversation on that. So I want to turn it over to the audience here in a second but I'm going to be a big stickler on this lightning round so I really mean like ten seconds here. You kick off I think an interesting question, how big can this get? You know in the context of I think most people don't realize this 20% of American higher education ballpark is the traditional residential college student. The norm is the non-traditional student but how big do you think these innovations can get? And it's hard for me to call them innovations when they've been around for 40 years but prior learning assessment competency based education where do you see the movement going to a normal store with Sherry? Actually because the movement's been around for so long we say we're finally in the spotlight. I think as long as we really take advantage and work together to you know whether it's the badging movement or competencies or credit hours or say the foundation courses examinations I think that we could really, this could go very far actually. It's making people think about learning differently. I would agree with Sherry, California's a big place. I think as people start thinking about competencies rather than just seat time that's going to be the key that turns the lock. To be honest I think what's the indicators going to be how many students have transfer credits and we can hope for 100% of students in the future having some transfer credits that they either got for free or for very cheaply that would be a mission accomplished. I think there's really two pieces to what you said but I'm just going to give them five seconds each. When we talk about competency based learning professional schools already do that. This is really an innovation only for the liberal arts. Our nursing school does that at the applied science school. This is not an issue for them. The other piece as far as prior learning assessments concern I think it's really going to take somebody doing some studies somebody who gets a degree that's primarily accomplished through prior learning assessment what do they then go on to do? How skilled are they? How do they compare to people who... Yeah, I think we're taking steps in the right direction and California is proving that by even considering this bill and the other one that they have on the table of having a fifth type of university system of just going for assessment. Which actually if any of you were at the summit last year Michael had that idea and told Virginia to do it but California I guess came around to do it at first. But as Mark said we also need time and we need data in order to support these models so California goes ahead and does it and then they start producing a better caliber employee than other states than that is a marketable thing and that will kind of hopefully resonate. I'm sorry, the thing about California I just want to mention the things that are going on in California in Tennessee, Wisconsin and Florida they're really based on money though they're not based on improving the quality or making things more accessible to students they're just like how do we get them out cheaper? And that's a very different... What more capacity that's not being met in California? Well yeah and so rather than build new campuses let's do this. And that's not really addressing the same issues that our institutions are. Alright so I'm under spotlight I can't really see you too well here but Hi I'm Doug Mellinger I've heard a lot about what you guys are talking about I'm on the board of two colleges out in California. One is a nationally accredited board certificate and associate degree program and the other one is a bachelor and master's degree. One of the things that we've seen out there is that the vast majority of the students that are applying are quickly unprepared to come to medial math that has to be given because of what they're getting in high school is absolutely staggering what we've had to do out there. When I think about what you said Mark in particular was actually because when I think about this country the number of people who ever stepped foot in a college and the number of people who actually ever get any sort of degree is absolutely ridiculous. I'm not really hearing about how are we going to deal with the vast majority of people who really never are taking access to education and the outcome of actually preparing people for jobs where we've got millions of jobs in this country that are available with nobody to take them because the skill sets are not much and so I just think the whole system is completely broken and I haven't heard a lot of the real fundamental problems coming out of this. What do we need to do to actually start solving some of this problem? I think this competence is very interesting. How does that match up? Well in my opinion one of the problems that colleges and universities taking on the burden of remedial education themselves there are tons of resources out there that students could use to learn remedial courses but what usually happens is students will go to college and be forced to take a test and then be told that they then have to at that college take a remedial course which if they don't pass they can't move on to take other courses instead of colleges and universities directing those students to free alternatives to better prepare themselves for those resources and then be able to back when they are prepared and do have the skills necessary to actually achieve what the college wants them to achieve and I think that's one of the big issues is that so much money and so much time is getting spent on colleges not doing what their actual major skill set is which is the higher level learning and the other resources that go around it and reteaching what is being taught or what should have been taught in high schools so I think they need to spread out some of that and make use of the free education providers and other resources that are available to them and kind of better guide students in order to better suit their needs and set them up to be more successful. Anyone else? Yeah, sure. I know the again two parts to your question but the second piece the AAC and you has done a number of surveys I guess with the Peter Hart of employers asking them what kinds of skills they want and those surveys never say I want somebody to have 12 credits in math. They say I want somebody who knows how to communicate effectively, ethics, global perspective and things like that and it's a question really it's something that can't really be addressed by the current system it's something that some of these more competency based programs or ones that are focused on those specific learning outcomes are the only ones that can get close to that it's hard to convince people who are used to teaching their specific subject and their specific discipline that what's really needed is a better alignment with what students are going to be doing after they finish school. As far as remedial work is concerned Thomas Edison State College offers no remedial courses at all. We consider our students to be self-directed adults that's the phrase we toss around all the time but we've been looking at OER as an option especially OER that's self-paced where students can take as long as they want and kind of focus on different areas as they prefer and then once they've gotten that to a place where they feel comfortable then they can enter I'm not sure that would be the solution for everybody but as Devin said there are all kinds of open resources out there that address those kinds of things. A bridge to success is the one that comes to mind right now whether there are dozens of others. I see a hand over here. Isn't one of the problems to get to the first part of that question of what you were addressing just a complete lack of interaction or collaboration on curriculum and development certification between the employer and the educational system. It seems like the bubble of OER or the bubble of educational resources still gets caught inside its own thought as opposed to making that connection with properly employable workforce. I know Coursera was planning to work directly with employers to develop courses that would address these specific issues I wouldn't know if others are but I can throw in there. I was just going to say that certainly community colleges work all the time with their local employers to figure out what they're looking for and what they need but what they need I think for me anyway the bigger issue is we certainly wouldn't be talking about MOOCs if they weren't being offered by prestige institutions. A lot of us that have been in the field forever if one of our colleges or universities put up a massive open online course that had maybe a 10% success rate we would be left right out of higher ed and would be really on the fringe and I think what it points to is this huge digital divide in our country of the haves and the have nots and our aspiration to be able to taste a little bit of MIT or Harvard through MOOC but we have students who come to us, one in particular I can think of right off the bat who is homeless living in a shelter, she's driving to a library to get internet access. When you talk about the disconnect with developmental education because the K-12 hasn't done its job we really have to be more than just lip service think of the P-16 system and better align the quality of education throughout the system and we've been so decentralized and have so many feet thumbs that I totally agree that it is broken but I think we have a lot of smart people that are really trying to think about ways to make it better. So let's do one more question and while we're doing that this is not an endorsement but we haven't talked about two of the big players in digital. The University of Phoenix I hear from a lot of employers does some of the best work in terms of structuring degrees around competencies that employers prefer as do community colleges and Pearson has been working on digital developmental products for a long time as well. Any, okay I see one back there to the left. John Edelson California State University Monterey Bay I think I attended some of those faculty meetings in California that the first speakers spoke about. Also the e-portfolio California project we were an outcomes based campus that really moved away from high states testing to outcomes and portfolios. I see the movement with MOOCs and something she described is really going more towards high states testing and limited testing that doesn't provide as rich of an understanding and high state testing as we're so well with no child left untested. What are your thoughts about other alternatives to high state testing and portfolios? I think high states testing it sounds like when you're talking about high states testing you're talking about the heavily standardized testing. I would consider testing that leads to a college credential to be high stakes whether it's a portfolio an essay, a term paper, anything like that. But to get to what I think you're asking when you're talking about a multiple choice machine scored exam determining the fate of a student versus are they doing a portfolio or are they doing something more interactive. I think there are all kinds of good assessments and there are all kinds of bad assessments and it's possible for just about any type of assessment to be either good or bad. There are a lot of bad multiple choice tests out there. There are a lot of good ones. There are a lot of bad portfolio evaluators out there. There are a lot of good ones. So part of the trick is not to make sure that what you're assessing is really what you want to be assessing and that whoever's designing the assessments is finding the right way to get what you want to assess. If you want to assess whether somebody can marshal a bunch of facts taken from many different sources and write a coherent argument right now the technology doesn't exist to score that by machine. So having somebody try to do a machine scoreable essay or multiple choice test is not going to be a good way to assess that particular competency if that's what you're interested in. Then you have to get to something that's more expensive that probably involves humans but even there just because it involves humans doesn't mean it's high quality because then you have a reliability problem. So I think the trick is to have an increased awareness of the appropriate and inappropriate roles for assessment and to make sure that people who understand that are designing the right assessments for the purposes that they're purportedly for. We're going to wrap it up on that. Please join with me in thanking our panelists for an excellent session.