 Hello, welcome to another Dr. Sather Chalk and Talk. This time I'm going to talk about something that may seem a little bit abstract, but actually concerns anybody who's involved in education. And as we're going to see, that's a much wider range of people than one line original thing. The topic that I'm going to talk about is assessment of student learning. And the question that I'm dealing with here is not one that I've actually been asked in VU or in YouTube or emails or something like that per se. Instead it's coming out of, you might say, the last year of my involvement in re-agreedination of Fayetteville State University. And there's a lot of different views on this topic, including at my own university. What good is what we call assessment of student learning? Why should it be done? How should it be done? What's going on? And in addition to going through re-accreditation under SAACS, which is one of the toughest accrediting bodies out there currently, which is actually in some ways a good thing because it forces you to do a lot of things and otherwise you wouldn't do it as an institution. We're also going through some pretty hefty budget cuts in part because, you know, our state, like all the other states in the nation, is struggling with decreased revenues. And so there's some political issues about how the money is being spent that we're pumping into education. And the budget revenues, appropriations issue and the assessment issue, I'm going to argue here are actually intimately connected. And they really ought to be. That's another thing that I'm going to say. So here's the way I'm framing the question. Why? It's really a two-question. Why and how should colleges and universities carry out assessments to mark it? Obviously, I'm not going to go very deep into this because you can, you know, find entire libraries full of how to do assessments and your learning. I'm just going to talk about some of the very basics of it. But I'm going to be really concerned with the why question. But if you're going to answer the why, you also have to understand some of the hot. So when we're talking about assessment of student learning, what are we talking about? Assessment takes place a lot of different ways through a lot of different tools and at a lot of different levels and also for a lot of different motives. So let's only look at a few of those. Let's think about levels. You can assess your students in your class every time you do a test, every time you do a homework assignment. You are assessing their knowledge or their skills or perhaps even their developed dispositions on something. You might be doing it just to give them a grade. You might be doing that in order to help them improve their skills. That may be part of the process. You might be doing it in a diagnostic way. These are all different possibilities. You can have, you can weave these assessments together into a matrix for an entire course. And you can do this going up the levels. You can do this going from the class to the program, say the philosophy program to the department, all the way up to the college, in this case, College of Arts and Sciences. You can do it going all the way up to the university, academic affairs. You might assess certain skills, for instance, those that are taught in philosophy, one in ten here, critical thinking and required course. Critical thinking is one of those sets of skills and dispositions, as well as some content knowledge, that really has to be diffused throughout the entire curriculum. As a matter of fact, unless you have some modicum of those skills developed, you can't really do much of anything, can you? Just like your communication skills. It's great if you have wonderful ideas, but if you can't communicate them effectively, if you haven't developed those skills, you're out in the cold. There are some other sets of skills like that as well, we call fundamental skills. You can assess the entire university, or you can assess cohorts this year's freshmen, compared to this year's juniors. This year's freshmen compared to where they are two years from now. Those are all different levels of assessment. What are you actually assessing? Well, there are different ways of framing this. I like to think of it in terms of knowledge. What is it that you actually know? What can you reproduce? The fundamental building blocks that you have to have in order to make things work. For instance, if you want to talk about critical thinking, you have to have the concept of argument, and you have to actually understand it. That's not even at the level of the skill you have, that's just some knowledge. Then skills are what you can actually do with that knowledge. Can you analyze an argument? That's actually not just a skill, but a whole set of interwoven skills. And then dispositions. Critical thinking again includes a number of different dispositions, as well as skills. Dispositions being something that's actually woven into your character, or the fabric of your personality or your being. Education, if it's actually doing its job, should not just be imparting information, not just teaching people how to do things, but also helping them to modify who they are so that they realize their full potential. That's at the level of dispositions. You can ask where students currently are. You can take a snapshot of their skills, or their dispositions, or their knowledge at any given point. You can also, by assessing them at different points, try to figure out whether they're improving, whether they're actually declining. Sometimes your assessments might show you that despite your efforts, or perhaps because of some things that you do, students may actually get worse in some things. Assessment can also indicate to you the problem areas that you need to address. How do you do it? Well, there's a lot of different ways to assess, and again, I'm going to generalize very, very liberally here. I understand that there's more than one way to do it than just what I'm saying here. But it really comes down to do you assess performance, or do you assess approximate? Are you, with a proxy, are you substituting something else in place of, say, the skills that you want the students to demonstrate? So again, let's take critical thinking for an example. If you want to know whether students can actually diagnose fallacious arguments in real-life arguments, and you give them a multiple-choice test, which is rather artificial, you know, Bobby says this, is he committing the fallacy of that hominem, appeal to pity, et cetera, et cetera, et cetera, that's a proxy. If you require them to actually do tasks that require the use of those skills in situations that are fairly like that, or if you, in fact, ask them to tell you what somebody in such a situation would be able to do, then you're assessing performance in an authentic way. I mean, proxy, there is performance involved in that, right? But it's a performance that's being substituted for really authentically looking at those skills. And this is why multiple-choice tests, you know, I'm not knocking them totally. They are made sometimes for assessing content knowledge, but very often, they're not going to be good at getting at skills and dispositions. So ideally, you want to be assessing not just by proxies, but assessing performance. Now, quite often when people get involved with assessments, there's two extremes that they go to, and both of these are actually problematic. One extreme, I never was at myself, I was never tempted by, but I understand the temptations of is to try to quantify everything. And that ends up inevitably introducing a lot of proxies. A lot of the end-of-grade tests that we seek are those type of assessments. And you can teach to the test, and that's one of the dangers that happens, but when you're teaching to the test, you're actually teaching more just content and not actually skills. You're not helping the students to develop the skills, let alone the dispositions that they need. Now, why do people assess that way? Well, it's easy to do that, and it's easy to crunch those numbers. And you can turn it up and say, you know, this school got 68%, this school got 88%. This school went from a 68% to a 78%. It's very unclear what you're actually assessing there, whether it connects with what education is supposed to be imparting to your students. Now, if that is the model of assessment, then you get the other extreme, and I wasn't that other extreme myself. You get a lot of people in the arts, in the humanities, and even some in the natural sciences, because they're fine with assessing things, mathematically, statistically, in terms of physical things, but not when it comes to psychological things. Social scientists tend to love, you know, the numbers-based assessments. Sometimes you get people in the humanities who misunderstand the nature of assessment, and they think that it only includes proxy assessment. And so their view is that why should we do this sort of thing? It's a waste of time. We're imposing foolish things on our students. We're teaching them the wrong thing. Not only are we emphasizing too much of the content knowledge and not enough of the actual skills that integrate that content knowledge, but we are actually teaching them that education is just about learning units of information, which then they regurgitate. Now, if that was indeed the case, and that is indeed the case in some, you know, some regimes of assessment, that's a bad thing. You don't actually want that, although that's better than doing all quite frankly. That's better than just, well, we all just kind of feel like we're learning. I mean, I would rather actually get an end of grade multiple choice test to students than not do anything. But there are a lot better ways of doing this. Assessment includes performance. And you can do this by using rubrics. And there's a lot of great information out there. There's a lot of workshops out there on how to create not just quantitative, but qualitative rubrics for all of the type of skills that you would want to assess. All even dispositions. I mean, there are rubrics out there for development of ethical application of ethical knowledge, integration of it, the change of one's life, demonstrating the ability to process these things at higher levels. There are a lot of different ways in which we can steer students towards making these qualitative leaps step by step by step. And that doesn't necessarily have to include proxies. Again, I'm not knocking. Proxy testing. There is a legitimate place for it. But it can't be the whole of it. Now those people who, you know, in the humanities think that's the whole of it, or in the arts, or even some of the natural sciences because they think that that's going to be exactly like what they're dealing with physically. And then they say, I don't want that applied to, you know, psychological traits about people. If you think that that's all there is to it, it's understandable why you would be against assessment. I was in that camp. What changed my mind was being introduced to a very powerful tool for assessment of critical thinking, communication, and problem solving skills called the CLA. And I'm not going to talk about that here. That can be the subject of another chalk and talk at a different time. What it showed me was that it was possible to assess student performance in those skills and dispositions in very meaningful ways that allowed me as an educator to say, yeah, I'm actually assessing something. I'm actually checking what they do know and what they can do with that knowledge. And I'm doing it in a way that's not really artificial. That's not just make work that by which I can actually demonstrate growth in my students if I am doing my job well. Here's where we get to the other part. One assessment allows you to do is allows you to show to your stakeholders that you are actually doing your job. There's a lot of educators who I think have gotten into the mentality that just give me the students and I'll teach them and get the hell out of my way. Well, that's not going to cut it anymore. We are under what's been called a regime not only of assessment but of accountability, especially as the budgets have been getting cut, especially as programs you have to justify themselves. Philosophy is one of the key ones, actually, that has to come up with some sort of justification for why it exists. And I am not at all sympathetic and I'm in the story of philosophy so I love the history of philosophy for its own sake but I am not at all sympathetic to those who say, well, we should just get to teach whatever we want and whatever the students take from it, good for them, but our subjects are so valuable that we get to do whatever we like with the taxpayer dollars. I don't think that's responsible at all because we really are going to claim that studying philosophy benefits you and it can be shown. You have to show it. Same thing with English, same thing with natural sciences, same thing with the social sciences. One, he has to demonstrate. One, he has to talk to, I think, three kinds of stakeholders. Who are you? I'm being kind of cutesy here. Who are you and who is the university responsible to? And I think there's really three kinds of stakeholders. I'm going to call these the bankrollers, the contributors, and the integrators. And what I mean by this, the bankrollers is pretty, pretty intuitively obvious. Who's paying for it? Who does pay for education? Well, even if it's at a private university, in large part, it's the government. Federal. A lot of money is being spent to contribute to education. Also, you know, other stakeholders like ranchers, those who are putting money into these institutions of higher learning. When we talk about the government, where is the government getting its money? From the taxpayers. The taxpayers, especially if we're talking about education. The taxpayers have a right to be shown to have demonstrated to them in some tangible way that when they pump all this money into education, it's not just going down into a single hole. That something is actually taking place. That improvements are taking place. That they're thoughtful improvements. That you can not necessarily measure every single aspect, but that you can provide some sort of justification for what you're doing. The days of lack of accountability, I think, are over. Who are the contributors? Oh, there's another bank holder too, parents, and of course, who goes through the entire thing. Students. A lot of students are financing their own education. They actually, I believe, have a right to have demonstrated to them that we are actually doing something for them. We are actually making them not necessarily forcing them to become better educated, but we are creating all of the opportunities. We are observing ourselves. We are doing this in some sort of concerted, thoughtful way to make sure that whatever level they're at, all of my teaching has been low to your underfunded institutions. Whatever level you're at, you can be working to improve student learning, to improve the opportunities that they have to flesh out the curriculum, to get rid of the things that don't work. I think that students really have a right to that information from the universities, from the faculty. Parents do too. Who are the contributors? Who provides the environment in which learning takes place? Well, faculty are at the absolute center of that. Faculty are a stakeholder. It sounds like everything is being put on faculty, but faculty also have a right to demand, I think, in any good institution. The faculty in one area will often be able to demand, especially when it comes to core education, what are you people doing over there? Show us that what you're doing is actually working. Who else? Of course, administrators. And I think the board of trustees or whoever else is in charge, they are involved with that. I think students are also part of the contributors. And if you have something going on like service learning or internships, I would also say the larger outside community. But I'm going to put them over here and I'm your integrators who actually has to take in these students who come out of our institutions. It's the outside communities that they go back to. If all of your students are, you know, local students, then your community should be caring whether education is really taking place. Otherwise, you're just keeping kids off the streets. Unless you can actually demonstrate in some sort of tangible way that these kids are coming out better than they came when they entered four years or five years or six years later, the community should, you know, have a beef with you or not. If you're not actually demonstrating that you're producing better educated citizens, people that can hold good jobs, that can contribute to their community, you're failing as an institution. And I think there are a lot of institutions that are doing that because they're not thinking about this. They're not grasping that. It's not that the institution is actually bad. They're not getting this. Who else? Employers. We should be asking employers and the better institutions are doing this. What do you want from our graduates? Are we actually giving you in our graduates what you in fact need for the workplace, for today's workplace? For example, thinking skills, communication skills, some sort of ethical set of values and a work ethic. Are we inculcating that? Work ethic, by the way. That's not knowledge or skills, that's disposition. Are we inculcating that? If we're not, and if we can't show that we're doing that, then they want to say why should we take your graduates? Why should we bankroll you? Why should employers be part of the bankrollers? Why should they be interested in that? Why should employers be here as part of contributors taking in students as interns if the interns are not going to do good work? Or they're not prepared? We have to demonstrate that we're doing a lot better than we have overall. Now I'm not saying that every institution is on the same level in this respect or that every institution has to do as much work. I think that some institutions have been very, very good and can actually be models in showing how assessment can be thoughtfully woven into the process of continual improvement and generating more and more student learning, especially in these difficult circumstances. There is one other thing I'd like to say about assessment. If you're going to try to assess the institution as a whole and we are doing that here with critical thinking at it at Fayette Listing University, critical thinking, problem solving and written communication if you're going to try to do something like that which I think that you ought to do not only for prudential reasons but also if you have an ethical duty to do it. You have to be able to address these stakeholders where they are. It's one thing to be able to present complicated assessment data to granting agencies like the Carnegie Foundation. They understand that. You also have to be able to provide information to taxpayers and to parents and to businesses and to people in the outside community in terms that they can understand. This generates some real challenges because the finer the instruments that you're using for assessment the more complicated it gets, right? And the more simple the instruments the more simple the output that you're giving them in nice graphs or whatever however you like to present it you're losing some of the solitude and some of the process by doing so, aren't you? So it takes some real thinking about how to do this well. It may be that you want to have several different ways to present the same assessment data to present the same processes. And again I'm going to put a plug here for the CLA of the Collegiate Learning Assessment Performance Test Methodology only in part because I think that it actually provides you with something that on an institutional scale if you're doing longitudinal studies that's very complicated statistical sampling over crunching and that's not going to be helpful to show say to parents or to taxpayers but you can't actually generate by use of other ways in which you look at the CLA data, that's what we have done here at FSU by the way with the Rising Junior and Entering Threshold Exam. You can show using that rubric how have our students developed where are they right now how have they developed Entering Threshold to Rising Juniors how are they going to go from Rising Juniors to Exiting Seniors You can also use it and I have used it in my classrooms to give students feedback on their critical thinking abilities and what they need to do to improve them. That's an example of a good assessment there's a lot of good assessments out there and I believe that there are a lot of good assessments waiting to be born out of the minds not quite fully formed like Athena popping out of the mind or Zeus being born out of the minds of those who are applying themselves to these problems but I think so to recap I think that we have an ethical duty to be explaining ourselves to these stakeholders which include us the faculty but also include all of these other people so that's why I think universities and colleges should be carrying out assessment of student learning I haven't gone very into how we can't do that in this short time but I think that at least hopefully I've put the case before you about why it is that we should be assessing student learning why we should be publicizing it why we should be thinking about how we're going to present this to these stakeholders and I think we have to be very thoughtful also about what means of assessment we use I'm not going to leave you with an ending question of this one because I think I've already posed a number of different questions and people may be somewhat confused watching as if they're not specialists in education but if you have any questions about this or you want to say I've got it right or say that I'm totally wrong on this or that assessment isn't something we don't have to be doing or we don't have to justify ourselves with them email me or put a comment on this