 So in order to ensure that we get through the session today, I am going to go ahead and start. Thank you for coming to European Open and Distance Learning Week, which is hosted by Eden. And as I mentioned, there are wonderful speakers with us here today. We've got Orna Farrell, who is from the National Institute for Digital Learning. She's coordinator there in the Digital Learning Design Unit from Dublin City University, Ireland, as well as a Eden NAP steering committee member. We've got Richard Powers. See, I told you, I've been zooming too long today. Richard Powers, who is a project coordinator at the Professional School of Education in Stuttgart in Germany. And we've got Aaron S. Blackwalder, who is a teacher at the Woodland School District in Vancouver in the United States, which, you know, he's very early this morning to be with us. As I mentioned, please enter your questions into the Q&A and say hello in the chat box. And we will start with our first speaker, who is Richard. Richard, as I said, is a project coordinator at the Professional School of Education in Stuttgart. He's a professor emeritus of English language and literature from the University of Maryland's Global Campus. An expert online teacher, trainer and instructional designer, he has taught online since 1998, using WebTICO, Blackboard, Moodle, Brightspace and Alias. Currently, he is a project coordinator at the Professional School of Education in Stuttgart for blended learning courses for diversity, for teaching and training. And since the coronavirus outbreak, he continues to be a key member of two task forces, preparing students and teachers to teach remotely in this very confusing and unprecedented time. He currently supports 3,000 teachers and 25,000 students online. And at Stuttgart University, he is developing project courses and trainers for 3,000 teachers and training. So Richard, please go ahead and share your screen, and we are looking forward to your presentation on the importance of online authentic assessment. Thank you so much, Lisa, for that wonderful introduction. As Lisa mentioned, today's talk is we'll be exploring new avenues to online assessment. And I'm part of the Professional School of Education Stuttgart Ludigsburg in Germany. We oversee teachers and training. So you can imagine how important assessment has been since the pandemic. I wanted to start off with kind of a case study. In the States, people are very much familiar with SATs and ACTs, and over here, standardized testings in the form of the PISA study, the tests in Bologna that people take with these standard things. So just before the pandemic come to background, there was already this terrific momentum of movement away from traditional types of standard testings and testings that set the groundwork for what's happening now in rapid fire. Because they weren't inclusive, they're not really diverse, and people were thinking a lot with these standards types of testings. You weren't really qualified. It wasn't a significant kind of effect of how qualified you were by taking these kinds of tests. It was more like testing how good you were at taking tests. Then, of course, last year too, we had this notorious kind of scandal on how these standardized tests were, you know, celebrities kind of made the headlines for it, which is kind of the way things kind of work with that too. But we had kind of some famous people whose children they paid for to take the tests, and that created, you know, all kinds of public outrage about how effective these tests were and what were going on. So there were a couple of things moving on, and then the pandemic hit. So suddenly, you couldn't even do these tests digitally or any otherwise. So something needed to be done about how these things were going to work. All kinds of tests were canceled. The schools still needed a way of seeing what's going on. But the whole background to this on my point before we get started, not only were the standardized tests were already kind of broken, there were all the kinds of assessments moving forward, trying to do something as we moved online. There had to be some kind of way of doing that and moving it forward. Now, how relevant was traditional testing anyway? One of the things we'll be talking about, at least with mine as we move on to the other speakers, is authentic assessment. One of the things that authentic assessment likes to look at is, what are they learning in schools and colleges, institutions that prepare them for the workforce so they can hit the ground running when they kind of get there? This cartoon was kind of taken out of New Jersey with something that was going with teacher's training. But you see the doctor saying, well, we can't remove it until we decide what to replace it with. And that was kind of what we are faced with at least the European universities, the one I'm involved with at Stuttgart University. Everybody knows they want to do something different from the standardized testing and also above the standardized testing. The typical university test is at the end of the term, you might write a paper and then take a test. And that's what you spend 16 weeks kind of learning. That's how you show that you've learned something going forward. So everybody knows that that final assessment now for hygiene, safety reasons can't be done anymore. So what do we replace it with? So one thing that authentic assessments have been there for a while, but now they're starting to see a lot more publicity and light because we need something that's going to replace it. Now this quick slide just by a survey of 1,600 institutions in the United States that Bayview Analytics had done. But just look at the 63% and the 46. 63% of faculty said they changed their exams and 46 said they dropped some of those exams. That can't be good. Come into authentic assessments and how we kind of move that online. Authentic assessments had always been in the classroom. But now what does an authentic assessment online look like and how do you kind of replace that moving forward? Grant Wiggins who passed away I think in 2015 or so. It's kind of early publisher of using the term of authentic assessments. His book, Understanding Design, came out with Jay Matig, very, very important kind of moving things forward, saying grades aren't everything and testing certainly aren't either. So some of the quotations you can read faster than I can speak. Main thing that comes out of Grant, of course, is the personally meaningful question so that things like reflections were also kind of important if you're trying to evaluate learning and how that works. Training had also been a force. This big debate about education versus training, for Patrick's theories about the four levels of training are very important for educators today. In particular is the different ideas of these online assessments come. How do you measure learning? And then shouldn't we be measuring learning that takes place six months after the fact, right? If we're trying to see, well, is behavior changed? Is the learning patterns changed? How do you know really something, somebody's learned something moving in? Anderson, a little from Elon University, have a nice definition that I'm using from my little talk here too. Authentic assignments, ask students to perform the intellectual work. Notice this at an apprentice level. So it really kind of uses that heavy duty workforce jargon going in there too that, look, you're training people, they're learning, they should learn the academics of writing an academic paper and being able to present at conferences for sure. But really in reality, what percentage of the students that you're getting degrees end up in academia for that? So shouldn't their skills also be for the workforces that they're going to meet? That people in your field put them in real or realistic situations so the students can use those skills in the course and then put them all together. And that's who I think this has a tremendous benefit for today's online assessment in our courses. Now here, I hope everybody's sitting down, just got kind of blown away by the 27 characteristics of a thought in assessment. There's not a quiz, right? That would be horrible if I gave you a standardized test after a lecture such as this one, certainly not, but I did want to call some important points out of these. Everybody can see pretty clearly the ABCD part, nice. Let me took these and put them into what are some characteristics of authentic and I would like to put online authentic in there because that's really kind of what we're talking about. It's not what we're saying, authentic assessments are new, but now what's an online authentic assessment and what does it look like if that's the environment that we're involved in? So structure and logistics is certainly a part of that. The intellectual design features are important, grading and scoring, right? And then fairness. Now look at the spotlights that I have without the seven, right? Make feedback to students so central that school structures and policies are modified. Now my particular work at the university, remember, my traditional assessment, take a 16-week course, take a test, write a paper, that was kind of it. So just this having feedback of stages as the learning is moving forward is a wonderful part and the LMSs and the online can do that. So we've seen already a lot of success with traditional teachers, professors, very busy people. The only time they have to do is go in, profess and then give a test at the end. Well now they're starting to do a little bit more of the LMS with Ilias and doing intermediate stages with the feedback. Bravo 4 and 6 are almost combined. If you look at B4, involve the students' own research, and I put this together with six representative challenges of a field or subject. So if they're taking a course in health studies, or they're studying to be a doctor or a nurse, that their research is not perhaps 100% in the medical books, but they're out there researching a hospital and then figuring out what are these challenges that are going on there and then writing some kind of report dealing with a real world situation that came out of that and doing a report that way. With grading and scoring, Charlie 1 and 4, you need criteria that assess the essentials. So if they're online, so how do you know that they're really learning? That's a hard part. What are they being judged and evaluated in? And also make self-assessment, the reflections part of it. So that's where you get all that kind of, you know, sometimes people say the buzzwords of why am I doing journals? Why am I doing learning and reflections? What, how is that enhancing the learning? And studies have shown that as students think about, and we as learners, as we think about what we're learning, why we're learning, how we're learning, the chances are that the measuring or the assessments will stay with us a little bit longer. What's the process of how that's doing being with critical thinking? And these last two under fairness, the bullying aspects and also the identity questions where you're getting a grade for something and now that has a value input where you take it personally. So it minimizes needless, unfair, demoralizing comparisons of one students to another, getting away from all this competition and moving it maybe into a world of compassion and love and kindness. If the pandemic is sending all those vibes, the politics right now and all those things too, getting out of a competitive world and moving more into compassion and kindness for those kinds of areas. And this last one, the appropriate room for student styles so that if you're not a good test taker, suddenly you can move forward and then move that in. Another thing that came out of authentic assessments certainly is backward design. And the first two are always there, right? You start off with your objective. What do you want students to learn the outcome? And then there's a traditional assessment of some kind. The new part here, maybe at the third for online learning that can certainly help is now in between this, even if you still have to, for whatever reason, you still have to put 400 people in an auditorium and have them all spaced out and good hygienic social distancing things going on. You can have intermediate stages of authentic learning activities with feedback in the middle and all those can occur online, right? Whether you're using an LMS or if you're just using apps or if you're using teleconferencing such as we are here today with Zoom. Quick example, samples, none of these are new, right? Open book projects and how you design them, collaborative projects, but as you go through them, then think about how you might be able to do these online. And I put art projects up there because okay, teach literature classes, but it doesn't matter. You can teach any of the humanities, the social sciences, the sciences, but a project could be whatever the concept that the student is learning is maybe choreograph a dance or maybe write a song or a symphony or maybe involve yourself with some kind of sculpture or poster that has to do with artistically applying the concepts that really require deep-seated learning and understanding of what the concept is moving forward. Portfolios and reflections are certainly another great way. We have an expert on our panel today who knows a lot of his written, you know, so much about portfolios. We'll probably get to hear a little bit about that today from Dr. Werner. And also recordings, right? Online, YouTube, all kinds of different areas and events, can make for this kind of a different experience instead of that traditional paper or exam that's going on. They're concentrated on this real-world submission type of way. The Cumbria Group back in 2018 came up with different case studies. They went out and surveyed all kinds of different ways like current trends and assessment in Europe. And I deliberately picked this because I wanted something from before the pandemic to show how this was already kind of out there. So up at the top, you see how this basic essay course paper could then turn that into an abstract and a paper submission as a call for paper in the field, right? Okay, something like this, right? Eden's got a great conference coming up, sends out these calls for proposals, who does presentations. Well, encourage your students to apply for that. Chances are they won't be selected, but it's a process of doing a professional application and it prepares themselves for doing that perhaps when they get to real-world situations and doing that thing. So the real-world call for application is no matter what your field, looking for that and having them treat themselves and think of themselves as scholars and professionals and do that online. Remember the handwritten final exam? Those can then become a series of short essays over the term and the LMS. Instead of one 90-minute or three-hour exam at the end, stop those up so that students are getting feedback on individual questions and they're learning all the way instead of cramming and trying to do everything at once at the end. Computers and programming. Instead of having that programming final exam, find out at your institution what programming project they need done and then have the students do it rather than out-searching or consulting. So it's a real-world project that they're all working on. Change that academic paper to maybe having them in healthcare or medicine. Pick an image or a picture or if they're able to work as interns already in that field and then they reflect on what they're seeing in that image and then how it relates to their field coming in there. And also the final paper to collaborative would change to a group type of peer review. Instead of me submitting one 15-page paper at the end, my draft is ready by week eight. Three or four students look at the draft, make comments, and then I turn in the final later. It's an improvement online over that traditional way coming in. Other interesting things that the CUMBER group came up with was maybe taking that final exam, getting rid of it, and then turn it over to teams of two so that people are creating Wikipedia entries. So they go online, find maybe a concept in your field that's not done well, and then they become responsible for 12 weeks of modifying it, adding images, and really supporting this scholastic activity of it so it has a real-world approach. Language courses, that's what I was mentioning about YouTube. Certainly helpful, you're learning German, you're learning French, you're learning Russian. You have to speak in that language, but now also you have to make sure the captions are correct and done well. So you're working on those kinds of skills too. Exams using web tool projects. There's all kinds of great digital skills out there. This is important for me because I work with teachers and training. So having them do wakelets, padlets, book creators, canva, pow-tunes, doing all of those instead of papers, it's wonderful and they seem to really enjoy it because they'll be able to do that in the classroom. Learning journals we've kind of talked about, and now as I kind of end my talk, I'm going to move on to the other panelists. Part of this is the online assessment world. And so I want to talk a little bit about the online proctoring tools that are out there. Some of you might have heard of Respondus or Lockdown. You have an LMS like Brightspace or Ilias, and it has browser lockdown. So what that really kind of means is the student can't surf. Now that doesn't mean they can't cheat on their phone or do other kinds of things, but the commercial market out there has started coming up with different ways to do this. Now Pearson, I think we all know as a publisher, each one of our publishers are going to have these things. And out there, Pearson views, it's called, they'll do photo and workplace identification. So the students at home, and there's a live greeter who comes on there, asks for the student's ID, you know, some kind of, or Ausweiss or kind of identification paper. And then that's verified. If you're in the workplace sometimes it requires them to take pictures around them to see what's going on. And they send that back, take the exam. It's usually a timed exam where they have a limited amount of time to keep that focused and move forward. A European model proctor exam is this dual view proctoring. So there we see the student, she's just finished her tests. She's put it down. And if you notice behind her is her camera. So one of the requirements is she has to set up her phone behind her. So it has this dual view proctoring. So somebody's watching her while she's taking the test and somebody's watching her or is behind. Now everybody can already imagine, how do you do that large scale? You've got 400 students and also the cost for what that would kind of be. Now there are, I'll put this link after my presentation is over. There is open educational resources or there are resources for software proctor software. And I've got an article put in the chat for the seven free software devices that are out there. If this is interesting to you, you know, I felt like today was about online assessment and a lot of universities are looking to see also home schools, how this is going to kind of work. So is this, you know, optioning, you know, fuel coming in? I think what's happening, if you're like me, it's just moving from that model of the assessment being a final paper and a exam at the end. And now what you have is a blended learning concept with the professors really going into the platform, the LMS and then using it every week to put the assignments in there so that the learning is according all through some of the platforms like Brightspace for competency based without grades. You take the learning activity and that you take the pass the quiz or they pass the discussion or the assignment. And now that means that they've passed the learning objective that it was tied to. And now three or four learning objectives are tied to a competency. So the LMS and some LMS is their abilities to trace just by competency based on the learning activities and how they do. So some of you might be able to tap into your LMS. It's coming in. My last little point was about managing resistance because this is huge, especially like mine. Nobody wants to get into any of this stuff online because of the cheating, the things, but necessity is going to kind of drive that train. And I think what we know, there are different types of tests like a medical board exam or a lawyer better exam is a little bit different from a freshman level course in essay writing. So depending on what is happening in those two, if you're at the high level institution part, you're trying to get buy in for all this stuff with faculty, know your audience, do your homework, go to your meetings backed up with studies, videos and presentations that show that authentic assessments online work because your whole point is to convince leadership without your senior leadership. I mean, presidents, rectors buy in. You won't go anywhere with this. So then how could you do that? Publicize and market department websites. Start using those buzzword words like authentic assessments. How are you doing online? So the department heads are asking their instructors and little focus groups. How are you testing online? What ideas do you have? What are we going to do? How can we kind of share those things do it? But then it's an ongoing process that we need to train the educators in assessment and online, the teaching assistants, the tutors for your courses and the instructional designers. And then in your syllabi and your term documents, maybe that will increase enrollments if they say things like no final exam but a project is required. No course academic paper, but here are the activities that you will do and they have some sound like they're fun and engaging and moving forward with it. Okay, that was it. Sorry if I started talking rather quickly towards the end. I wanted to make sure that there's time for other people. You can follow me at Twitter. I'm all about kindness and love and compassion right now. Maybe three years ago it was competition, competition, competition. That's my little Twitter has a heart in there and I want to do a shameless plug. November 10th, we have this nice webinar and global citizenship courses for the 21st century that we've arranged with UNESCO and the Mahatma Gandhi Institute for Education and Peace. They're going to show some of their global citizenship courses with artificial intelligence and demo them. Kind of amazing stuff. It just puts my little Moodle and Ilya stuff in another kind of century. So I want to turn this over to Lisa and stop sharing and thank you for everybody's for your time. Thank you, Richard. There's quite a few comments and a couple of questions. A lot of comments about the proctoring and one of the questions that we have or first a comment, then a question. Proctoring is based on the assumption that the students will cheat and there is a considerable body of opinion that this is unfairly treating them as the enemy and reinforcing the us versus them divide. And I have to say when I was watching and I had a similar reaction, could we as instructors instead make more efforts to design authentic assessment that doesn't require proctoring? What are your thoughts on that? I love that because that's my answer is that not spend the money and the time and get all into that proctor and get the whole gamut. But I understand big decision makers, they have to go through that research. They have to find out what's scalable, what's affordable and what's allowable. Sometimes institutions, whether you're in Europe or you're in America or Asia, you've got a higher body that you have to answer to a higher body of rules and regulations. And they will come down sometimes with standards that prevent you from trying some of these neat authentic kinds of activities. So even with my courses, again, I usually just consider it's going to be open book and it's allowed, but I tried to change the questions for making basic blooms taxonomy to apply and evaluate, right? Go and research and find two organizations of the business English course of how they market their green, going green programs and then report back. So it's a question that they couldn't really, surely could cheat if somebody else is doing it for them, but chances are if they've been and I trust students probably, you know, I've got a good optimistic feeling the students throughout the course, they haven't done a good job at the course. When they get to the exam, the exam is part of the learning. It's not a final thing. It's like that final, bring everything together capstone that when they leave and finish the project, they kind of thinking, wow, that was a satisfying experience or really kind of learn there. So to everybody who put that in the chat I'm with you and it's all about the design and the questions of what is on that exam. Okay. Thank you, Richard. We're going to move to our next speaker. There will be time at the end of the presentations for us to address the many questions that are popping up within the Q and A box. Please enter your questions there. Our next speaker today is Orna Farrell who is from Dublin City University in Ireland. She holds a PhD in education from Trinity College and she is working within DCU's National Institute for Digital Learning. Her research interests include online pedagogy, online student success and engagement, and e-portfolio based learning. And she's written some really, really great articles on this and I just encourage you to Google them and to look them up. It's really good stuff. She's principal investigator of the National Forum Funded Project, OpenTeach Professional Development for Open Online Educators. Orna is a founding member of e-portfolio Ireland, a professional learning network for e-portfolio practitioners and researchers. Orna, the floor is now yours. Thanks, Lisa, for that lovely introduction. Hello, everyone. Nice to meet you. I'm just going to start sharing my screen. I was the one putting some of the inflammatory statements in the chat box there about proctoring, I have to say. I'll admit it. I saw a very good presentation from Sheila McNeil. I think it was about six months ago and she really started me thinking about what kind of culture are you creating if you surveil your students. And it really stuck with me. And I think she's right. I mean, obviously there's always challenges around academic integrity, but good assessment design should hopefully deal with most of those things as well as text matching tools. Also, you know, literacy for students around academic integrity. So I really dislike the whole proctoring thing, but I understand the point that sometimes professional accreditation or professional bodies require it. So that's my two cents. There I am. And there's my email address and Twitter handle if you'd like to get in touch, please go ahead. Lisa mentioned some of my publications. So my presentation today is kind of drawing on two in particular, but there's others as well. My newest one there from portfolio to e-portfolio, which is a history of portfolio and it's open access on Jime. And then the other one as well is about a case study of online students using e-portfolios. So anything online is very relevant at the moment. So I thought I'd kind of go through a few questions about e-portfolio, the what, the why, and the how in the short time I have. The what first of all, there's two images from my history article there of e-portfolio through the ages. So I had some fun digging around looking for historic documents and images of portfolio. So there's two I found. The one on the right is a French one from a fashion magazine and the one on the left is a kind of a drawing representing, I think it's 18th century England. So the portfolio as a fashion item. So what is one? Also the thing about portfolio and e-portfolio is there's so many definitions. It's also a very flexible assessment method or tool and the flexibility is brilliant. But it also I find causes quite a lot of confusion when I'm talking to students or introducing it to students and the same introducing it to faculty as well. So I often start off with the what. And you can see on the right there a diagram I adapted from Tracy Penny Light's work. So what is an e-portfolio? It's a pedagogy. It's a process. It's a product. Is it an assessment? Is it a framework for learning? Is it a technology? And the answer is all of the above. Here are some of the most commonly cited definitions in the literature. I dislike nearly all of them. I got to say that straight out particularly because they're very technocentric. But they also it's interesting the time they come from. They come from that decade 2000 to 2010. And that's when I suppose a tech online learning was really coming into its own. And these definitions I think they're from that decade. They're no longer relevant. I much prefer this definition, which is from Quirley and Zubzerita, which talks about a learning portfolio. And the reason I like it is because it talks about the learning and the student experience of learning with the portfolio rather than the tool. And I think that that shift in thinking there, you know, is quite representative of the shift in thinking generally in higher ed, focusing less on the tool and more on the student experience. And also one thing that I feel is very important about using portfolios with students is actually the main learning happens in the process rather than the final product. So therefore we should reward the process more than the product. I see some chat activity there. So I'll come back and I'll take a few of those towards the end. Okay, so why use one? The research body on ePortfolio has actually gotten quite, it has improved. If you looked at this about 10 years ago, you would have found weak evidence to use them as an assessment. And a lot of articles about technology, but it's really come on. And one thing that spurred it on was a very good literature review by Bryant and Chitham. And they identified that essentially there was huge gaps in the literature. And in response to that, lots of people interested in ePortfolio have started to fill some of those gaps. So the body of evidence is really improving. And as you can see there, there's some good strong evidence as to why you should use it with your students. So first of all, helps them integrate their learning between modules. So connection sometimes used as a capstone type assignment. In an online course, which is I highly relevant at the moment, can foster a sense of community and belonging if you integrate it with peer-based learning and peer-based assessment as well. And finally, it enables students to self-regulate and develop skills and gives them opportunity to reflect on their learning in action. So there's the why. Now the how. So that's meant to be how to mix a lovely drink, but really how to mix a lovely ePortfolio is what we're talking about. One other challenge, after you've overcome the first challenge of figuring out what a portfolio is or what an ePortfolio is, the second challenge is what are you going to do with it? Because as it is so flexible, what purpose are you using it for? So I've kind of identified about five purposes in the literature. And I just have a question there. Which purpose is most relevant to your practice or to your students? Because really you could use it in all of these contexts. You could use it as an assessment for a module or a program. You could use it to document student informal learning. So for example, students are class representatives or they're participating mentorship programs or back when people could travel, they might have a year away or an Erasmus year. You could use a portfolio to document that experience. It could be used for developmental purposes, particularly if there's a particular skill like critical thinking you're seeking to develop or a transversal skill, or for example, linking it to a framework like DigCom. It could be used for career development. So while you're a student studying to develop your career readiness skills and post graduation for continuing professional development, and that's starting to happen in quite a number of professions. Nursing, a lot of medical professions. In Ireland, the pharmaceutical society. And I suppose anyone involved in education often has a portfolio requirement. And finally placement. So a lot of courses with placement attached. Again, nursing, teaching, or even work experience would have placement. And the portfolio is used in DCU as part of the intra-program, which is our placement program, and students are using that to document their learning while on placement. It also serves a second very interesting purpose, which is to actually keep track of those students while they're on placement or say while they're on Erasmus because sometimes there's actually a pastoral care element to using it to checking in. Are you okay? How is it going for you? Do you have any challenges? Because sometimes keeping track of students when they're out of the university is hard. So some quick tips from the literature and from practice on how to design an e-portfolio assessment. So I tried to keep it fairly simple. It's not that simple though. So purpose, identify it. Tell the students why they should do this. This is the second, a third thing that comes out in the literature. Students don't often really understand why they have to do the portfolio. So link it to the learning outcomes. Communicate it very clearly to the students. Train them on the platform of choice. Show them examples. If you've got other colleagues involved in the use of the portfolio or the assessment process afterwards, make sure they're trained to help the students by giving them prompts for the reflection. Show them how to write reflectively because they generally don't know. Lots of example portfolios really helps. And I find students often then get it when you show them an example. And then typically a rubric is a very useful way to market. On the e-portfolio Ireland website we have a very nice rubric which anyone can have a look. I'll find the link and stick it in the chat in a bit later. Another nice resource you might like to draw on is this e-portfolio assessment e-book that was created by the e-portfolio Ireland community. So in 2018, gosh, this is a long time ago now, we had an unconference. And in one of the unconference sessions we collaboratively started working on this book. So it's case studies of e-portfolio assessment in action in lots of different disciplines and different types of institution. So I'll share the link to that shortly after but it is a nice way to start if you're trying to imagine how will I use this with my students? One of these case studies and they talk about very much in detail how to design, assess the student feedback lessons learned. So very nice resource. There's some student examples here as well. Once I've stopped sharing my slides I'll actually show you one or two if you'd like to see them. So you can tell me in the chat, would you like to see some examples? And here is the e-portfolio Ireland website. So we are a community of practice based in Ireland but happy to meet friends from around the world. We host events, professional development. At the moment we're working on a special issue of the Irish Journal of Learning Technology all about e-portfolio. And they're mainly Irish authors in there, mostly female too, except we have one Canadian representation from Athabasca, which is great. Bit of diversity. And there are some references. So I might just stop sharing and get load up some examples and have a quick nose at the chat. Am I doing okay for a time, Lisa? Yeah, please go ahead and show a couple of examples. Okay, one sec. And you can ask me questions while I load them up if you like. If there is any. Yeah, there's a question from Helen Williams who's asking an e-portfolio's fantastic tool, serves so many learning, reflective assessment, showcasing purposes, thanks for all the examples. And she understands that DCU allows its students to retain lifelong access to their e-portfolios. Do you find students continue to maintain their portfolio after graduation or even after the particular e-portfolio assignment has finished? Sometimes she feels that students miss the point of it being a lifelong tool. Okay, so a couple of questions there. So we started down this journey about three years ago now. So we only just have some graduates now. So the answer to the first question is I don't know yet. The second question, yes, students do continue to use their portfolio. We found that once they've got used to the space and enjoy the space, they do often continue of their own volition. So I seen that in the journal article I showed earlier with case study, several of those students kept going. Yeah, so yeah, that's an interesting one. And I would be interested in the post-graduation one too. But again, that might be a question for next year or the year after. So a couple of examples. So our platform that we use is Mahara. So we were a Moodle-based university and we have Mahara and it becomes Mahoodle. But we call it Loop Reflect. So just show you a couple of examples. Now I have permission to share these examples. So this one is a marketing student. So I tried to give you ones from different fields. And this was part of a marketing assignment. And the portfolio was really used to bring the components together. So you can see the student has brought together several different types of resource and mixed it with reflection. And one thing that's really important is that they haven't put pictures up for no reason because that happens. They have put them up with purpose. And another thing is we really encourage students to reference, you know, just because it's a different type of assignment doesn't mean you don't use your referencing skills. That might sound a bit pedantic, but just because it's true. Maybe you could show us one more example. Yeah, I've just loaded up. And maybe post a couple of the web links. I'll post the slides. Yeah, I will. Thanks. Well, here is one with a giant picture in it. And again, different format again. So you can see this student is a music student. And they've been doing kind of virtually weekly reflections. And you can see what I like about this is with the music flavor, this is one of the theory they're working on, the method they're working on, and hand signals, a video of the student putting some of that methodology into place. So very nice kind of applied learning. And you can see, again, examples from their teaching practice here and videos, I think, of their performances as well. So could this be achieved as a blog? Absolutely, Alistair certainly could. I see blogs and portfolios to be quite similar. But the problem with blogs is they're very open. And that's not necessarily a problem, but it's for assessment purposes. It needs to be locked down for the quality assurance. External examiners need to see it. Students need to be prevented from altering it after the fact. So there's certain reasons to use a portfolio such as this that I suppose give you that quality assurance element. And there are my slides gone into the chat. Great, thank you, Orna. Really, super presentation. You know from our past professional interactions that we also use portfolios, and I'm a real proponent for this form of authentic assessment. So thank you very much. I'd like to move on to our final speaker today, who is Erin S. Blackfelder, who is a teacher at the Woodland School District in Vancouver in the United States. He teaches high school English and coaches boys and girls golf in southwest Washington. He's the co-founder of Teachers Going Gradeless, host of the podcast Beyond the Curriculum and an educational contributor to Spectrum Life Magazine. He is a Washington State English Teachers Fellow, was nominated for Washington State Teacher of the Year in 2019, is a five-time golf coach of the year, is the husband and father of two boys on the autism spectrum who inspire him to become the teacher who meets the needs of all students. Welcome today, Erin. I'm looking forward to your presentation and with no further ado, you may take the floor. Well, thank you very much. I appreciate the introduction. So let me share my screen now. Let's see. Okay, so today I'm talking about assessment and reporting and how do we move towards a student-centered approaches in this assessment and reporting process. So the problem with traditional grades and grading is they don't communicate learning. As a matter of fact, they are meant to rank and sort students. Ever since they're beginning back in Harvard, back in the 1800s, it was an attempt to rank and sort students, who were the best, who were the finest. And then as we went through the Industrial Revolution, again, it was about ranking and sorting, who would be working in the factories and who would be managing the factories. But today, we don't need to rank and sort students. We need to develop students in engaging tasks and figure out who they are so that they can become contributors in the world that they live in. Grades are also extrinsic rewards and they demotivate risk-taking and intellectual tasks. We see that Althea Cohen has written a lot about this. Daniel Pink in his book, Drive, he talks about the problem with external rewards and how if we just, essentially, he does it in the business world, but if we just pay people what they're worth and we take money off the table, if we take grades off the table, people will generally do what they need to do and they will feel challenged to move beyond. They decrease learning and engagement. Ruth Butler, which I'll get into in a little bit, has written extensively on the damaging effects of grade and grading. The problems with standards-based grading, so traditional gradings have their problems, but standards-based grading also has their problems. Standards-based grading limits the scope of education. There's a story I love about a young girl who is, she is developmentally delayed and she has autism and she's sitting in the psychologist's office and the psychologist shows her a picture of a mother waving to a child on the bus and the psychologist asks the child, so why don't you tell me the narrative of what's going on in the story in this picture? And the child points to the corner and says, I want to know about the cat. Who's taking care of the cat? Where's the cat's home? Who's feeding the cat? The psychologist keeps redirecting the child back to the major narrative of the mother with the child. No, no, no, no. What's going on here? And the child keeps drawing back that. Well, the problem with standards-based grading is we don't know if this young lady would have developed to be a animal rights activist or a veterinarian if she's allowed to explore what's going on with the kitty cat in the corner. Standards-based grading really limits the scope of what kids are able to do. Standards-based grading also facilitates data mining. I mean, really, all we're doing is just producing data for neoliberal corporations to basically sell products to our children, sell products to our schools. What we need to do is develop our kids and pull something out of them, rather than find new software or new books to help us reach testing potential. And standards-based grading limits students' desire to achieve. If you set an exemplary at four, well, kids are just going to work towards at four. We need to set a floor and leave the ceiling open for our kids. And those are some of the problems with standards-based gradings. So how do we assess learning if we remove traditional grading and standards-based grading? Well, let's get back to that idea about what assessment is and what is the purpose of it. Assessment is about learning, and it's about helping our students to unlock their own potential. And back in 1987, Ruth Butler did a study on the impact of EGA, or the impact that grading and assessment has on the ego of the child. And on the first session, you can see where the children scored on these tests. And then later on, the comments-only group, she had a group that was comments-only, grades-only, praise-only, and no feedback at all. And you can see that the comments-only group, the amount of growth within those children, and you could see the negative impact of grades, praise, and no feedback at all. She continued, and you may ask, well, what about if we provide grades and comments? Again, she did the study in 1988. Similar results. Comments-only provided the growth. Grades and comments and grades, as you can see, comments and grades actually had worse effects on the children's learning. So how will we know how our learners are doing if we get rid of grades, grading, and all these types of things? Well, good teaching is like good coaching, and as Lisa said, I'm a five-time coach of the year as a golf coach, and I've learned a lot about teaching from coaching. And as a golf coach, I have to start with why. Why are my athletes on the field to begin with? And so one of my whys for both my teaching and my coaching is, as a golf coach, golf makes us better people. As a teacher, learning makes us better people. And really what we want to do is we want learning to make our kids better people when they're finished with our courses or when they finish with school or later on in life. We want them to know that learning is about improving who they are as people. A good coach understands their content and knows how to teach it, but that's just the basic level. They understand the uniqueness of their athletes, and they can develop the athlete's skill at the level that they are at. They can identify exactly where they're at and help them on an individual level. They prepare athletes to showcase their work. As a golf coach, we practice constantly and we're getting ready for the matches. We're getting ready for the games. So I need to get my kids ready for the game. They make learning challenging and meaningful and most of all, fun. There's a great quote. I just absolutely love this. If the objective of a class is for kids to fall in love with the class, then that could be the highest objective that we can reach. So have fun. There's nothing wrong with fun. So much in play is just, there's so much learning in play that I think we need to explore those ideas. Good coaches know how to develop personal character and care for the individual. There's a lot of metaphors about coach, about golf that I relate about life to my athletes. And they know the impact that they make on the lives of each of their athletes. Good teachers do the same thing. They establish a purpose. They understand their content and how to teach it. They understand the uniqueness of their students and develop their students to go beyond. They prepare students to showcase their work, not just have a test at the end of the unit, but showcase their work. They make learning challenging and meaningful and fun. They develop personal character and care for the individual. And they know the impact that they make in their students' lives. That's what good teaching does. So if we're doing all these things, if we're not reporting grades, if we're, if, how will stakeholders know how our students are doing? Because our students, we do P-based learning. P-based learning meaning project-based learning, problem-based learning, place-based learning. So if you're a math teacher, we live in an area that floods quite frequently along the river. So at what point you have your students solve equations where they're trying to determine what is the best point in which we open the dykes and open the dams and what is the cost affected or where is the cost going to be inhibiting our community or at what point do we do this and where do the cost outweigh the safety of the individuals? We could do the same thing with COVID-19. At the growth of COVID-19, at what point do we need to tighten restrictions? Where does the cost of tightening restrictions outweigh the cost of the lives that we're dealing with? These are great questions that are all within problem-based learning, project-based learning and place-based learning. As an English teacher, I invite kids up to challenges by answering questions what is America and they create podcasts and blogs and video channels trying to answer these questions. But good project-based and P-based learning starts with a question and ends with solutions, maybe not the solution, but they end with solutions because in the real world, there is no one solution. It's the solution that we're willing to live with. And these are the challenges that we're going to put onto our students. And finally, how will they know how our kids are doing? Because we put their work on display. We have, we either publish it online through class websites, blogs, et cetera. As I said, I'm having my students create podcasts and blogs and we have a class website where we're going to be showcasing our work through COVID-19, but bringing the community in to see it and not just parents. And then local businesses to show what your kids can do. And maybe that will create relationships between the schools, the students and the community, which is really what we want to do. We can also create student-centered portfolios as Orna was talking about. There are great ones out there. Orna was mentioning several of them in the United States. CESA is a very well-known one, but there's also a developing one, the Mastery Transcript, which is more for high school students. And the Mastery Transcript is both teacher and student contributed to. Students can display the work that they've done, and it doesn't just have to be an essay for an English class. It can be a project that they did for, say, they are an athlete and they rewrite the playbook for their football team. Or if they are a member of, say, the Girl Scouts and they put together a drive or something like that, they can document that as part of their portfolio because that's what learning is. It's not just, can you fill out this worksheet? It's what we do in the real world. So that is teaching and assessment. That is where we need to go with assessing learning. And you can follow me on teachersgoinggradeless.com We are a global organization with articles from contributors around the world from kindergarten through college representing all content areas. You can also follow me on Twitter. Yeah, thank you very much for listening. Thank you, Erin. This topic really resonates with me. From my own perspective, we've been playing around with doing less grading, where we do formative assessment throughout the semester, but we still have the issue and this is something that Manwala has also brought up in the questions, that we have to give a grade at the end of the semester, that we're required to do that. The ways that we kind of gotten around it is we'll give, like, pass-fail or we don't call it pass-fail, pass-no-pass, or we just keep giving formative feedback and then at mid-semester, we say, okay, if we were to give you a grade, this is what we would give you and this is how you can improve on your current situation in the course, just to kind of give them some, you know, some kind of framework to work with. But what do we do in those situations? Do you really think authentic assessments can completely replace these final exams that require us to hand out a grade? Well, yeah, I absolutely do. Now, as far as a grade goes, so that is a challenging piece and it is part of what I'm required to do as a high school English teacher and we want to push back on that as much as we can because really, you know, if we really think about it, grades are only really important to the student, their parents, and possibly the college that they're going to attend to and the insurance company who's going to give them cheaper car insurance. I mean, that's really who the extent of grades is important to. So really, it's about the student. As I like to go back to coaching, as a golf coach, we spend most of the first part of the season getting ready for our matches and so everything is practice and there's nothing recorded. It's just a lot of feedback. How do we get better and how do I develop my athletes? Well, we can do the same thing in the classroom and then at the end, we start doing our matches and we start playing. I have my 11th grade English students right now. They're, as I mentioned, they answer the question, what is America? And we're going through each little unit we've gone through, the indigenous peoples in the America. We've looked at the first pilgrims that came across and we're going to continue to systematically go through American literature historically and then when we get done in the beginning of December, my students have been introduced to a bunch of different voices but each unit that we do, we do a little bit of a project. They write a blog post, they make little videos. They do a podcast. They're doing all these little pieces. Once we hit mid-December, then I turn it on to my students, okay, what do you want to do? Do you want to do a blog? Do you want to do a podcast? Do you want to do a video channel? Do you want to work by yourself or do you want to work with partner? Then tell me, how do you want me to grade this? What is the level of quality you want me to hold you accountable to? So if you meet that level of quality, if you do the project that you said that you were going to do, you told you did it how you were going to do it and you met the level of quality that you said that you were going to meet, you should have any in the class. If you don't, then I'd like to hold you accountable until you get to that point because really that's what happens in the greater world. Nobody's giving me a grade on whether I did my attendance correct in the morning. No, I have a conversation at the end of the year with my administrator and we talk about what are my strengths in teaching and what can I do better next year? So all really, really excellent points. I would encourage you, Erin, to look through the chat. There have been some really, really positive feedback about what you've presented. This from Helen talking about just how she's been implementing something along these lines and working with the students so that they're actually doing assessment of themselves and then doing a comparison with how she would then give them a grade. I've also done this and I think it's good for students because they think about what it is that they want to learn and how they would assess that learning. And I've also found that most students, the grade that I would give them is, the grade they would give themselves is often the grade that I would give them as well. Okay. We're coming up at the end of the session, but I would like to go just 15 minutes over because we do have such an awesome panel with us today with these presentations. I mean, it's residing throughout the chat. Everyone is very impressed with the presentations today. So I would like to have some more questions. So if you have questions, please enter them in the Q&A box. But one question I have for the entire panel is, you know, I've been doing a lot of these webinars as part of the Eden the Eden organization in the last few months because of the pandemic, we did the pivot online and then we were providing numerous webinars to help teachers that were moving and transitioning into an online environment from the traditional face-to-face environment. And back in March or in April, I guess we started, we talked about authentic assessment and the teachers or the people that were attending the sessions were pushing back. This is way too much. You're talking about a solution that will take us weeks and weeks and weeks to design. It's something that is just going to be so much extra work and it's just, you know, it's just not going to be possible. It would be nice to find out now if they would feel the same. But how would you advise teachers who have the limited time to really go and do a full-blown e-portfolio design for assessing their students or any other form of authentic assessment? How would you advise them? How can they move forward in realising authentic assessment with that limited capacity of time? Who would like to take that question? I'll take a crack at it Lisa, thanks. So, I was just thinking like in March I think staff were feeling very overwhelmed by resources by all this new information and for some staff it was a big transition. But certainly in our university over the summer a lot of time was spent looking at the assessment approaches and changing modules to 100% continuous assessment just with the understanding that campus-based traditional exams weren't going to be possible. So that work, a lot of that work was carried out over the summer and in a few instances some people wanted to use Moodle-based exams as well. So a separate Moodle platform was set up for exams and essentially now that system is in place so people have kind of continued along so a lot of that reimagining had to take place over the summer and from what I'm hearing some very creative assessments have come up a lot of video-based assessment students co-creating artefacts video, blog wikis, like some really great stuff and of course there is some question multiple choice exams in there too but generally I think once people got their head around the initial period and they had a bit of time for space and reflection that a lot of good authentic assessment seems to be coming through. Richard or Aaron, you want to take a stab at that question? Aaron? I'm kind of like the worst person to ask that type of question because I've always been the type of person who just when they feel passionate about something or know when something is right we'll just jump in and figure out while we do it I just believe that if we know something if we believe in something we'll make the time for it we'll provide the resources we'll put our energy into it and if there's any hesitation in that then I got to ask do we really value that because I really do believe that when we value something when I married my wife 23 years ago I knew I valued her and I wanted to get married I mean there was no question at all about that when we wanted to have kids we went ahead and had kids there was no question at all we didn't sit here and plan out every detail of what it was going to be like to have kids no we just kind of jumped in it and did it and I think we've done a pretty good job raising our kids well I kind of run my class that way too I learned something I do try to do the research on it but sometimes it's just going to take us time but I don't know I just believe that we have to do what's right for our kids Thank you Richard Yeah Aaron and I want to have both said amazing things and I feel exactly the same way where I work in my place I'm with the push back from the instructors so I have probably about 80% that are just against authentic assessment you know okay that's a figure that's coming from all from spring and what I want to say is very true the people that were open to it you know tried it and now it's wonderful and Lisa's question was how do you get people to invest to try you showed them those wonderful examples that Aaron has showed his websites and has shown those wonderful e-portfolios they see that and they even the ones who are totally against it go wow I don't do that in my course and then they see the learning because they might not know authentic assessment you know and they see that and they go wow look at that application so that would be me if I were somebody out there that were going I don't know about this stuff I got to give them a grade you know all the systems things that I can't escape from what is this stuff about I want to learn more I look for some good examples of it and then slowly as Aaron saying I think eventually you can kind of come around to that when you're willing to try it out and if you can also make the jump into a real world we had such a good somebody asked in the chat a company you know why can't that a problem at a real company a real problem at the school institution then becomes the project why can't that be used if that were then done by the leadership in the department where the department would say to everybody okay we're going to try this everybody authentic assessments but that buy in I love what Aaron had to say you got to believe it and the problem is they don't believe it so how do you get them to at least wanting to try it to believe it and that's only as you said Aaron but then also through the great examples that you and Orna are showing that's how you're going to make believers of people because seeing is believing now I'm going to jump on and share something that caused me to transition over it was about 15 years ago my school started doing what we call student-led conferences and students would bring examples of works and students had to lead a parent-teacher conference and it was one of the first couple it was like the first couple meetings that I had that first year that we were running them kids were bringing worksheets to showcase for my class and I was embarrassed I was absolutely embarrassed and I thought you know what I can do better if I really want to be proud of what my kids are showing then I got to have them create things that they can be proud of to show their parents so I think again pushing things like showcasing work student work on display is I don't think teachers kids want to showcase things that they're proud of and teachers don't want to be the ones that are left behind where the kids aren't showcasing their work so I would push for showcasing student work in one way shape or form great we do have another question within the chat box or within the question and answer box and I think this comes from the STEM area what about practical skills that students need to accomplish things like dissecting frogs using a microscope all of these things that get done in lab how can these be addressed with authentic assessment I mean that seems like something that has to have a process one word video take a video of yourself dissecting a frog there you go done there's nursing nursing students in DCU handwashing was one of the competencies they had to achieve so that's exactly what they did they were given instruction they were given some practice time and then they had to create a video of themselves carrying out the proper handwashing procedure and submit it to their portfolios other comments what about engage oh go ahead go ahead no I was just going to jump on what that was what I was going to say but when I said to and of course the other answer is there's all kinds of these lab simulations you can find a lot of them online and practicals but I think the question is excellent because you all know deep down there's just going to be these rocks that are kind of steps in the journey towards getting to 100% authentic but it's a culture that we kind of have to change or that almighty grade you know that's used for so much and this just hurts people's identities in the long run you know they think of themselves almost for life you know this is what I got on my you know test score whatever it just has to be done with where they want to be lifelong and learners and move on and I can jump on to that too like yeah creating a video for say hand washing I love the idea of hand washing as an assessment that is a good idea but really you know maybe have them create an infographic explaining why hand washing is important and how to do hand washing properly I mean a video of kid washing their hands is one thing but explaining why it's important and how to communicate why it's important take some a deeper level great example really really like that just one other question before we close up and that is you know how do we get our students to engage with authentic assessment is it your experience that they prefer this kind of assessment or that they prefer not getting grades I mean what where is the I guess the incentive for them to engage with authentic assessment my experience has been totally sorry positive with it every authentic assessment I've used my business communication classes my advanced verbal communication classes with the interteaching online has been positive I think it's because I have so many college students who have to take a test and write a paper they've got five courses and every single one of them is take a test and write a paper and they're all in the same two week period where everything's due at once so when they get something that oh no you can see the smiles and then at the end of the course they say this was harder sometimes or I spent more time with it but the appreciation because I happen to do a reflection in it it's pretty rewarding even though the protests about wow I had no idea what I was getting into that this was a little bit more involved you know that too so I was wondering if the others had comments yeah I agree for the most part most of my students really enjoy it now I do have some kids who have been who do push back on it they kind of want me to do the traditional read the book and take the test because it's something that they're familiar with and I but over time they get it they understand that this is not what's going to prepare them for more schooling but this is going to prepare them for life outside of school and I think when we frame it that way it helps kids work this in what we're doing or no you want to finish up okay well I suppose I've had mixed reactions often they're often students are initially quite skeptical but because sometimes they feel like an essay would be easier to do but you know there's a kind of a sales job I found at the start when you're working with a new type of assessment you know you have to sell it to them in that you know this is going to be a richer experience you're going to learn a lot you're going to learn some new skills and tools and often I think that that's an important part at the start you got to build you got to build the belief but you know even just thinking back to the case study group I worked with out of the kind of 30 or so students I think there was only three who really disliked it they still did it though because obviously they wanted to work but the majority of them quite enjoyed it towards the end and they also kind of understood it more as time went on they could see the development in themselves and they liked that fact they could see their own progress Yeah I've experienced that as well where students I think you push them out of their comfort zone and then they reach a point where they're kind of like hey I'm actually learning something here and this is a good thing I'm definitely a proponent for authentic assessment so I just I want to thank our speakers today just fabulous you guys were great I mean so many great ideas so many practical tips so much information it's going to take a while to just process all of that and I don't think I'm the only one that thinks what a great bunch of speakers we had here today if you read through the chat and please do any wonderful comments and feedback on what you presented today so thank you thank you very much for all of the work that you've done and for your presentations today and taking the time to answer our questions and to share your experiences and your research with us so thank you very much just as a reminder we have European online distance learning week is continuing we will be having more sessions tomorrow there are two sessions one is practical and pedagogical ways to assess your students online which will be led and moderated by Orna who is here with us today and so we'll be exploring the whole assessment thing again and in more detail and from different perspectives great lineup of speakers there too and then there will be also the topic of next normal including students views into shaping the future of higher education and how we can go by doing that so please do come back and visit us and watch the sessions again and thank you all of you for coming today and for attending the session we'll see you later