 So hello, you're very welcome to today's webinar as part of Eden's European online distance learning week 2020. So we've got a fantastic panel lined up for you today and our topic is practical and pedagogical ways to assess your students online. I'm Orinna Farrell, I'm going to moderate the session with my colleague Vlad Mihescu. Sorry Vlad, if I butchered your name. So please go ahead and say hello in the chat as we're coming in. We're expecting quite a large group this morning. So we might just give everyone a few moments to join and do say hello and tell us where you are and even better, tell us what the weather is like because I always like weather information. So we have a brilliant panel for you this morning. First of all, we're going to hear from Suzanne Stone and Rob Launey from Dublin City University, and they're going to share a brilliant resource that they've been involved with the creation of. It's a lovely crowdsourced resource on alternative assessment. And then we have the FAB team from ePortfolio Ireland, a community of practice who obviously like ePortfolio. Tom Farrelly and Karen Buckley will be joining us from ePortfolio Ireland. And then last but not least, James Brunton is going to share his experience of open pedagogy assignments within an online psychology program based in DCU Connected. So fantastic group. I see about 90 attendees in so far. So I'm going to crack on now in just one second. Rainy Lithuania, that's disappointing on that now. So during the session, guys, feel free to interact in the chat. If you like, you can also put questions in the Q&A box or questions in the chat. And we will have time at the end to answer and chat about the questions, but also we'll have a brief pause after each presentation to take a few questions. So you should have lots of time to ask questions to the panelists and find out a bit more. If there's relevant links or any resources you think would be useful to share, please go ahead and pop them in the chat. And we'll also do the same with the presenter's slides and any resources that come up as we go along. So without further ado, I'm going to introduce Rob and Suzanne from Dublin City University. So Rob Launey is a learning technologist in the teaching enhancement unit. He works with academic staff to enhance the teaching learning assessment practices on their modules with digital technologies. Rob has a particular interest in empowering academic staff to implement technology enhanced assessment. Other areas he's interested in are flipped learning, mobile devices and education, learning analytics and reusable learning objects. And then Suzanne Stone is also a learning technologist in the teaching enhancement unit in Dublin City University. Suzanne has been involved in research relating to educational technologies over a number of years, including projects on a drop-in approach to professional development for Moodle, the development of a vocabulary assessment tool, student engagement in live online classrooms like this, the use of social media applications as back channels. Suzanne holds an MSc in education and training from DCU and is currently engaging in a professional doctorate in digital learning. So over to you Suzanne and Rob. Thank you very much, Orna and Vlad and all the rest of the team for inviting us here and for that lovely introduction. As you said, my name is Rob from DCU and my colleague Suzanne is here as well. And we're going to take you through the first presentation here in this webinar. We'll divide it up half and half. So I'll go first and then I'll hand you over to Suzanne. So first of all, what we'd like to do is give you a little bit of background to the development of this open education resource on assessment exemplars and where it came from, where it sprung from. Then we'll take you through the rationale for creating the OER, the process that we used to crowdsource the entries in the OER. And then we will round it off by giving you access to the OER itself. So this all kind of started with the Enhancing Digital Teaching and Learning Project, which is a three year nationally funded project funded by the Higher Education Authority in Ireland and coordinated by the Irish Universities Association, which is the representative body of the seven public Irish universities. And this came to being, I suppose, because as we all know in recent years, there's been a lot of attention given to the need for academic staff to develop digital skills and competences to embed digital teaching, learning and assessment in their curriculums and in their modules so as to improve the digital learning experience of students. And the IUA capitalized on this and received funding from the HGA to put together this project to achieve that particular aim, which is ultimately to enhance the digital attributes and the digital educational experience of our university students in Ireland. And that aim is also complemented by developing and implementing a staff development program to help improve the digital skills and competences of all those who teach in Irish universities. And we particularly use the term all those who teach. We don't just focus on full professors or assistant professors or anything like that, but we take a very broad approach to treating people who teach and people who support teaching. The project largely is centered around campus based instructors and helping them develop their digital skills in teaching and learning. And even though the project itself, the title is about digital teaching and learning, really it's about digital education as a whole and it covers digital learning design of programs, curriculums and modules, it covers teaching and learning obviously. It also covers assessment, which is a particular area that myself and Suzanne are interested in at DCU. I see Sharon Flynn is here as one of the attendees. Sharon is the project manager within the IUA. So please, Sharon, do jump into the chat if you have any other contributions you'd like to make. The approach is ultimately to mainstream digital teaching and learning in Irish universities by addressing the professional development of all those who teach. A lot of us would be very familiar with this concept of champions or the one or two lecturers or professors in a department who are very, very skilled when it comes to digital teaching and learning. They're the go-to people whenever anything digital takes place in teaching and learning. And I suppose the approach of this project is to move away from that idea of champions but really to help mainstream digital in all that we do in university teaching and learning and to help mainstream everybody's skills in universities. The project is underpinned by four pillars. The first one is this concept of not starting from zero because we are aware that professional development, training, non-formal learning in this space takes place in all our institutions. Many institutions have accredited programs or accredited modules or offer suites of workshops or mentorship or occupational training, et cetera. And this project certainly doesn't aim to come in and replace all that but really build on what the activities that are happening in the seven universities. We very much think about pedagogy first. We don't come in with a suite of tools and say you need to use these digital tools for assessment and learning and teaching. But we talk about the why and we talk about the rationale first and the technology comes later. We operate on a discipline basis. So we work with teams, teams of lecturers who are grouped together because they work on a particular program or they're all part of a particular department or a particular discipline. We don't work with kind of ad hoc lecturers or we don't work with a mishmash of lecturers from across the institution. We're very much discipline focused on working with teams to help build up their skills and competences. And the student voice is very important to us. We have students involved in the project at the student group level. We ourselves, myself and Suzanne have good relationships with the students' unions and student leaders in DCU. And I'll come back to a new development we have as well around the student voice, which has been particularly useful and exciting in recent times. All that we do is aligned to the DigCon BDU framework, which I'm sure most people here are familiar with. So all of our activities are linked to areas or to competencies on this framework so that the staff who are participating in the professional development activities have some sort of understanding or some sort of map of where their skills fit in the grand scheme of things and where they could potentially go to develop their skills further. So that's quite an important feature of the project as well. As I mentioned, the project is across the seven Irish universities. We have a steering group made up of a number of members of senior management from the seven universities. And we found that's quite important to have their buy-in and their support with this three-year project because that will go a long way to having some long-term impact with the support of senior management. I mentioned Dr. Sharon Flynn, who's here attending the webinar today. She is the project manager, the wonderful project manager in the IUA who coordinates the activities and keeps us on track and keeps a bird's-eye view of how the project is running in the seven universities. And then within the seven universities, there are project teams. So myself and Suzanne are the project co-leads in DCU and we have six counterparts in the other universities as well. We do have a student intern, Rory O'Gallagher, who works with Sharon in the IUA. And recently, we have also recruited a number of interns within each of the seven universities and we're blessed to have a wonderful intern, Laura Hans-Scanlon, who came on board with myself and Suzanne to bring the student perspective and the student voice around all things digital teaching and learning. We conducted pilots in the last academic year in each of the universities. Again, taking the approach that we're not starting from zero, we were aware that there were activities in all of the seven universities and it just didn't make sense for one kind of super approach to come in on top and be bedded in from the top down. This is really a bottom-up approach to developing digital skills and competences. So even though we all ran a pilot in the seven universities, the pilots were all very, very different as you can see on screen there. And we in DCU decided to focus on the area of assessment. So we wanted to build lecturer skills in digital assessment through things like structured workshops, training, consultation, guidance, and so on. And Suzanne will delve into that a bit deeper shortly. How our project maps with the underlying principle or how our project in DCU aligns to the overall pillars of the national project are that we in the teaching enhancement unit, the TEU in DCU run a number of different professional development activities on an ongoing basis in a number of different areas. So therefore, when we started our EDTL project in DCU, we didn't again start from zero, but we looked at what existing offerings we had in DCU and where we could take those and adapt those and build on those and bring in new offerings and bring in new workshops and so on. We very much focus with our staff around why to use technology in assessment or we look at why to use a particular assessment approach and what's the benefit of that and what are the pros and what are the cons, et cetera. And then we very much look at the how-to later. So a lot of our workshops, for example, would be very discussion-based and teasing out assessment issues with staff and teasing out how technology can possibly help. And then we would usually follow up later with maybe some technical support and technical guidance. Our workshops are nearly, even though we have a suite of workshops available, they're always tailored in one way or another to a particular discipline. So if we're working with the school of languages or if we're working with the business school, they might be interested in broadly the same topics and workshops, but they'll always be tailored specifically to them. As I mentioned, we have our intern Laura Ann, who's wonderful in DCU. We have good relations with the Students' Union and their voice and perspective is important on this project. And we also run focus groups with students to hear from them as to their digital experience and their digital attributes during their time of study in DCU because, again, ultimately the aim of the project is to improve the student digital learning experience. So we're quite conscious of that. I'm going to hand you over to Suzanne now if that's okay. So I will stop sharing my screen and I'll let Suzanne jump in now and share hers. Let's bring in Rob, thanks so much. So continuing on that conversation, just to talk a little bit about why we decided to focus on technology enhanced assessment. I guess the research and the literature show that students are very much focused on assessment and that's why we took the approach of tackling assessment head on because it has an influence on teaching and learning in all its facets. So that's why we chose assessment. Our project is built around four work packages centered around a staff development program and Rob spoke a little bit about our approach to that program, but one of the four work packages and you see all four of them there is the development and maintenance of online resources and that's what we're going to talk to you about today in terms of this open educational resource. So as we piloted the EDTL project at DCU, we realized that we had a need for an open education resource or an online resource of some kind. We didn't immediately think that it would be open, but we needed a resource to support staff to imagine alternative types of assessment. We found that many academics were wedded to that traditional form of assessment, whether it be an assignment or even say from a technical point of view, an online quiz, we wanted to move beyond that and give them some ideas around the types of assessment that they might consider which would be supported by technology. And actually we did find some resources online and through the literature, but we decided that we needed some more because I guess we're working from a discipline-specific point of view and in many cases, for example, in kind of the humanities, there weren't a lot of exemplars or case studies available to us. So that's where the kind of the idea for the open education resource was born and why did we take a crowdsourcing approach? Well, we felt that there's a very rich community around technology-enhanced learning and we felt that it was wise to draw on the community knowledge and experience. And then why take an open education resource approach? Why take an open approach? Well, it aligns with the teaching enhancement unit open practice. The teaching enhancement unit is the unit at Dublin City University, where myself and Rob are based, but it also aligns with our own personal philosophies of education. And also, leaving the resource open allows for continuing development and a building of the resource over time. So that's why we decided on the open education and the crowdsourcing approach. Just to speak to you a little bit about that crowdsourcing approach, we obviously hadn't intended for it to all happen online, but obviously events conspired in March and that's how we have developed the resource. We started at the all winter conference last December, so that was before the COVID, but we had intended that this would be something that we would present at conferences and that we would do in a face-to-face context as well. But we have, in fact, developed it online since last December. And it's building nicely. We have 30 exemplars in our resource now, but I guess one of the difficulties of co-creating a resource online is in our first outing, we realized that we needed to kind of counteract that silence. It can be quite a non-nerving experience for the presenters or the facilitators to have a session where people are working independently, because it can be quite silent and you can feel quite alone. So our approach was to take a kind of a relaxed and fun approach. We had some music playing in the background. We had some spot prizes, which went down very well. And then other kind of pointers if people are considering this approach, there is a level of scaffolding required. We scaffolded it through discussing what we were seeing people work on throughout the session. We didn't pressurize people to contribute or to create an exemplar on the spot. Participants were invited to browse if they wished. So that kind of took the pressure off people. And then because it was, you know, every time that we've developed this resource in an online session, they've been short enough sessions. So sometimes people need to go away and think about the exemplar, the case study that they've contributed to the document. So we've left the document open for a couple of weeks after each event so that people can go back and edit if they wish. So again, that takes the pressure off to contribute very quickly on the day. We also invited contributors to discuss their case study in the webinar, but again in a kind of a very nurturing fashion, no pressure. And we've kept the atmosphere extremely relaxed throughout all of the sessions. Just to introduce you then to the resource itself, it's available here. We'll pop that into the chat box. And I think we're sharing the slides after this session in any case. So it'll be included in the slides as well. So we've published a couple of times. We've published earlier in the year the first version of the resource. And we've published again just this week a second iteration of the open education resource. And what we're going to do is leave that as it is for the moment. And then later on, maybe next year in the spring at some point, we'll open it up again for contributions. But at the moment, this is our published resource. And just, I just want to open it up and just show you very briefly one example. Can you see that case study, Orne? You might nod or rub. Brilliant. Thank you. Excellent. So you'll see there's a small introduction and Rob has spoken to you a lot about that. We've got actually a table of contents. It's created in a Google Doc. And I know we probably could have looked at something a little bit more polished, but actually it just facilitates the contribution really nicely. So we've stuck with the Google document for the moment. So you'll see there, title of the exemplar. We've got 11 headings in total. So you get an overview. This is an extremely comprehensive entry into the OER, but you get an overview of the technology enhanced assessment, an idea of the discipline that was used or the disciplines that the contributor feels that it might be relevant to, the learning outcomes it could align to, skills, graduate attributes it could align to, which speaks to the DigComp edu framework, referring back to that. The technology is involved, some practical guidelines and considerations, possible grading criteria, some testimonials, which is fantastic. And then we've invited people as well to link outwards to something like a student assessment brief you'll see here. So that covers an institution name, just in case you want to contact the person to have a chat in more detail and other notes. So people have chosen, Felicity here has chosen to add some resources into our exemplar. So that's it guys. I'm going to share out the link to the resource into the, I think I've stopped sharing, just give me two secs, pardon me. Go back to the slides. Yeah, it's just left for me to thank you for your attention and we're available obviously for questions. I think, or now we're going to take a quick question here at this point and then some more questions at the end of the session. But thank you for myself and Rob, and we're delighted to be invited today. Thank you. Thanks, Suzanne and Rob, fantastic resource. I have a question for you just to get the ball rolling, which is your favorite example? I mean, there's some, I noticed there's some really nice additions since the last time I looked, you've got some really interesting stuff in there. I don't think the podcasting one was there before, very nice. So you've got a really, like you've got a depth of types of assessment, but also a real depth in terms of disciplines as well. Yeah, absolutely. And that was one of our, our reins of the, the, the creating the resource in the first place, because we found that in some disciplines, there were plenty of case studies available around technology enhanced assessment. But for others, I think I mentioned earlier already for the likes of the humanities discipline, we didn't have as many. So that's growing. Still, we, we, you know, we're always open to more contributions, but in terms of a favorite, a recent favorite of mine is, I don't have the number in front of me, but it's an assessment that uses, that combines two tools. So the Moodle Glossary Tool and the H5P Open, open source content development tool. So essentially what the user, what the exemplar explains is that the Moodle Glossary was set up so that students could create and share H5P content. And the glossary was used to collate those students commented on the, the H5P content types. So I thought that was a really nice idea combining, it sounds reasonably complicated, but actually when you break it down, it's a really nice, simple idea and builds a lot of digital skills for the students through that process. And you've got a lovely bit of peer review going on there too, which is just really nice. And that glossary tool is a great tool, vehicle for that Moodle. So any other questions at this point? There was one in the Q&A, which I might just put to you as well. So is it specific to Moodle or other tools? From me glancing at it, it seems open to lots of tools. Proctoring doesn't feature, I don't think, does it? No, it doesn't. Rob, do you want to come in there? Yeah, no, we don't have any examples of proctoring in the OER document. I won't open the debate on proctoring. We might be here all day. But generally, as an open education resource and as one that's crowdsourced, the tools are varied. They're not specific to any particular VLE. And that's exactly what we want. We want it to be as diverse as possible and we want to crowdsource as widely as possible. So the more open, the better, I think with this resource. Absolutely. And I'd say staff really appreciate those examples because imagining what an assessment can look like, it can be really challenging. So having good examples like that can really help people design some good assessment. And I see Diane coming in there and proctoring. Actually, at the session yesterday, we did start proctoring bashing. So don't be afraid. That's one of my hobby horses too. So I don't want to explode the chat box now. But thank you, Susanna, Rob, really interesting contribution on a super resource. I have to say I really like it. And maybe if people from the session were interested in contributing to the resource, they could get in touch with you if they had some interesting examples. We would love that. We would absolutely love that. Thank you very much. Super. Thanks to you both. Now, for our next dream team, we have Karen and Tom from ePortfolio Ireland. A fantastic community of practice. And I am completely biased because I am also in this community of practice. So just to introduce them both, Tom is a member of the steering committee of ePortfolio Ireland and a social science lecturer and educational developer at IT Trili. His research interests are in technology enhanced learning, open education and ePortfolio. He is editor of the Irish Journal of Technology Enhanced Learning. And Karen Buckley is an assistant professor in the School of Inclusive and Special Education at Dublin City University. Her interests include inclusive pedagogy, inclusive practice. And she's a doctoral candidate at Maneuth University where she's exploring professional development in higher education. So over to you, Karen and Tom. Thanks, Orna. And hi, everyone. It's great to see you all here. I'm sure you can see my screen. Orna, you might just give me a wee thumbs up. Fantastic. My name is Karen Buckley and thanks for that warm welcome and introduction. Orna, delighted to co-present with Tom here today on behalf of ePortfolio Ireland. We're a small network which includes Lisa Donaldson, the esteemed Orna Farrell, Tom and myself. And maybe just to give you all a little overview of what we do. ePortfolio Ireland is a professional learning network. We are ePortfolio practitioners and researchers, and we're all based in Ireland. There's a bit of a Dublin City University slant. We're Orna, Lisa and I are based in Dublin City University. So poor Tom is representing the South and he does it very, very well with lots of classes, you can imagine. And we were really established, I guess, back in 2017 as Mahara IRL. But we branded quite quickly back in 2018 to better represent the broader interests of all members. And again, I suppose this was a decision based on, I suppose trying to consider all tools that are used for ePortfolio practice as well. So we are a grassroots community. We have a voluntary steering group. We're really quite active and online and face-to-face events, or we used to be more active and face-to-face events. But really our main mission and aims is to support the professional development of all those who teach an ongoing collaboration between ePortfolio practitioners in higher education institutions. We've also broadened out to further education or adult education institutions here in Ireland. And actually we're seeing a lot of members in our community who also work in second level education settings as well. So we welcome all and we're delighted to be able to present to you today on some of the activities that we have. And I suppose I may be going to start a little bit talking about what type of ePortfolio activities we have in DCU. And then we're going to pass you over to Tom then where we're going to talk a little bit more about some of the opportunities that ePortfolio offers to be able to engage with pedagogically robust alternative assessments. And that's really our aim for today. So some of the activities that we do and I want to maybe draw your attention to our website which is ePortfolioarland.wordpress.com I have it on screen there. And what you'll see is there's lots in our website about what we do. Some of our recent activities. Also some really nice free open source resources that are available to you with the events that we've held over the past three years as well. But really our ultimate aim is to share our understanding of ePortfolio practice in Ireland. To support robust assessment. To support employability skills. To support good teaching, learning and assessment across all elements of our practice. And we're really lucky that we have I guess a great network of ePortfolio enthusiasts with broadened or the variety of different skills and knowledge and competencies with regards to ePortfolio. And even I guess within our steering group we have different levels of expertise with regards to ePortfolio. And that certainly has been and it's shaped us to be one of a really interesting grassroots membership because we were able to maybe share practice, we were able to promote ideas and also be able to connect practitioners with what is happening on the ground as well. It's been really, really interesting. How ePortfolio has offered us an alternative assessment, that's pedagogically robust. I suppose we in DCU in particular have considerable history with ePortfolio based assessment in particular. ePortfolio has been fairly widely adopted across all faculties, all five faculties in DCU to serve a variety of teaching, learning and assessment purposes. And we see that from an initial launch back in 2017 of our loop reflect as we call it we have an increase in the number of active accounts of up to about 42% of all users across the university. Which equates to about just under 10,000 active accounts in 2019-2020. So it's a really significant and important approach to our I suppose teaching, learning and assessment practices here. We're really trying to support all practitioners to test and to try out ePortfolio as a tool to support teaching and learning. And as I mentioned that ePortfolio has been fairly widely adapted across our faculties I suppose in the last seven or eight months with COVID-19 it has really proven to be a catalyst for lecturers to embrace ePortfolio based assessment as an alternative assessment. And this last semester alone over 60 separate modules here in DCU are using ePortfolio with their students. As the pandemic has required us a rethinking and a reimagining of assessment due to the inability to conduct in-person examinations. And just the way that we are teaching in a more blended or a more hybrid approach. So I'm going to pass you over to my colleague Tom to talk about why ePortfolio is useful and why it can be used as a alternative assessment tool. Thank you very much, Carol. Absolutely professional and unflappable which would mean an awful position because I wing it most of the time like that and relying on my innate charm and good humour and apologies there's a little dog in the background you may hear barkies I don't have children, small children to put open but my baby is 25 so that's a slightly different issue. Anyway, I think Karen has really hit the nail on the head in terms of the last year I think we have been challenged obviously with doing a lot of things differently and paraphrasing David Bowd when he talks about we can students can to varying degrees of success get away with bad teaching but bad assessment is very difficult to overcome so it's opposed to the whether we like it or not we talk about education and all of that go to a lot of times the end game is the assessment strategy and we need to understand there's a lot of concerns about reliability and reliability and we would certainly be arguing that ePortfolio certainly provide that opportunity so we would I think we would talk about it because we think it's a very good process and very good education even for those who are more concerned with reliability and validity I still think it would certainly give a much broader aspect and more robust way and not just as if you think about ePortfolio come from some more art based subjects but one thing we've seen here is a whole wide range of academic disciplines and also if you look at a lot more professions I'm now moving towards that idea of continued professional development how do you capture that in a dynamic way and I suppose ePortfolio is in particular but in general but the ePortfolio is in particular because we actually do have that opportunity to move forward so as I said they're just quoting here a couple of quotes here they're student-centered activity one in which the student is free to choose what artifacts are included and so that is one of those major things it's actually showing if you look at sort of what's our higher end skills we're talking about blooms or the Revoyance Version and the Taxonomy there and at the high end we're talking about selection and creation and that's really showing that higher level of skills there so as I said it's really putting it back to them rather than here's a 3000 word essay here's the title I'm going to give you no I'm actually giving it back to you that can be quite scary and brightening for people you know a lot of people we've become so accustomed to what do I think what do I think sir so I suppose that's the thing there and as I said students so I suppose it's that idea that integration and inspiration it's really giving back that choice to say for innovative not-tending assessments and one of the things there myself and two colleagues and Tralead on some research last year they certainly bore out a lot of what we found I think a lot of people what have you where in the culture they didn't reuse any portfolios because fair enough if you're using pedal pad or pad right on your heart and then you're using it a definitive definitely portfolio platform but actually a lot of people were using the journal facility on their VLE or some people were using what they've got the students to do was curate six videos and write a description on YouTube that's an E-Portfolio I need collection of work where you've had the selectives over there so as you can see here there's a hell of a lot of choices in terms of what constitutes an E-Portfolio the main thing is that as I said it's innovative and that's authentic because you're actually in real life so the street use that awful phrase that's what you're been asked to choose can you do this for next week can you put a solution and this is an E-Portfolio is a manifest way of saying this is my work this is what I chose this is what I produced there so if you look here as I said this collection I mean one of the things I particularly love E-Portfolio the traditional for the portfolio it's just words and just text but if you think about it there's so much more like one of the projects inside that we've been involved in you know with E-Portfolio from North East June so one of the learning outcomes for North East that used to be a health promotion well now you can actually maybe make a poutune video or some sort of thing and then put it into your collection rather than just writing up I doing this and writing that's not to say writing doesn't have its part to play I think the whole reflective element and sort of teeing it up because that's also shown the process and how you actually capture that and as I said that also then I particularly like about Portfolio is where you can show that the student has written something the lecturer can comment on it and then you feed back that's real learning that's that real sort of learning and as I did the record learning from more placements there I think so many degrees now have some element of placement or some element of going out and putting it into practice so this is a tangible way because if you think about it a lot of students when they come out you know from university yes they have the degree but what is happening to show because there's lots of people with a degree so what's going to mark you out and this is a tangible way this is not just me sitting at the interview I can send you a link on so in advance of the interview it's already changed and become a far more dynamic process so there's some examples there Karen and me are we actually playing the video this is where they are we just so as I said so this I think in terms of actual practical resources and I'm going to let Karen now come back and hopefully show B portfolio where I'm pretty aware that owner is you know what he calls me unflappable but Tom you're an absolute pro so I won't take any heed of you know any more with this unflappable nonsense it was absolutely perfect Tom and I mean I think you absolutely hit the nail on the head it is about opening up new opportunities what are the potential for E portfolio we're all having these conversations now about alternative ways of doing and actually following on from Suzanne and Rob's presentation I think one of the real beauties about having these conversations like we are in the evening webinars is about actually trying to share some of our existing resources and I think just popped into the chat there a moment ago where we have some really nice examples of E portfolio based assessment in an ebook it's a free open resource that's available to you and and it really tries to provide a real life examples ways of trying out whether it's taking small starter steps in E portfolio practice or maybe if it's about refining or redefining how you use E portfolio for assessment in all of your teaching and learning but there's really some nice ideas it was a kind of a crowdsourced way that we could disseminate and to share some key practices in E portfolio that are going on across Irish institutions over the past couple of years so I think that would be a really nice resource for you to look at and for you to explore and I think it would complement Suzanne and Rob's recent publication as well I think they could really be helpful to any new practitioners or you know for any of those who want to re-examine or redefine what we want to do moving forward for the next year I think we will wrap it up and would love to take any questions that you do have on E portfolio Ireland and as ever we will share these slides and any resources that we've mentioned but please check us out on our website www.arland.wordpress.com or follow us on Twitter, we're quite Twitter active so get in touch with us that way and we'd love to answer any questions that you may have Can I give a shout out that we hopefully will have a special issue of Irish Journal of Technology and Enhanced Learning which has been guest edited by E portfolio Ireland so we're busy putting all that together and hopefully in the new year we'll have that edition out Thanks Karen and Rob and I've been setting off the chat a bit there getting a bit of debate going about which is important the process or the product so any thoughts on that to begin with and there's actually a few more in the Q&A I'll put to you in a second as well so process or product Oh I mean I'm a teacher educator I'm a moonshore by trade so I have to say the process I mean I would be fired on the spot if I said it was all about the product but ultimately I mean I guess the reason we're talking about assessment is that it's a huge part of our quality standards we can't ignore you know the end product is a way of measuring our student performance but you know I think it's combination of both it's trying to find that balance is the sweet spot that I think we all find really difficult I think the beauty of E portfolio is at least presenting the process to the viewer I think particularly for group work I often have some reservations where you see assessments characterized as you know this is about assessing the group dynamic and the group processes but actually when you actually look at the how the marks are allocated you're going to go no you've actually done it for the product so I think E portfolio is a way of genuinely mapping that process okay so yeah that was just me you know getting things going during the past you probably already those of you who know me probably already know my answer I would always reward the process more heavily than the product so I'd be in the waiting I would be giving the process 80 or 70 and the product the remaining because that's the message you want to give to students is we reward the effort we reward the messy learning not the shiny thing at the end because actually where the learning at portfolio sorry Tom where the learning happens in portfolios is the messy bit go ahead Tom but the problem is I suppose unless you have built a robust assessment system the easiest thing to assess is the product because you can see it you can feel it you can weigh it so to speak and I think that's the thing where something which you know you know the paraphrase another quote everything that matters we can't always count and everything that we can count doesn't always matter and I think sometimes is that unless we are prepared for it in the hard work to find a way of tracking that process we will end up just oh look the shiny piece of work that you exactly and you don't see the blood sweat toil and tears that went into the production to quote Winston Churchill and so lots of lots of interesting comments I might just pick out one other one comment actually which is a nice one any advice for someone just starting off with the portfolio or considering introducing it into their students start simple I think that's the thing like you know there is a temptation to become enamored by the technology and as I said like I'm not saying I'm not saying we shouldn't be raising the bar but the problem is you might be particularly adept and I think going back to certainly now that people are walking at a distance you don't know what broadband capability people have you don't know what software packages that they have I think I've had students you see here are sharing family laptops and it might be a seven-year-old laptop so even relatively what I think is relatively okay and easy to do so I think pick something which and above all else it's still about the pedagogy and Karen has said that and that was when she started off the whole thing it's still about the pedagogy you know can you make this fancy doodly or video square video maybe you can but someone else can't so sorry I'd agree with that I would agree with that Tom I think one of the key and most important features of a successful ePortfolio implementation actually has to do with time and I totally take on board the right technology is important looking at the right tool that's obviously important using what you have in your own virtual learning environments or whatever tools are available to you and your students but actually it's about building in a scheduling time actually regular consistent use of engaging in the process of ePortfolio development both for you and your students I would think is an important feature and a space to start for me I have a range of different ePortfolios some for my professional practice some for my studies my doctoral studies but also think about some of the micro blogging tools that you already use I mean are you engaged in social media do you blog on Twitter do you use LinkedIn for example as a blogging tool or as a way to enter some of your academic or work achievements but think about maybe some of the tools that you already have see what works for you see what styles of technologies will work and just put the time into maintaining it and to foster it that would be my recommendation. Thank you Karen and Tom some really interesting conversation going on that hopefully we'll come back to in the final Q&A session so some very heated debating going on about assessment which is brilliant and just to echo your sentiment there about Karen about the professional ePortfolio it doesn't have to be an ePortfolio platform if you've got a blog or if you've got a Twitter I totally agree that is you reflecting and collating your professional development equally as valid so our final speaker is another the fabulous Dr. James Brunton also from Dublin City University so there may be a bit of a DCU team if you haven't picked up on that yet so James is an assistant professor and chair of the DCU connected BA in Psychology major which is an online program based in DCU so online before COVID still online now. James is a member of the digital learning research network his research interests include the psychology of identity formation socialization and orientation processes for off campus online higher education students online learning design open pedagogy and digital assessment and so today he's going to talk about some really nice open pedagogy assessment work he has worked on for the last year or two James is it? Sorry my unmute button moved for the last three years I think 2018 was the first year that we started doing that can you see my screen? Sure can. Do you want to go full screen there James? You're just telling Andrea to be quiet. Here we go. You can see that now can't you? Great. As Orna said I work in DCU I work in a DCU connected program I've worked on online programs open access program since 2010 here and what I'm talking about today is part of our efforts to bring in more open pedagogy assessment into the program so also as Orna already said so I'm the chair of the psychology major program and that means in our kind of model we have a particular type of online distance learning model where it's a small full-time team who then coordinates with a bigger part-time adjunct faculty team so we're very much kind of at the cog of this big web and we're trying to make everything stick together so as part of that a lot of the drive for innovation comes from us as program chairs or module coordinators on these programs and I suppose over the 10 years that I've worked on programs in DCU I've gone through a series of innovations trying to increase the variety of assessment types in different modules it's a common theme across the different presentations today trying to shift the balance from being more heavily weighted in traditional essay style assignments to having a nice wide variety that feed back into different program learning outcomes so how people communicate, how they work together how they reflect on things bringing in e-portfolios into some of the modules and then working with the part-time teams to try and make sure that everybody in the teams is fully on board and trying to overcome some resistance that some people might have where they're like no, we should just have essays or report writing, they should just be writing all the time or people trying to get used to how do we assess this it's harder to figure out how to assess someone using this rubric for reflection than it was to just correct what people feel they kind of have in their soul they know how to assess an essay in some of our psychology modules this process was ongoing and I'm seeing things on Twitter I'm hearing things in events like this, I was especially in some conferences seeing presentations on really interesting ways to assess them some of all the global conferences and other conferences like that all the conferences in this area and I was seeing people talk about different ways of opening up assessment to get away from this idea of the disposable assignment that people do their assignments it gets corrected, they get their feedback and then we put it in an archive and it never sees the light of day again we just put it in the shredder and everyone forgets about it and I put out the call to my team to see what can we do what kind of different ideas where could we bring in real life examples or real life data sets where could we get students to make things that go out in the public where might students work in the public in the public eye or where might they produce things how can we get them making stuff that will have a purpose have a life of its own outside of just being something to be assessed so Megan Gaffney who works on a couple of our modules was the first person to come back to me and say I love this idea let's try that so on this developmental and educational psychology module we started to sit down and look at what could we do differently so I suppose another thing this is part of this is only one part of trying to be open we try to be open in lots of different ways like the program that I work on even the family of programs that it comes from the humanities programs they're open access programs anyone over the age of 23 who wants to come and study on a higher level higher education course can get in so we have open education we have flexible progression routes and we're always trying to see within the resources that we have where can we put in more openness how can we open it up in different ways so and as a quick segue if you want to look at more of these possibilities this is a total sidebar I'm involved in an EU funded project open game which is eventually trying to make an actual interactive game to encourage people into open educational practices but some of the early outputs from that from that project include a whole a whole handbook of examples of open educational practice what I'm talking about today is one of them but there are others on assessment there's others on teaching there's all sorts of different ideas in there so opengame-project.eu if you want to go check check that out so what we did I'm probably skipping a bit ahead in the slides what we did was there was an existing there was an existing assignment where students thought about how they might promote to a school a certain educational psychology topic and what Megan saw in my suggestions about how can we open up assessments was she always thought when she saw these assignments some of these would actually make really good pieces of information to go to schools like schools could benefit from some of the work the students were doing but like it didn't have that function so that's what we tried to imbue out of three assignments during the year their second assignment which was focused on developmental psychology we got them to do a more regular assignment where they would research a certain topic like let's say mental health and well-being or something like that from a developmental psychology standpoint and then in their final one they would kind of come at that subject or that topic again from an educational educational psychology standpoint but actually work that up into either an infographic or a pamphlet and a digital pamphlet that we would actually put out in the public eye so this is an example of one of the students sort of posters we let them take a couple of different formats infographic digital pamphlet poster and we actually we then collected these and put them onto a WordPress blog now I think there's different ways we could have done that possibly we could have done it better I know other people were talking about we could have got them to blog it themselves and then federated or pulled all those into one overarching blog but I suppose we were conscious of not wanting to ask the students to do too much so we got them to do it give their permission and then those who gave their permission we put them up on this website we wanted to follow up and we are still following up on how this is going so we did an online focus group after the first year that we did it and we analyzed it using thematic analysis and this is sort of the early results from that first year we have collected data last year as well due to 2020 Covid madness we haven't really gotten to the second year and we have decided that we want to do it again this year before we kind of really push forward and write it up but the results were really interesting in that students they really seem to engage with this as a good idea they recognize this as something novel something engaging they struggled with it a little bit especially those who were maybe a little less used to making making something graphically pleasing and I suppose the first year you could see vague instructions the first year maybe we didn't quite get it right I think we were figuring out how to release this to the public as the assignment was unfolding the first time but the huge thing the thing that I thought was so impactful was it really made them reflect on what they were doing it made them think about producing an assignment in a different way now that it was going to go somewhere suddenly there was all these ideas about how do you communicate to people how does the public get information at what point do I stop being responsible for this being out in the world will someone else take responsibility for this if there's something wrong with it and it really made them think about all these different topics I'm not going to talk through these quotes but you'll just see it gives a flavor of what I was just saying and it got them thinking about in their journey some of them towards being a psychologist or working in fields in psychology it was at what point am I good enough at what point is the information I'm giving out valid or you know can I stand over it or at what point can people get use out of what I'm saying so and also who is the public all this kind of thing like who are we talking to so that's what I would say is we're continuing to use this assignment there's a number of other places we haven't done anything as clear cut in terms of that I could point to and say look there's an open pedagogy assignment but I know that in some of our research methods modules in the psychology major people are using real life data sets rather than some artificially created data set as was that's one that's another form of open assessment and we have some other places where I want to keep pushing to see if we can tweak assignment so that what the students produce could be used can go back into the course as video content or other kind of infographic content and but we're just not it's not quite there it's like it's always about how many innovations can you try and put into each year given the amount you can actually do I think that I got my plug in for open game and everything so I think that's the I'll stop sharing Thanks James some really interesting stuff there and I shared the blog there in the chat and I was just having a look at some of the examples and it's very interesting the effect of having to showcase work on students and how that they had to learn about effective communication and how it had to be polished and well presented I think that obviously changed their way of doing things Yeah they really it really made them think about their work in a way that they never had before you know and that's what came out in the data collection that's it was just fundamentally different the only other assignment compared it to was an organizational psychology assignment where they had to record a video as if they were sort of talking authoritatively on a topic and then they peer review each other but that still closed and that's the assignment I think their scope to actually get them to just do a short piece on topics and then actually use some of them in the course but they thought it was a little bit similar to that totally beyond it because they were actually putting it out they were actually putting it out in the public the only thing about the blog is we haven't even updated it with last year's work yet because of 2020 hashtag damn you 2020 so James just to draw on some of maybe the background stuff that inspired this I was just thinking like Rajiv's work at Cyprof on Twitter might be a nice thing to share with the chat and what's that open cookbook thing they have which I think again there's an open is there an open pedagogy cookbook or something patchbook yeah that's really good you might want to lash them in there but certainly Rajiv at Cyprof on Twitter if you're interested in open pedagogy or open educational resources is well worth a follow I think some of the stuff out of that open game project that I was plugging earlier sometimes I think it can be intimidating if you go to some of the websites that talk about open pedagogy because open pedagogy is a complex intellectual thing to like come out from that angle often it's easier to think about what's going on in classrooms what's going on between you know teachers and students and what can you change by using some of these open educational practices and it's often it might be easier for people to start there and then start to learn how to do that and then they'll arrive at open pedagogy you know it's like if you're using a real life data set you're doing it if you're somehow taking something students are doing if you're getting them to blog and Pete that's kind of public but you're protecting the students and the students understand what they're doing that's open pedagogy you know rather than sort of looking at something where it's like it's a praxis of that's very complicated and some people might go I don't know what's going on here I don't like this it's scary it's very theoretically heavy area but actually when you see your example there of what you can do I think it kind of breaks down some of the theory and they do like to use some heavy language like praxis which is which can be a bit daunting there's a nice question there for you James in the chat I'll put it you know privacy issues around publishing the student work that's a good one so the students did not have to publish it and it's effectively like layer one it's a closed assignment they do their work individually they submit it through Moodle as normal with that then they can opt to put a permission with it that it can be taken and put online I then take everything that has a permission slip and I give it just a once over to make sure that the student has understood what was going on I think the first year there was a student who maybe misinterpreted a tiny bit and was putting official government logos on it as if it was a government piece of information was like okay that cannot go up because that does not have that shouldn't be on there but like so I just do a once over and then the stuff was shared online where the student had permission to do that and I mean a big thing is to ensure that students understand that and it's the big part of open pedagogical assessment is you have to protect the students or make sure the students are making an informed decision you know if students are blogging or vlogging publicly they're putting themselves at risk to a certain level depending on who they are and all that so I think there always has to be a plan B there always has to be an opt out and depending on the depending on the learning outcomes that might be more or less problematic but like you always have to think about that certain people might need to not be out there in the public eye or be putting their name on things or you know there can be sensitivities around that stuff so there always has to be a plan for how you students have a different way of doing things. Very nice point James Sharon is a good point there about how it gives an opportunity to speak about privacy also licensing are they did you create a commons licensing on those? I asked in what I put to the students I said you don't have to do this but if you're giving what I'm asking you to give the permission for is CCBY and I tried to explain that and I mean there's a number of I mean in asking them to produce something that could be shared publicly that required content in their assessment documentation and discussion with the students around what licensing meant because they then can take copyrighted images and put them on there because that's going to create a problem so it's again to come back to that open game one another the first output of that was a competency framework where we tried to figure this out and it is as soon as you start in on any of these activities there are underlying competencies where all of a sudden it's like you do need to know about licensing and copyright and how that works before you can kind of do this you do need to know how to lead students in knowing about that you know so there's this kind of a there's always a bit of a matrix of things you end up figuring out you do need to know about these things and if you're leading students in doing that you need to be able to help them get there substantially before you do it and you're really hitting some of those those skills on that did you work if you think about it too so me students are really developing their knowledge of internet safety licensing privacy so you're hitting a whole load of themes there which is fantastic so now I think thanks James your contribution very interesting discussion around open pedagogy one I'm interested in myself and one I have swam through the the open sea of praxis trying to get a head around but I think nice to see good practical example so I think now we have time for a bit of more questions to the panel so you know put your questions in the chat or the Q&A box for anyone of the panel we might take a look back at some of the previous discussion around assessment to start us off as well I will jump right in for now do it Vlad so I'm for everyone here I'm the diversity here you can figure from my name and from my lack of accent I'm not Irish so I saw a question in the Q&A and this got me thinking we have this discussion with other colleagues about new ways of assessing students and then they say okay but this works for a small group of students what about scaling I saw a question in the Q&A in relation to scaling for e-portfolios but I would extend this question about scaling for various types of assessment activities what are the assessment activities that cannot be scaled if that's the case and how you should work as a teacher to better do the scaling thing for the assessment this is a question for all the panelists one of the things that some of the courses I'm familiar with would have large numbers for our college we would have 90 students in a year that's a lot of portfolios I suppose one to do to scale up is to have a team of assessors the only problem I suppose is inter-rater reliability is one of the things that you do need to have a really robust rubric and a robust interpretation of that rubric because I think sometimes we trod about as some sort of panacea we need to have a good collective idea and I suppose the thing is particularly the more complex I suppose the more moving parts say we're in a portfolio I think that's one of the issues of scalability could be an issue but I think it's too much for one person so rubrics and training to get everybody to sort of have a good sense of what it is that would be my task about scalability just to jump in there as well and to invite my colleagues Rob and Suzanne to join in on this suggestion as well we were in the middle of conducting some research on the swift move of moving a very large class or a number of very large classes online since March 2020 here in Dublin City University and I suppose in that project we surveyed staff and students particularly those in large classes and we define large classes of anything over 100 students and actually we came up with some really clear immediate actions and considerations to take into account I suppose to answer your question about scalability what considerations need to be made I think ultimately what it comes down to principles for good robust effective assessment need to be maintained if we are quite clear on exactly what are the principles that support our learning outcomes we can scale quite effectively but as Tom said it's about tooling up it's about getting the support that we need and to do that effectively and if I may just share my screen very quickly I'm going to actually share with you some quick guidance that we have just published as kind of an interim report and here with our moving large face to face classes online we have some guidance here for HE teachers higher education teachers and I suppose that was contributed to by the data we collected from staff and students here in DCU but also informed by literature and large class teaching and learning and current publications on the swift pivot for COVID-19 so there's some immediate actions and considerations to take into account for assessment and then some ongoing actions and tips and techniques so we will pop that in the chat box for you as well might be useful Thanks Karen, yeah that's a lovely resource as well go ahead and put it in the chat I think I've put nearly all the resources in the chat that have been shared and I think James if you share your slides as well I think everything has been everything has been shared already and it will go up on the EAD website later as well along with the recording so if you've missed anything there's a couple more questions coming in unless anyone wants to ask James or Suzanne or Rob want to come in about the scaling question Yeah I would just say it again it's complicated it depends on your teaching and learning model and it depends on sort of the learning design approach in the modules you know if you have a way of you know layering on extra supports where you need them then you know it's easier to scale to bigger classes I mean it's the same with any assessment we often have this discussion because you know our our adjunct faculty are part time so you can't just design the best assignment ever that requires an academic to be you know working with the students 24-7 for four months or something you know it's just it's just that's not doable that's not scalable so you have to figure out what is what is practical where can the work be put reasonably and on the students with them being supported to do that work you know and to get a good outcome so it depends on a lot of variables but you can kind of see how and how something might be too complicated or not scalable enough but in that case you might want to start small and have like a two-three year plan in year one we'll get them doing you know we'll get them working in groups in year two we'll get them working in groups but they share their work publicly in year three we'll get some help and we'll actually have them do it right in Wikipedia or something you know so you have to move that way slowly I mean I want to get students working in Wikipedia somehow but it scares me still a little bit me too James you know yeah and it's just it's just you know baby steps and then you know I'm nearly ready for the year three one of my Wikipedia assignments I have them to building wikis in Moodle and the next phase is to to go into Wikipedia so yeah I think I think you're a point there about doing it in phases you know it is a good a good a good one because change is hard and it takes a lot of time and energy to think about how to do things how to design new things how to show the students do things so so I think you have break it into baby steps add a layer each time you do it makes makes perfect sense and obviously the scale thing with ePortfolio in particular and give the students very very specific limits tell them I want 500 words and three images and no more because Portfolio in particular can just go mental people can just show the world and it's wife in there and also I always say I don't want to see a picture in here unless it's relevant you know but I do enjoy the odd cat picture or dog picture you know they're always relevant and Orna I might just on that point go ahead Rob of taking things in phases and so on I think an important thing to consider as well as involving students in the assessment process in DC one of the projects we're involved in at the moment is students as partners in assessment looking at ways that we can bring students into the assessment process as well and give them a bit of agency and this can really help with scale if you start looking at ways students can self assess and peer assess and provide constructive feedback and support one another in the assessment process a that can create some efficiencies for staff in terms of grading and management because you're getting students involved but it's also fantastic for the learner's own agency and their own ownership of the assessment process it can help them learn it can help them feel involved and give them a voice as well so I'd be a strong advocate for partnering with students and involving them in the assessment I think it's good for a lot of things including including scale we'll have a forthcoming resource available on that in the next few weeks so be sure to keep an eye out for that on our website thanks for my band and Sharon I think may have shared a version of that or is that an early version Sharon has shared there with the assessment that was something similar yes that's from the EDTL project excellent sweet I just propelling the University of Edinburgh has a wiki medium in residence and there's some great projects coming out of what they're doing the University of Edinburgh there so I I've certainly come around to I think the whole idea of the co-creation wikipedia don't actually talk with Sharon Flynn there it's been a great session that she hosted a few years ago in NYG, shout out I shout out there Tom a wikimedia Ireland Sharon I'll have to pick your brain there's a lovely project I like talking about co-creation on wikipedia called Women in Red because like 70% of the entries on wikipedia are about men and so to redress the balance is this great great project so that's my idea is to go get the students to try and rebalance the red Suzanne I think you want to come in and answer the question there on the Q&A so examples of art music performance composition any examples of e-portfolios yeah actually in the faculty of education we have a group of users who have engaged with e-portfolios as part of an assessment and they're actually on my list to contribute to our open education resource so what I might do is just contact them try and encourage them to add it in they were they were open to it but I just need to follow up on that so that would be an example that would go into our open education resource and just in terms of scaling as well while I'm here one of our assessments on the BED program the bachelor in education program which is 450 students use an e-portfolio assessment and again in terms of scaling like Tom mentioned already they have a team of assessors and those assessors use a combination of rubrics but the interpretation of the rubrics is a very important piece of that assessment process so it is possible but yes needs some careful planning a time and resources is a man power or woman power as you say Vlad come on in thank you for the question there yeah sure actually Rob raised the ball for me with the discussion about involving students in the assessment process and I wanted to quickly ask our panelists what was their perspective about involving students in the assessment process and also in the decision about the type of assessment that we should use so should we involve students in the process of assessing and should we ask them how they would like to be assessed something like this is my question because I know that we have also students here with us yeah can I come in on that Vlad thanks for that question we had a discussion on learning day last year at DCU we had a student panel who spoke about flexible assessment and a really important point that has stayed with me from that student panel was that yes they embrace the choice available when choices offered around assessment but that too much choice there's a balance there too much choice is actually overwhelming and a certain level of choice is very much seen as a positive by students and I'd be really interested to hear a student point of view today so I think we need to kind of consider that balance of yes definitely offering choice but not to get to the point of being overwhelming because it's just it again it will just kind of tip that balance of the flexibility offered if we go too far on that as well Suzanne I know earlier on in the week Sharon Flynn here coordinated a webinar with various student interns of the EDTL project who are all giving their perspective on assessment and I'll pop the link in the chat to the recording and I can find it but very much a lot of that was coming up as Suzanne was saying students do want choice but it needs to be there needs to be some structure on it they do want creative and engaging assessments because they enjoy that more and they do want maybe some kind of practical scaffolding and practical support around managing and tackling assessments and things like that and anytime we speak to our student intern about issues around assessment it is so illuminating to get their insight around how they're getting on and what they're finding difficult and what support they need from us so I think the more we can talk to students and listen to them and take on board their advice and get them involved in processes it's a win-win situation because it's good for us and it's good for them that's very inspiring Rob I have to say but I totally agree there was one very interesting thing there in the chat and Karen I think you had I like how you say a bounded choice so you know a choice with some limits on it but I totally agree too much choice is overwhelming so it's that delicate balance this is worse than feeling like you're out at sea as teaching staff we often feel that quite a lot but our students know when choice is presented to them it needs to be structured, it needs to be bounded and it needs to be sensible providing a choice of assessment type of assessment or mode of assessment or even something like giving our students the opportunities to choose their own groups or to choose their deadline for submission of assessment unless we bound to that and we structure that in a reasonable way it can actually become very unmanageable for our students so there's just small considerations that we need to do to ensure that while choice is offered and freedom and flexibility is offered it needs to be met with a little bit of structure and a little bit of organisation from our end to ensure that we have the best outcomes for our students I think that's a point that goes across a lot of trying to open up things sometimes you have to make sure that that still seems structured it's already been said but if it's too much just like let's explore this together students just feel like they're adrift and they don't know what to do and they don't know often I think we forget what it's like to be an early undergraduate and it's like you seriously need to tell me what's happening here because right now I just feel like you're telling me to go figure it out myself entirely they need that structure please tell me what to do there was one thing in the chat there which I thought was very interesting from Naveen consider using Ipsat of grading that's comparing two things and deciding which one you prefer I've never come across that for ePortfolio but I think it could be very interesting because then you have your idea of what's quality and then you compare does anyone want to come in about Ipsat of assessment or evaluation there or not I had to look at I googled it as well Tom so I actually used Wikipedia to look at them on the example of the music portfolio I have an exemplar here from a student I might show quickly so from DCU so the student has given permission to share this okay so don't worry I'm not breaking rules there so here's our ePortfolio platform here loop and one thing one of our colleagues Lisa Donaldson does is provide a lot of exemplars of student work because obviously for both the teaching point of view but also from the student point of view seeing what it actually looks like really helps so there's a nice one here I can put this example in the chat as well from a music specialism and you can have a look through it but what I like about it is okay there's a picture of her class and you can see about this particular methodology the codeline method and you can see she's linked to her own development she's a video of herself using some of the methodology she then puts it into practice in her teaching practice and you can see she's got further evidence of that so I think it's a lovely example of an ePortfolio but also a lovely example of a music one so I'm just going to share that now into the chat if people would like to so we're kind of coming to the end of our session I really like to thank everyone for their contributions you've been a hugely engaging group I'm just going to hand over to me partner in crime Vlad to conclude thank you Orna well I don't have much to conclude because this is still a very open and flowing subject for me and I think for all of us because we still don't have the exact answers and the correct answers and we still need to do a lot of work but I hope that for everyone here today a lot of good examples were shown and you can put them into good use and then come back to us and tell us how it went thank you everyone for being here during this webinar and don't forget Eden is still having until next Monday a lot of interesting webinars thank you for all panelists and for all the attendees and I wish you a very nice day thanks Vlad and I think if you're interested in attending the next Eden session which is on this afternoon it's called the next normal including student views at five o'clock Central European so the details on the Eden website