 for everyone, so I think we can start this session of the European Distance Learning Week organized by Eden. Today we have a session on future perspectives in open route, the quality and attachment dimension. This session is offered in the framework of the Eden Network of Academics and Professionals that I hope you all know but that if you don't you will join very very soon within the network of academics and professionals which I chair. We try to use social media channels to improve interaction of professionals. We support professional development. We offer different kind of webinars on themes of shared interests and we try to listen to members' ideas and we try to support new groups, new research networks. But this was just to introduce our session today and to tell you that besides me, as I told you I chair the Network for Academics and Professionals Theory Committee. I come from the University of Roma TRE and so from Italy. Here you have the list of speakers today. First of all, Eva Oceaniston, Eden Executive Committee member, Eden Seagantel Group Chair, Eden Fallo, ICD OER Advocacy Committee Chair and ICD Ambassador for the Global Advocacy of OER. Also Daniel Ehlers from the Baden-Wuttenberg Cooperative State University, as well as Eden Executive Committee member and Eden Fallo, Dr. Don Orcott Jr., Rebel Consultant, Eden NAPS FC member, in fact Don is a very active member in the Steering Committee from the NAPS, Eden Council of Fellows Vice Chair. Francesca Mendoi, PhD student, University of Roma TRE, Eden NAPS member and at the moment developing period as a PhD visiting scholar at Princeton ETF. So she's connecting from the UN. I will just give you some brief instructions for this session. Each presenter, each speaker has about nine, ten minutes for each presentation. You can write down comments in the chat area and questions as well. Microphones will not be activated for participants and the webinar will be, of course, recorded and accessible at the Eden website afterwards. I don't want to steal time to our speakers today, so I just leave you here some information regarding myself. So if you want to contact me, you have all the details here and I leave the floor immediately to Abba Oceania Son and our presentation on future perspective in open room. Thank you Abba for being with us and the floor is yours. Thank you. So thank you very much, Antonella, for your introduction and for the presentation of me as well. Thanks for the invitation being here today. Can I have, let me see. Yes, I've got my slides. Oops, they went away. So yes, thank you. So the topic for this session webinar today is about future perspective in open routes, the quality and assessment dimension. And especially as you have seen for the brief introduction for this webinar, it is mainly focusing on OER, open education resources, and I have also chosen to primarily talk about that. The slides are coming and going, what is happening. So yes, I'm Professor in Innovation and Open Online Learning and as I'm speaking about OER today, I will also say to you that I am leading the ICD OER Advocacy Committee. The reason why I've chosen to talk about OER today is because we soon will have UNESCO, United Nations, OER recommendations coming out. But first of all, although I have chosen to especially focus on that, I will strongly argue that for the quality agenda for open online learning, there are strong demands and needs to redefine the quality agenda. Because there are so many things that are rapidly changing in our field. So we can't any longer talk about quality as we did just some years ago or even last year. So there is a huge need and urgent activities and missions to redefine the quality agenda in that way to take more the learners perspective, for example, to look more at impact on the individual level, at the institutional level, also for the impact education has for society, for citizenships, etc. And also such areas of satisfaction and engagement and the impact for the individuals, as I said. So I have chosen to not speak so much about that today as many of you have heard me talk about that also before and that is really in my heart. So we are saying that when I will continue with more focusing on OER. And first saying that I'm also sharing the Eden Special Interest Group on Technology-Enabled Learning, TELL and Quality Enhancement and of course OER is involved in that topic. And we have had this special interest group for some years by now. We are working both internally but also externally. Internally I mean with the Eden and Eden members and together with Eden that and some webinars we are organizing together as this one, for example. And also we will have two upcoming in spring. We have a special page of the Eden web page where you can find more information. We are some 10 people in the core group, Ulféles Daniela, who is here today as well. He is also in our core group as well as Antonella as a member. So externally we are also trying to have a voice with a quality agenda as such, where it is appropriate. For example with OER recommendations is one of the issues. Focusing on OER, you may know that UNESCO definition was redefined at the UNESCO meeting in Paris in May. And to read like this, there are some small changes as you may notice. Learning, teaching and research material in any format and medium that resides in the public domain or are under the copyright that has been released under an open license that permits no cost access, reuse, repurpose, adaptation and redistribution by others. Together with this redefinition, there was also a redefinition for creative comments, which you maybe have seen, also slightly modified. The work on the forthcoming OER recommendations are of course based on the Cape Town Open Education Declaration, the 10th anniversary. And more precisely, that means that talking about OER is not that we are talking about just the resources, the materials as such, that's also the whole area of openness and open practice and open culture. And the Cape Town Declaration also talked about communication open, empowering the next generation, thinking outside the institution, data and analytics, how that are used, beyond the textbooks. Quite often, many people are even among our community, when they are talking about OER, they are mainly thinking about open textbooks. So you see that it's a much wider area. And I think that is very, very important to strengthen that, that was one of the reasons why I wanted to talk about this today. Connecting with other openings, like open access, for example, open education, just to mention something, open education for development, open pedagogy is a huge area. And if we're really allowed to take OER beyond the textbooks, there are strong needs to go for open pedagogy. And that has been a very common concept also nowadays. I think you have heard about it many times before. Opening up public funded resources and of course the copyright reform for education. And then maybe there are some x cards as well, which are not so explicit. So you see, you need to see this ecosystem where OER is just one part. And also to reach the SDG codes number four, it is said that the use of open education and OER is the only way to reach the SDG for education for by 2030. I don't know what's happened with the slides. Sorry for this. I would say that for those recommendations, there are five main recommendations and that is about being the capacity of stakeholders to find, reuse, create and share OER, develop supporting supportive policies, ensure inclusive and equitable access to quality OER, nature of the creation of sustainable models for OER, and facilitate international cooperation. So those are the five main recommendations which will be taken next week. So I will then just finalize about a report which was come out two weeks ago, I think it was from Commonwealth of Learning about guidelines for development of open educational resource policies for you while working in this area. So with that, I will finalize for now. Thank you. Thank you so much, Heba, for your presentation. I really hope we will have time for for discussing more about the input that you gave us with your presentation, especially on the Eden chat that we will have a fix later. But let me pass the floor to Wu Wu. He is, as I told you, from the Ben and Wuttenberg Cooperative State University. Eden Executive Committee member and Eden fellow, please work the floor. You as well, you know, all about nine, 10 minutes each. So please. Hello, everybody. Good afternoon. I hope that you can hear me. Please let me know in the chat if that's not the case. I would like to thank Heba very much for her talk and would like to continue with a little bit of a complementary topic. I would like to focus on the assessment dimension, actually. And on the question, how can in times of open resources and open learning, open pedagogies assessment look like, actually? The sources, which I'm presenting to you, the content, some of it is taken from two books, which we have just in the last years worked on. One is called Open Learning Cultures, A Guide to Quality Evaluation and Assessment. It kind of fits to our webinar today. And the other one is called Changing Cultures in Higher Education. It's about the culture change through technology and openness. So, first of all, when we think about assessment, there are lots of different assessment forms and methodologies. And there are also lots of different assessment purposes. And I think it is important to keep in mind this diversity of activities, which we are employing and the diversity of purposes also, which we are using to when we talk about assessment. Assessment for, for example, personal purposes could be just to see, where am I as a learner standing in my learning progression? Assessment on an institutional level could be to have learners passing certain exams so that they can enter from one area of their studies into another area of their studies. And then, of course, also there is an assessment on the institutional level where you are assessing certain and accrediting certain programs. So, there is a wealth of assessment purposes and forms. And that's important to keep in mind because also assessment is something we can, some of the assessment forms we can use better for open assessments and some of them we can use better for closed assessments. I've tried to work on the issue of open educational practices. Open educational practices is what we define as using technology first and secondly allowing students to choose in an open way their learning pathways and their learning objectives. Whereas we teachers, we are just alongside them and trying to support them. So, this is how we define open education practices. And when you think about now open education practices, how can the assessment look like and how would it look in closed environments? Closed environments are environments where teachers are determining the learning pathways and teachers are determining the objectives of learning. Maybe also where little technology, little open educational resources and little open education is used. So, when you try to group the assessment forms, you can see that for open education practices assessment which gives the autonomy of judgment and the activity in the hands of the learner very much. So, for example, formative assessments of own learning, the assessment for learning, which means you're using the. I think there's a problem with the connection with Wolf. I don't know if, yes, I see everyone is typing on the problem. We will try to understand if Wolf can restart his connection, otherwise don't see any reaction. But of course this might happen. The, as you can see, all the presentations are focusing on the topic from different point of views. And this is that is really useful to have a wide picture of the subject we are trying to address today. And that's also the video is not working anymore. I don't know if he's trying to connect again. I think so. Now I see he is there. Okay. Here you are. So, sorry about that. I'm just continuing. So, this is just a list of different assessment activities and you can see which ones are more suited for open and more suited for closed. And when you think about what are we actually assessing when we are in open environments, you can see that these are also different objects and activities we are looking at. The focus is different actually. The focus of open assessments and open environments is much more on the participation process, for example. We are looking much more at how do actually learners interact? How do they participate in the learning? How do they maybe create artifacts? Okay. So, we are looking at learner created content. We are trying to understand the personal learning environment. Learners have been building around them and the quality is often not assessed through teachers or through experts, through the lens of an external, let's say elevated expert, but it is rather assessed through peers. All in all, what is becoming more and more popular is actually to see that whereas in the traditional assessment philosophies, we are separating learning process and assessment, whereas the assessment takes some kind of evaluation or proof of learning step in the end of the learning process. Whereas in open learning environments, the assessment is often not automatically, but can be embedded actually in form of peer interactions and form of self-evaluation, formative assessments throughout the learning process as an embedded activity, which is informing the autonomous learner about the learning process. If we only look at the question which kind of assessment methodologies are suitable, are suited, are actually carrying this kind of characteristics that they are formative, that they are embedded into the learning process, we can see that these, all these different assessment methodologies are suitable for that. That means that there is a wealth of assessment methodologies actually out there. For example, self-evaluation, which is, if you look at it from the original concept, it is a very elaborated concept, a very elaborated methodology where learners are creating in cooperation with teachers criteria and then apply these criteria to their own learning and then try to derive from that development aspect for their own learning and then share that with the other learners and the teachers. Self-assessment, responsive evaluation, formative evaluation, I mentioned them already, but also all these social and community participatory assessment forms like peer review, peer reflection, peer assist also, where you are together solving problems and assisting yourself in your own learning problems, where you form a learning group, peer learning, bench learning, where you try to understand how is my own learning ability compared to the learning ability of another learning group. These are all requiring, of course, very, very high reflection levels of your own learning, of the learning of learners and are there for something which is suitable for learners who have a high autonomy. When we now think also, that's my coming to the end, my last slides, think about the future, where in the future soft skills, a new kind of skills, we call them future skills will be much more important, where autonomous learning, self-responsive learning will be a much more important step. Then in open learning environments, open learning assessment forms will gain a lot of relevance and not only that, we have created scenarios in this research. I show you the report again, which you can freely download at nextskills.org, which talks about the next 10, 15 years of higher education development and the future skills in this. In this report, you find also four scenarios. You can see also that open assessment can also or has also an additional component, which is the component of micro-certification of certifying small bits and pieces of learning in a way that they are documented in an understandable and evidence-based way, so that in universities in the future, how you can see here, students will follow not just a study path in one university, but they will follow a study path in several universities as lifelong learners in a very personalized way. This will also be an important aspect of assessment, leading to small bits and pieces of certifications so that an autonomous learning biography can be in a way generated and taken by the learner with themselves, so that it really forms part of their own learning biography. So that's from my side. Thank you very much. And I pass back to Antonella. Break of connection. The different presentations today really fits one with the other in composing a wide picture of the focus we are carrying out. So thank you so much. I wrote down some notes and I wish there would be the time to discuss some of the issues you proposed. But as I said, on the Eden chat, we will have time also to remind and to recall different instances that came out of our presentation. Again, I don't want to steal time to the next speaker. And so I pass the floor to Don Alcott, level consultant at the member Eden Council of Fellows Vice Chair. Thank you, Don, for being with us. And the floor is yours. Good afternoon, everybody. Thanks, Eva, and thanks, Ulf, and thanks Antonella for your interesting and insightful comments. As Ulf said, he was going to bring a complementary view to our topic today, and I'm hoping to do the same. I want to talk a little bit about all these great resources and ideas we have. How do you make it happen? How do you make it happen in your institution? What I'm going to talk a little bit about today is leadership, how to navigate the culture in which we work, and how do you manage change? Now, we all bring a variety of experiences around the issues of quality to our work, and these are four that stand out to me that have sort of struck me over the course of my career. Language matters. How we define quality and the context in which that quality operates often is really important in terms of our conversations and our discussions. I don't know about you, but oftentimes I found I was talking at cross-purposes with my colleagues about what we understood to be quality, and I think this leads into the second one. Where one stands on quality is influenced by where one sits. I can assure you if you ask your Vice Chancellor or President, or you ask your Dean of Engineering, or you ask your Department Chair of Music, if you ask a student, an employer, an alumni member, a member of the Board of Trustees, they are all going to be representative of a major stakeholder group to make up your university, and hence they're going to bring some different viewpoints about what constitutes quality. The gatekeeper of quality is the institution. I want to first and foremost say the oversight agencies and accrediting groups play a marvelous role in helping us as institutions stay focused on those quality issues, particularly in an era of open and distance learning where we have new processes, procedures, and methodologies that are new to the accrediting group as well. But my experience over the years is at the end of the day, it's up to the institution to really take the responsibility for assuring quality in all its guises. The last thing I want to say in this particular area is quality and open content must be core values of the institutional culture. We've made incredible progress in OERs over the last 10 years, but I would also say we haven't made as much progress as we hope we would. And I think part of the reason for that, that the adoption rate hasn't been as vast as maybe many of us would have hoped for, is we have not been able to embed it into what are the core values and the central culture of the institution. Now when we talk about quality, what do we mean? Again, this is my slant and perspective only. You bring your own to this. I only share these as an opportunity for you to reflect on what it means to you. Okay, do we normally associate rigor and high expectations with quality? Yeah, I think we tend to do that. That doesn't mean by having rigor and qualifications, or excuse me, the rigor and the high expectations will necessarily result in an outcome that's useful. Ulf was talking about soft skills. I can teach students rigorous and have high expectations, but when they walk into the door to their employers, they didn't learn the right skills. So even though we had rigor, we had high expectations, we taught them the wrong things. Review. This one is really important for OERs and for setting quality standards for open educational resources. We must engage the faculty as an essential stakeholder in that review process. We are doing that to some extent. I think we need to make a greater effort to do that. Relevance. Yes, quality is tied to relevance. Again, back to what is real world skills and knowledge that our students should be learning. Some areas of quality or what we define as quality can actually be linked to responsiveness. When we think about student services for open and distance learning, the time of the response, the type of service, the agility to respond to student and faculty needs, all ties in with responsiveness. And finally, renewal. This is the one that I find to be actually quite interesting. The bar changes usually when we're not paying attention. Let me give you an example. Your institution goes from 10,000 students to 50,000 students. In that process, and not because of malice, that's a good thing. It creates access. It's the mantra of open education. Open the doors to everybody. But in the process of doing that, you have now created a system that is going to put a great stress on the student services of your institution. Can you gear up fast enough and quick enough? Some of the issues that are facing and have faced open institutions that serve unbelievably high numbers. If it's a catch-up game all the way, they're trying to get those services back into place so that they can serve that many students. So even though we think of renewal or setting the bar as typically going up, sometimes, even though not our intention, our efforts, our decisions are somewhat flawed. And in fact, we reduce the bar. We reduce the level of quality. Many of you know Professor Rory McCrell from Athenbasket. Rory and I go back way too long. I'm not going to tell you our age, but nonetheless, a long time. And I always like to ask Rory very direct questions because I can always be confident. He's going to give me the most direct answer on the planet. He doesn't pull any punches. He says things exactly what he needs. Here's what he had to say when I asked him about quality related to open educational resources. I've highlighted what I think could be some of the interesting aspects of his response. He would suggest that the licensing agreements, creative commons that we have with OERs, need to be pushed more. He believes that the advantages of those license is not fully understood. Remember, I'm sharing what he said. I'm not agreeing or disagreeing necessarily. Sort of what I said earlier, depending on where you stand on quality is often influenced by where you sit. Quality assessments is quality for one, does not mean quality for all. But he also goes on to say what we were talking about earlier and what I mentioned was, on the other hand, a good measure is that if a number of qualified academics attest or vet the quality and or respected institutions, then that could be a good indicator as you can get. I like this because it links a little bit to a lot of the words that EBA and EBA or OTH have also done around benchmarking. In other words, comparisons with other institutions and other units on what constitutes quality in OERs. Now, everybody needs a good cartoon and I love this one. Can you read what it says? It says, for a fair selection, everyone's going to take the same exam. Please climb that tree. Now, a picture is worth a thousand words. This is talking about students and I remind myself and you that every time I teach, you know what this reminds me of? It reminds me that I cannot individualize instruction, teaching and learning and assessment for every one of my students. I wish I could. I wish I was able to do that, but I don't have the time, the money or the resources to support that kind of endeavor. You can take this picture by the way and it could be faculty, it could be any of your stakeholder groups. Incidentally, a little humor goes a long ways too with the challenging work that we all have. Where are we in 2020? If you look back over the last 20 years, this is where I think we are. You may agree, you may disagree. Overall, in the last 20 years, I think the overall quality of online, face-to-face and any other ODL instruction, video-based, audio conferencing has improved. I think our faculties play a greater role in the various aspects of online, just as they do with and what we want them to do with open educational resources. And finally, I think one of the really interesting outcomes, though we do need more data to support this, is that distance teaching and face-to-face teaching now are mutually reinforcing. That means they inform and support each other. They make each other better. And I think that's pretty cool. I've had faculty members say to me, I'm a better classroom teacher today because of my expertise in my experience teaching at a distance. To sum up and lastly, what kinds of things over the next 10 years, maybe at the institutional level, might come to the forefront. They may be there right now and a couple of these are. One of the things and one of the article I read recently I thought was very insightful. And it has to do with the concept of our adjunct faculties becoming a quality indicator for the institution over the next decade. Institutions have moved to using more adjuncts on an average at least 50%. In some sectors, as much as 80% of the teaching staff are adjunct faculty. The other feedback we have about it is our institutions don't necessarily have a good rewards, incentives, and treat these adjunct faculty very well. Will this become a quality indicator that institutions will have to pay more attention to? I don't know. The quality and scope of student services, this again goes back particularly in instances where you have accelerated growth. Can you keep pace? Will that become even increasingly more important over the next decade? And I think tied into skills development aligned with what we've had to say earlier. The number of student graduates is an imaginary key performance indicator. The real indicator is how many graduates you have that can get jobs. Presidents love to talk about how many people they have graduated. They don't ever tell you that a good majority of them can't find work. So I think we're going to see that continue to be a valuable indicator. That's the end of my comments again. I think that my only concluding comment would be good ideas, good innovations get pushed aside every day. Not because they're not good innovations, not because they don't have great potential, but because we don't know how to introduce them, manage them, lead them, navigate them, and manage change with them within our organizations. I encourage you to spend as much time on that and that process as you move forward. Thank you very much. Thanks to you, Dawn. Again, very inspiring presentation. I have lots of notes here. I don't know if there will be the time to discuss it anyway. We will start and we collect ideas as you said, as you prompted us to do, and we will keep on talking about that. Thank you so much. So now we have Francesca Menduni directly from ETS Princeton. And again, Francesca's presentation is related to a project and to some result of a project of an international European project devoted to the assessment of quality in an open virtual mobility MOOC. So in this case, we will have an empirical instance of what the group tried to do within the subject we are trying to focus on today. So Francesca, the floor is yours. Thank you for the introduction. I'm very happy to present empirical results of this deep theoretical background. I wanted to ask all the participants to write also what is your connection with open education and quality, and we will be happy to read your answers after the presentation. Today, I would like to show you some preliminary results of our pilot phase in which we are trying to assess the quality of a virtual mobility MOOC, massive open online course. The MOOC is composed by Open Educational Resources. The project Open Virtual Mobility Erasmus Plus was founded by the European Erasmus Plus program. The main idea is to promote virtual mobility in European higher education. I don't know if you are familiar with the concept of virtual mobility, but virtual mobility is a concept complementary to physical mobility. The idea is that people that cannot access to physical mobility for economic reasons or for other kinds of problems could more easily access to virtual mobility. The project is organized in seven intellectual outputs, and the group of the University of Roma Tre, coordinated by Antonella Poccia, is responsible for the output system that concerns open educational resources, massive open online course, and the pilot phase. The aim of the pilot phase is to assess the quality of the virtual mobility MOOC. The pilot phase is aimed not only to assess the MOOC, but all the features that are that compose the MOOC. In particular, our matching tool and a group of formation, e-assessment that are quizzes, e-portfolio, and field assessment, gamification and badges, and learning material and course structures organized in forms of open educational resources. The theoretical principle of our pilot phase are design-based research, and they add the model proposed in the quality framework of the project in the output seven, and the pilot was organized in three phases. The first one, we carry out the first phase, first pre-pilot phase between December and January 2019, and the results of the pre-pilot phase were presented after the last Eden conference in a poster. We are now carrying out the first pilot phase cycle between September and December 2019, and we will carry out the second pilot phase cycle between January and May 2020. In order to assess the quality of the MOOC, we designed a questionnaire that is organized mainly in six sections. The first section is concerns personal details, such as age, gender, affiliation and role. The second section regards general questions on the MOOC. The third section is about badges and gamification. The fourth section is about technical aspects of the class four. The fifth section regards questions specific to the pre-levels of each MOOC. Each MOOC is organized in pre-levels of difficulties, foundation, intermediate and advanced. Today, I will not present these results because, as I said, our pilot phase is in progress, so we will present the results for the four initial sections of the questionnaire. For a section from one to six, participants are required to express the level of agreement with the participants regarding the MOOC on a legal scale from one strongly disaggregated to five totally agree. At the moment, the MOOC with most participants was the active self-regulated learning MOOC with 47 participants. The second mini MOOC most attended was media and digital literacy MOOC with 23.6% of participants, followed by intercultural skills, open-mindedness, open-educational virtual mobility, networked learning, autonomy-driven learning and collaborative learning. These mini MOOCs were designed according to the skills necessary to be involved in a virtual mobility experience, and these skills were identified in the output of one of the projects. If you are interested to know more about it, you can find all the information on the website of the project. These results were calculated one week ago when we had 250 participants, but I have just checked and we have arrived to 353 participants, so fortunately the number of participants is growing and growing. The Havera J.E.R. War is 24 years, and at the moment participants to the MOOC are mainly students, university students, and a small percentage are teachers. Despite most of the participants come from a partner university, we have also a small participation from external institutions, so this is a positive result that the MOOC is followed also by external institutions to the partnership. The level most assessed was the foundation level with 95% of answers. Intermediate and advanced level were assessed less compared to the foundation level, and this may be because for intermediate and advanced level it is necessary to activate tutoring because there are activities such as group formation and peer assessment that need to be configured manually by tutors. On the other hand, the foundation level could be followed without tutoring. Regarding technical aspects of the MOOC, we see that participants expressed general positive evaluation. In particular, the length of the video was considered good, the sound of the video was clear, and the technological environment is considered friendly and usable. A less positive agreement regarding these two statements, MOOC layout positively affects my learning experience and it is easy to use, with other age lower than 3.65 but still higher than 3, so it's still not negative. Regarding statements concerning the MOOC, participants stand to disagree with negative statements regarding the MOOC that you find at the bottom of this picture. For example, the negative statements were the amount of knowledge to acquire is too much high and participants were not disagree with these statements mainly, and some information was taken for granted and not well explained. Also in this case, participants stand to disagree. On the other hand, participants stand to agree with positive statements regarding the MOOC. In particular, participants appreciate the use of different kinds of content. In this way, we had also an indirect information regarding the quality of open educational resources. Regarding the budget and identification, participants expressed general positive evaluation, in particular toward the badge design. The next phases are, since the sample will be enough consistent, more than 350 participants, we are going to carry out more elaborated statistical analysis, not only descriptive analysis after the end of the pilot cycle in December. For example, we will try to answer to this research question. Are there any statistical differences among scores for each MOOC? Any statistical differences among scores within a MOOC for each level? We will collect feedback to implement the MOOC before the second pilot phase, and we will use this data to implement the quality of the MOOC, and we will collect data after the end of the second pilot cycle, and we will compare the answers from the first cycle to the answer from the second cycle in order to see whether the quality perceived by participants is improved. Thank you for your attention. Please, if you want, visit our learning app, and if you want, you can subscribe to one of our MOOCs. As I said, the foundation level can be followed without tutoring, but if you want to follow intermediate and advanced levels, please write us. We will implement everything in order to allow you to follow the other MOOCs, and you can write to me at Juan Canella Pocha in order to ask information regarding the pilot phase, if you want to provide feedback, or if you want more information about it. Thank you again for your attention. Thank you so much for your presentation, for the summary and the details of the data you presented. So, an example of how we could interpret quality assessment, and because of course we're, unfortunately, we're running out of time, I would like to ask each of you, starting from Abba or Dawn, I mean, the order that we followed, just one question, the same question to all of you, and I get the input from Dawn's presentation. He said that whatever stakeholder we could meet and ask them, what do you think quality is? We would get different perspectives, different interpretations. So, to each of you, starting from Abba, can you define quality in open route, which is your favorite interpretation? Well, the simple and easy answer is quality is in the eye of the beholder. Then we all know that quality is a very complex phenomenon, and as I said in the beginning of my speech as well, we really need to redefine the quality agenda, because there are so many other kind of aspects, issues, dimensions to take into account in the digital area, which we are in right now. Like more about satisfaction, engagement, impact for all the stakeholders. I see, I see. So, this is the short answer. Yeah, yeah, the main concept there. And what about you, Hal? I think that research is, when we look at the papers published, then we can see that the locus of control of quality in open and personalized learning environment is shifting towards the individual learners very, very much. And that is interesting, and that is also a challenge, because in such cases we have less and less external objective standards. However, when learning is individualizing, more and more it is of course clear that the learner is much more in the driver's seat. And that is the challenge for institutions to define open method of quality development and of assessment. And that is how my presentation is fitting in there, because it is challenging still today for institutions to adopt these kind of open assessment methodologies. Thank you. Thank you, Erf, for your comment. And I know you already talked about that interpretation, but we want a final comment. I guess I'd make two comments, and I have, you know, I have to agree with Eva. You know, the best answer is not the one that says it depends, but the reality is that sometimes that is the answer. I guess what I would say is quality with a little Q and quality with a large Q. And I think at the minimum, it's a minimum level of performance as defined by the creators of the knowledge and the users of the knowledge. Now, that could be a skill. It could be a soft skill, a hard skill. It doesn't matter. But I think we're talking about some minimum level of performance. Now, people who promote competency-based education will tell you that too. But I think at the very minimum, it has to be something like that. Just a footnote to that. Do you know that in 40 years in this profession, I've never heard one person, not one person say that the standard of face-to-face classroom instruction was a good measure of quality. And that's what open and distance learning has had to live up to for the last 40 years. We were placed at a disadvantage. I once had a president say to me, you have to make sure that the quality of teaching in distance mode is commensurate with what is going on in the regular classroom on our campus. And I said, Mr. President, if you knew what was going on in the regular classroom on our campus, you probably wouldn't say that. So I guess, you know, again, once again, the culture in which you work and the norms of that culture play a great role in determining our flexibility in setting those standards. So I'll stop there. Sorry, I had a problem with my mic again. I don't know why. But anyway, thank you, Don. It was very effective. Your comments were very effective. The last word is to Francesca. I agree with what the other speaker said before. And I want only to add one thing, data, data and data. We need to integrate data from different sources. And I think that learning analytics will help so much in the context of quality assessment. Thank you, Francesca. In fact, the idea of using those data in a critical way in order to improve the system, the offer, the teaching and learning dimension we are all engaged in is actually the key word. So I thank you all for the very inspiring webinar we carried out today. We will keep on discussing on Twitter at 6. So don't miss our tweet chat at 6 where we will try to summarize what we have been saying during the webinar. As I said at the beginning, this webinar is recorded as all the others offered within the European Distance Learning Week, but also all the others that we offer as using that network of academics and professionals. So don't miss our offer. Go to our website, disseminate our work and we'll be in touch on Twitter in a while. Thank you all for being with us.