 Okay, I think that was a good time as any to start today's webinar. So once again, thank you very much for attending and the topic we're going to be looking at today are insights from an internationally co-developed MOOC on digital citizenship. I will be moderating this session and I have three colleagues and friends with me. Firstly, Fabio Nascimbeni, who's a senior expert from Uni Made in Italy, Christina Stefanielli, who's a project manager also from Uni Made, and finally Beatriz Cidano, who's a research associate at UNED by institution as well. Okay, so what I'm going to try and do now is share my screen with you, which I'll do like that, and then I can swap over to our presentation. So you should be able to see our presentation. Okay, insights from an internationally co-developed MOOC on digital citizenship. As I mentioned at the beginning, an awful lot possibly too much has been said and written on MOOCs and what they actually mean for online education, access to open education resources and a whole range of related subjects. The reason that today we decided to focus on this particular topic was actually motivated from an Erasmus Plus project that we're actually running at the moment, the Nexus project, which is running from 2019 to 2022, and in this project we're actually trying to enable particularly migrants and younger people to improve their digital citizenship, become more active and that's typically more online these days than in a face-to-face context and we're trying to do this by reaching them firstly with the MOOC as part of the training, although if the pandemic actually allows us we hope to have some face-to-face training later on. We're also reviewing the role of service learning in the community and how to really engage with this social collective and then looking at the digital tools that can be used for this particular activity. So if you looked at the agenda again, I'm doing the first point at the moment. I'm introducing the session for you, trying to motivate the webinar because essentially what happened is as we were working together as an international team on the MOOC for this particular project is when it became evident that we really had some things to say about this whole developmental process. And then my colleague Fabio will actually take more of a critical view of how MOOCs are typically developed and contrast between the different aspects that Bayer will take over and actually focus on how we did it in this particular project. Christina will present the results of our MOOC so far, although it's in the final stage of being developed and where I will be releasing it in April. So that's something for you to look forward to. And then I'll contextualize this process because I used to be in charge of the Open UNED MOOC platform and project. So when we kick this off in 2012, I've got a different perspective than how we've done it and maybe can make some interesting comparatives there. And then at the end, we'll have a collaborative activity. We set up a padlet for you to share your insights and opinions on this particular topic with us. So just to let you know that when I'm moving these slides, I won't actually be able to see the chat. So I won't actually be able to answer any comments you make in the chat. And I would please like to ask you to put any questions you have in the Q&A tool and not in the chat because since the chat is just a sequential list of texts, then it'll get to the stage where it's just not possible to see the earlier questions. And it would be a shame to miss any. Okay. So without any more ado, I'm going to hand over to my colleague Fabio. Thank you. Thank you very much, Tim. And great to be here in this webinar actually tackling a very much discussed issue that is MOOCs. As you were saying, Tim, probably even too much discussed in the last years, but I think it's during the Open Education Week is something we should at least look at to see if new things are emerging. And actually, we hope to present you with something new. So let's start from just a brief summary of the typical critical points of MOOCs. As you all know, MOOCs have been depicted as the great revolution, the avalanche, the thing that would change education as we know it. This was back in 2012-13, and then they were criticized for a number of reasons. And of course, in the last year, another boost due to the pandemic and the social distancing. But actually here we collected a few of the critical points of MOOCs, looking at literature and the different debates around this. So a typical critique to MOOCs is the very high rate of dropouts. At the beginning, we're speaking about only 6% of people who are starting the MOOC are finishing this MOOC. But still, of course, these are still very big numbers because, as we know, a number of MOOCs are reaching thousands and thousands of people. And also connected to the second point, the low percentage of active participants. Many people register in a MOOC, maybe look at something, participate a bit, don't finish the MOOC. But actually, this is what happens on the web. And since we are for most of us in the area of informal learning and of curiosity-boosted learning as well, we think this is also something quite natural for these sort of courses. Again, another fierce critique is the fact that MOOCs are not typically built on OER, on openly licensed resources, but still they are still usable for free. So it is better than nothing. And there is now these very days, a big debate on a very non-open education mailing list about the importance and the reason why we are working on open educational resources. I mean, the reason for this is to reach as many people as possible. MOOCs actually are doing the job somehow, if you look at the real figure. And then, of course, MOOCs who are often presented as the thing that can democratize education can reach everybody on the planet with the best courses. But actually, if you look at the data, it seems that the typical, not the only one, but the typical participant in MOOCs is a well-educated, is a person who knows what to learn, how to learn, and actually is Caucasian and with good income. And typically from the north, of course, we have different ecosystems of MOOCs in China, in Brazil, in India, this massive number. So this is a bit in generalization, but it is true that MOOCs have been criticized a lot for this, let's say, missed promise at least. And then as well, most of MOOCs are designed and developed in English by Western institutions, by Western professors, typically US, with the Silicon Valley Communication style. And so many people talk about the colonialist aspect of MOOC, which is in some cases definitely an aspect, but not in all of them. You have a lot of new MOOC platforms again from the south of the world, which are actually doing very well. So this is also a critique which is, let's say, debatable. And then another interesting critique is that MOOCs are not pedagogical innovative. It is like if, especially when we talk about the so-called X MOOCs, so the MOOCs, which typical Coursera MOOCs or Future Learn MOOCs, where you don't really have a lot of collaboration, but you tend to replicate the transmissive logic and the transmissive method of teaching. Let's say it seems that MOOC designers are putting a lot of emphasis on numbers, a lot of emphasis on content, a lot of emphasis on also packaging the content. But if you look at how innovative are these courses with respect to other courses online there, which are not called MOOCs actually there, you can see that the level of innovative is not always present. So you have, of course, a number of experiments still going on with the more connectivist approach and with the more, I would say, different kind of pedagogical approach. But it's hard to respond to this critique actually because most of the MOOCs, if now you would just search randomly three MOOCs on the major platforms, you don't find a lot of pedagogical innovation. And in the next slide we see actually connected to this point, the key, actually, we believe is how you design a MOOC. Of course, quality of content is one thing, the knowledge of the instructors is another important thing, the pedagogical approach is another very important thing. But actually when you start designing a MOOC, when typically either from the top in a university comes an invitation to a professor to design a MOOC, or when a professor decides or a team of professors decide to transform a course into a MOOC or to design to create a MOOCs from scratch, they can do it in different ways. Now on the left side you can see a picture I took from a paper on MOOC designs. I won't mention the paper because I find this to be a bit too, I would say, engineering style. I mean, it's really a process with different steps. And then in this paper you also have key performance indicators, like if creating a MOOC was like creating a new product for the market, which somehow it is, but it's not only that. Then in the middle you see a more collaborative way, which is different heads, thinking about different things and hopefully also about the same thing, so collaborating around the design of a MOOC. And then the third the third kind of design is the typical transformational design. So you take a course, even a face-to-face course, and you say, okay, I want to make a MOOC out of this. I want to reach thousands of students. So what I will do, I will translate it in English and there already you must be able to translate it properly. And then I really want to put it in a platform to structure it differently and so on. So just to say that MOOCs can be designed in many ways, and the way you design, the way you approach actual instructional design in the MOOC is very important for what the MOOC then becomes and for the final learning experience of the users. In the next slide you see, let's say, a generalization, a caricaturization of what we call one-man MOOCs. So you know the famous one-man bands that play on the street, actually there you have a person that does everything, does everything very well. It's typically impressive how these people can do everything. And here the one-man MOOCs are really MOOCs that are built and developed around a single person, typically around an educator. So there are a team of educators from the same institution, but let's say one educator, let's generalize even more. So there you have one perspective, so it's pretty clear, so you can only hear the perspective of the person who is actually providing the content. You have one teaching style, so typically from the beginning to the end of the MOOC, you have the same style, so you have the same tone, the same kind of activities and the same kind of assessment, for example. Typically, these are a transformation of frontal teaching. So normally it's not always the case that I mean the thought put into this transformation process is often not enough. So typically, if you look especially at, let's say the classic again, Corsair or similar MOOCs, we have a person or a few more persons talking, people listening and then if you're lucky you can respond in a forum, many times nobody comes back to that forum, you have an assessment and that what happens in many university classrooms. It's not bad per se, the content is good. Again, I don't want to criticize this per se, but just in terms of international collaboration and innovation potential, the idea is that with MOOCs you could do more, especially in a moment where you have such a big participation on the digital world by so many people. And of course they depend on the leading professor, not only for the style and the content, but also for the quality of the overall experience, for the updating of the MOOC. So it's typically the MOOC manager in a university. If they want to update the MOOC, they have to go back to this professor and if this professor is very busy or has moved to another university, then the MOOC is a bit orphan. So it's difficult then for somebody else to come in and teach and update that MOOC. And you have, if the MOOC is based on some cases or some practices, think of business studies, but also as in our case of digital activism and digital participation, the cases normally come from the ecosystem of this professor, from the country of this professor. So the idea here is to go international, to go global, but typically with a very local perspective, which again is not a bad person. But when this perspective is, as typically happens from the UK or from the US or from the North, of course, one of the first critics that we saw at the very beginning is appearing even more. Then in the next slide, I just want to say that this is not always the case. Here it's an example of my friend, Professor Juan Queimada from University of Technology at Madrid, who designed a fantastic one-man MOOC, actually. This is a professor of software engineering, very successful, and he has been teaching generations of students on how to develop software in Spain. And at a certain point, Miriada Equis, the Spanish MOOCs platform, asked him to transform his teaching into a MOOC. And he said, yes, he took the challenge. And in the video down there, you can see himself telling the story. It's really interesting to see how he started by saying, yeah, that won't be very difficult. I will just record myself, and that's it. On the other hand, when he was doing this, he started to understand that doing this right, it's a totally different story. So you need to rethink totally your key concepts. You need to rethink the thresholds of your students. You need to rethink the technology, of course. And so it took him, I don't remember if one summer or even more, of full time to really rethink his very successful course already. He did it well, because actually now his MOOC on software engineering, software development, is I think the most used MOOC in Miriada Equis in Spain and Latin America. So it's a huge success. And he's also one of the most visited and used MOOCs globally. And there is not the content itself. You have plenty of MOOCs on software development. But this is really, I suggest if you want to have a look at this MOOC, it's really nicely thought, nugget by nugget, concept by concept, every concept builds on the other. So you can really see how this person, how this educator has gone through self-reflection and self-change management process in terms of teaching. In the next slide, you see actually what we did. And so different from the one-man band, we tried to orchestrate the MOOC. So there you see an orchestra. We were not as organized as that orchestra. So we were much more, I would say, in the brainstorming mood for many, for many sessions. But orchestrated MOOCs are typically MOOCs that come where you have different people from different backgrounds, from different countries often, and also from different language and different ways of looking at things, working together to develop a course. Now, in this case, you have many perspectives, which is good on one side. You can perceive it. You can provide students with more than one perspective, but it's also more difficult because you need to find often one idea for a number of subjects. And then, of course, you have a mixed teaching style, which is on one side a bit more chaotic. You don't get always the same thing, but on the other side is much more varied and much more interesting for the student. They are normally co-designed, internationally co-designed. And this means that you leave a lot of space for serendipity, for spontaneous ideas that may be coming in and then disappear and then come in again. And it's a very nice, normally, again, labor-intensive, but nice process of co-design. And, of course, when a MOOC depends on a team, the quality, the updating, and the kind of cases of practical information in the MOOC are also different. So, the quality is guaranteed not by one person, but by different people. Of course, each one of them with a different understanding of quality, but still different people looking at the same thing. Updating becomes much easier because we have, of course, more than one leverages for when you want to update this MOOC. And, of course, cases come from different contexts, which is normally more interesting. So, I just wanted to present you these two, I would say, stereotypical ideas. A few MOOCs are totally orchestrated because also in our cases we had a leader in the MOOC and we had, of course, quite a linear process we needed to have. And, of course, also, one-man MOOCs are not only one-man. There is a team beyond this professor and there are technical people working on it and so on. But these are the two, let's say, limits of how you can design a MOOC in terms of a single person or collaborative effort. And I just wanted to present you these two archetypes in order to then facilitate discussion later on in the webinar. I tried not to assign any value to my sentences because I wouldn't like to push too much for one or the other because, of course, there are also issues of efficiency, cost, time, which come into the picture. But these are the main two that we started our reasoning from. I don't know if I have another slide, Tim? Yes, the link one. Yeah, exactly. So, here I just want to show you for a moment the padlet that we intend to use later on. So, basically, the exercise that we would like to do with you is to reflect together, going beyond the easy answer that, of course, orchestrated is good and one-man MOOC is bad. This is never the case. You have some wonderful one-man MOOCs, as I was telling you before. Of course, you have many issues coming into the picture, but I would like you to propose this to us and then also to help us going on in this reflection and maybe to transform also this initial reflection into research paper. Why not? We could look also at other MOOCs, other orchestrated MOOCs to see how it is going. I think this was all as an intro for my side. I don't know. Thank you very much. Thanks. A wonderful contextualization of the issue at hand. Just before handing over to Bea, you can see there's a short link at the bottom of the slide here. So, I'm going to keep talking for a couple of minutes so you can type that in if you want to. And we'd very much like to later on when we move on to the interactive debate on the padlet page, you'll have access to participate on that. I think one thing that's worth adding is it's interesting that I think Fabio's comments are applicable to all courses, not necessarily just MOOCs. I mean, the standard joke in the MOOC community is that MOOC itself is the only acronym that you don't actually have to obey any of the letters that make up the acronym itself. I mean, it can be massive or not. It can be open or not, et cetera. And I think this is actually an interesting point because when I talk to people about MOOCs and quite often somebody will come up and say, you know, I'd really like to do a MOOC, but the thing is I don't have access to any institutional platform for my course. And, you know, it seems to be quite expensive to contact one of these giants to do the course on there. And I think something that needs to be taken into account is you don't really need these days to have a platform. A platform is more about branding and about certification than actually about course dynamic structure and content. So that's the good news. I mean, if you're watching this webinar and you've always really wanted to get into doing MOOCs, well, you can. There are different tools and ways of actually doing that. And that's why I think that Fabio's comments are still applicable to a very wide range of courses. So thank you very much for that, Fabio. I hope you've all had time to catch the Padlet address. See you there later. I'm going to hand over to Gavio. Thank you, Tim. I also add the link of the Padlet to the chat, okay? So we will come back to this later, right, Fabio? To the Padlet. And we will explain. I'm going to try now to summarize how we did our MOOC, the process of collaborative MOOC. So in this figure, you can see the steps that our team followed to create the MOOC. First, a neat analysis of potential students and research of assisting offer. Second, you can see the step of instructional curriculum design of the course. Then the elaboration of materials based on the collaborative work. And final, the test of piloting of the courses in the platform before the launching. So actually, we are in this phase of final revision, okay? So I will explain a bit every step with some interesting facts that we took in accounting in the process of designing. Tim, can you go down a slide, please? Yes. Here, I mean, I think you are going to see the infographics maybe a bit small, but I'm going to just to to summarize some interesting results. So as I said, the first step of the process was conducted a neat analysis in order to know, on one hand, what was already done. So we look for other MOOCs on open and equational resources on the topic of civic education. And we analyze them to identify gaps and to create a different MOOC from the assisting offer focused. As we explain later, my colleague Cristina will explain focus more on the digital aspect and the participatory tools. And on the other hand, we launched a questionnaire among students from diverse backgrounds, from the partners universities in Spain, Italy, Sweden. And we got more than 2,050 responses and we summarized the results in these infographics. But I just will comment a few things that were interesting for our curriculum design. For example, on the top left, in terms of participation, around 60% of students of respondents thought that they don't have the means to make their voice heard in the internet, digitally speaking. And that's four. On the other side, you can see that there is a need for a change. 97% said that they want their opinions to be heard. And 60% would like to improve their level of social engagement. Also what's interesting in the middle, down the topics of interest, what are the interests of students, potential students for the MOOC. And they pointed out topics as social justice, migration, feminism, that were useful to choose our case studies. And to finalize in summary, you can see at the right of the infographics, most of them, they said they are ready. They feel ready to invest time, also effort to receive more training in digital civic engagement. And that the best way, because they were university students, will be as an extracurricular activity like a MOOC. And they were very, for the methodology, they would like something practical, learning by doing. What was the main benefits they could see? They said with this they could have more knowledge of what are the tools assisting for participating in the digital civic engagement. They also said they wanted to participate to know more initiatives, and also to reach and connect people, connect with people, that is something interesting in a MOOC, and also to exchange ideas. The idea of collaboration was already there from their side. So can you pass the next slide? So after the need analysis, we had the step of instructional curriculum design. That this step was already performed in a collaborative way by all partners in the team. First, the responsibility of this activity proposed some instructional curriculum design, then the rest gave feedback. There was a collaborative revision, implement changes and revision again. So everything is being collaborative and iterative. So we included this image figure, where you can see one of the models that we follow for our instructional design. Among others, this one is proposed by Kono in 2014, and it's called the Seventh Siege of Learning Design. Everything is with us. Which follows like seven principles, you can come the steps, but they are not just linear. So I will summarize to explain how we did the design. First, the conceptualized thing is to conceptualize the pedagogical vision of the course, which are the topics, learning objectives, which will be the target audience based on the need analysis, and also the pedagogical approach. So at the end of this phase, we had the structure and the flow of the course. Then the second, after many collaborative revisions, then the second is the creation of the educational resources, like learning materials, activities, new materials, selecting open educational resources. Then the third, communicate is refers to the selection and configuration of the tools to facilitate communication that was already very important in the need analysis, such as the forums of the course and the activities. Four is collaborate, that is, it refers to the establishment of mechanisms to encourage collaboration and group work. So we decided to include, as I will present, in a bit collaborative activities, also within the MOOC. Then we have to consider that refers to provide also ways to reflect and demonstrate what has been learned through assessment and also final reflection and putting practice that we put at the end of our course. And also the important that Fabio point out of having also a team of teachers that in MOOCs are called facilitators, yes, that they will give feedback to the students, meaningful feedback. Then we have the combined, this is like an iterative step during all the design where all materials and activities are put together and contextualized in the overall course structure. This was done at the beginning with the first proposal, then another time after they will reflect on the conceptual framework and flow and then also after all the creation of materials. And finally consolidate the last C corresponds to the implementation and piloting. So make the final adjustments and finally that we will do shortly launch the first edition, after which we will also evaluate the design in the real context and we will make improvements for future editions. And the last side from my side team, please, the collaborative work. I mean in this slide you can see the type of materials and activity that were elaborated in a collaborative way between partners. So the process was the same as the curriculum design. There was one partner responsible for the elaboration in this case of one of the modules of the course, then all partners gave feedback, then made improvements on that new revision implementation to finally piloting of content and technical aspects of every module and final improvement. So it's an iterative, it was, it has been an iterative process. And what we have in our MOOC is videos of course. We have like introductory videos that were also collaborative made because the scripts were made collaboratively and recorded by one partner with the technical support from another partner, everything of course from the distance with Zoom and other tools. We have also existing videos and one of the aspects that Fabio was also talking about, we included cases, cases studies. So we did interviews with national and regional experts, showing examples of civic, of digital civic engagement and examples of how to use these tools because we were, as we told you, focusing on the practical side. And also this was made collaborative, the scripts, questions for the interviews and also collaborating with the experts. Apart from the videos, we have explanatory materials, theory, but from a, with a focus on practical perspective, examples of digital participatory tools in all modules. And for practice or assessment, we have self-evaluation test, reflection activities in forums, and as I told you, collaborative activities with external tools, tools like the one we are using today, Pathlet. So that's the idea how we created the MOOC. And finally, the result is going to be present very briefly by Cristina. Thank you very much, Bea. I quickly before passing over to Cristina, just a couple of points here as somebody who has participated in this process. I think in a way, a little tongue-in-cheek, we could actually call this a DMOOC. I mean, you've heard about X MOOCs and C MOOCs and L MOOCs and S MOOCs, et cetera. But I think we could definitely call this a DMOOC, DB and Democratic, because I don't think I've ever, ever participated in a development process that was so thoroughly considered in this Democratic fashion. I think it's been quite interesting. So let me leave you, the public, with the participants side, with the question to reflect on as we carry on. And then maybe we can come back to her later on. I mean, do you think that you can tell that we've been through this process? Or do you think you should be able to tell? That might be the better question by actually looking at the MOOC. I'll leave you to reflect a bit on that. And then thank you, Bea. I'm going to pass on to Christina now. Thank you. And thank you, Bea, for also for your great work as Orchestra Director during the course design and production. So this is the result of our orchestrated Democratic DMOOC. The title is Civics 4.0 Active Citizenship and Participation in the Digital Age. It's a six week MOOC intended for anyone, but especially with the focus on university students and youth from diverse backgrounds interested in digital civic education and digital participation. There is no entry requirement. And at the end of the MOOC, participants will be able to understand active digital citizenship, knowing more about what other skills needed to be an active digital citizen, being able to monitor policy, being able to connect with people in the community, identify tools on how to engage and organize a campaign in the digital age. They will increase capacity to participate effectively and responsibly in the community. And at the end of the MOOC, they will put in practice all the learning acquired through the course with final project work. Based on the current plans, the MOOC will be soon, so we are planning to launch it towards the end of April. And it will be hosted in the UNED platform and it will run for six weeks. So this was the promotional part of the MOOC. So stay tuned when the MOOC will be out. There will be announcements through our website and social media. Thank you. Thank you very much, Christina. I mean, as you say, as you can see from the structure of this course, we've tried to make it as active and interactive as possible and relevant to the social and cultural profile of the typical students who are our intended audience. It doesn't mean it's excluding anybody else, but that's the way we've gone through that. Okay, so what we want to do now is just pause for a second and compare this process with, if you like, a typical institutional perspective on how MOOCs are done. Now, I was actually the founding director of Open UNED, which started in 2012. And just very briefly, I want to walk you through the way that we did MOOCs at the time, at least several years afterwards, although it's evolved from there, obviously, and compare it with the way that we just talked about this orchestral way of developing a MOOC course. Now, back in 2012, if you remember, that was the so-called year of the MOOC. There weren't exactly a lot of MOOC platforms around to be very generous. In fact, there weren't any. So we banged out our own open software called Open MOOC, which we used for the first couple of iterations of the courses. I mean, Google course builder came online afterwards and Open edX much, much later. Now, the interesting, perhaps, perspective for us is that UNED is essentially an online educational university with a blended methodology. So we didn't have the first hurdle you have to cross when building this kind of course, which is that people aren't necessarily familiar with the sorts of materials you can actually put online. So, for example, you really wouldn't want to put a 30 page PDF document as part of a course. I really wouldn't be a key to success. So we didn't have this kind of problem because our teachers have been preparing online courses since the year 2000 when we had our first online campus. So we're reasonably familiar with that. But that doesn't necessarily mean that it was all sunshine and light and laughter and happiness because what we also decided is that since our main VLE, like most universities that have an LMS or a VLE platform that carry their official teaching and learning on, it has a very rigid structure, not because of any technological, educational or methodological reasons, but for typically our political reasons because you want to keep things carefully under control with a structure that has to be approved, that reflect as far as possible the study plans that have gone through official bodies, etc, etc. So things are always done in a particular way. What does that mean? What it really means is if the only tool you have is a hammer, then you treat everything as nails. The courses are pretty similar. So, for example, if a student is doing a course, I don't know, a degree in economics and wants to do a free module for mathematics, then they should find the environment to be exactly the same in the structure of the materials and the support and quality. Now, what we wanted to do when we kicked off the MOOC platform was basically let people play around and do different things. So we had a different platform and we provided some degree of support. I mean, obviously we didn't have the funding to give huge amounts of support, but we basically said to the teachers, hey, play and see what kind of MOOCs you can come up with. And we gave the support that was actually what we could actually give at the time. Now, the interesting thing is that had some advantages, but also some disadvantages because even though they had different platforms, they still came along with the same expectations for support, quality and structure. And as Fabio said earlier, quite a large number of our students weren't, in fact, people who had been excluded from the formal educational process who we would have loved to have included in our learning program. A lot of them were actually people who are on our degree courses that just thought it might be interesting to do some of our courses. And of course, they came along and found a different platform, different support mechanism, different way of applying quality mechanisms to the materials. So there was a bit of a mismatch of expectations, which actually led to a bit of a friction and difficulty at the beginning. So I mean, okay, I'm going off topic a bit, but I think it's an interesting point to make. Now, having said that, we actually gave people carte blanche to be able to develop the MOOCs in the way they wanted to. So to begin with, we had a range of courses. We had the one man band, single lecturers who wanted to take some topics, some material they had that perhaps wasn't particularly being used in official teaching and move it online. And then we had other people that wanted to form small groups of teachers and also prepare their courses in that way. To be blisteringly honest, I have to admit to begin with, since this was a platform we developed ourselves, the technological possibilities with the platform weren't that great to begin with. So a lot of the courses ended up looking pretty similar for that reason. But then as some of the platform got developed and we gradually moved over to Open edX, which is a reasonably good platform, then it gives a lot more choice and possibilities for the flexible development of courses. So I mean, basically we came across these different approaches and groups of people who wanted to undertake the courses and they'd come along and say, okay, I'd like to do this MOOC on this particular topic. And we'd say, okay, that sounds good. Have you thought about the kind of structure you've used? Are you bringing materials from other areas? So I mean, typically a MOOC will last as I think I've already said earlier, like the structure we have in a moment is about six weeks. And that's typical for a MOOC. I mean, six weeks in the sense of having four core weeks of working with a one warm up and a one cool down week at the end of the course. So you really want to think about how to structure that kind of time to make it reasonable for the amount of effort that people can actually give to study in that particular process. So if sometimes someone would come along and they've got perhaps material from a big course they'd run previously and they wanted to adapt it online, then they also have to bear that in mind, because otherwise it would just be overload for the students. And we had examples of that because at the end of the day, we could encourage, we could sway, we could cajole, but we didn't really have the right or the institutional muscle to be able to say to them, they shall not do a course in this particular way. So you had a bit of everything to begin with. And that was obviously quite an interesting learning experience for them, for us and also for the teachers. Another difference, which I think Bay has also mentioned as well, is the role of the people in the particular courses. I mean, in the sense that on official courses, the teachers, the people who prepare the materials in the courses are also the ones who are most active in the forums, the chat, and the synchronous and asynchronous communication with the students. They're encouraging them in the activities, they're supporting them, answering any particular questions and leading particular activities. Now, the big difference with a lot of the moves is that the teachers, they set up the course and they might be around particularly in some moments of a particular course, but they're not there all the time, that's not their key role in the course. That's when you need to think about facilitators, you've got people who are trying to keep motivation going through the particular modules and trying to keep the students actually engaged with what's actually going on. So this was something we had to try and communicate to our teaching staff, we had to sort of say to them, okay, well, who's facilitating this course for you? Are you going to do it? Are you going to get some of, perhaps some of the tutors to do this for you, other colleagues you have in other institutions, but it's something that needs to be looked at from the beginning, because what you can see by studying the studies that have been done on MOOCs, is that you've got a lot of effort at the very beginning and then it typically dies away as the course goes as the course goes on. So it's something that you need to give into account. Okay, so I mean, we've had examples of both types and I think our particular experience wouldn't necessarily be the same as other institutions, because I mean, some typical face-to-face universities have started very, very successful, very high quality MOOC platforms and perhaps that's required a little bit of adapting on the part of their teaching staff and once again, they sometimes combine materials they're having different ways in a particular course and sometimes you might have one individual teacher and other times you might have groups of teachers. Other particular recipes for preparing courses that I've come across during this period is people who, for example, were working on something Fabio said before, open educational resources, which has been a huge field of endeavor for democratization of education and social inclusion and then you'll find people who might have prepared open educational resources and they want to give life to these resources in a way. They want to give the flip side of the coin, the open educational practices and want to turn these OERs into MOOCs and sometimes it can actually be the authors of the materials who want to do that and sometimes it can actually be people who are interested in these materials but haven't necessarily been involved in their preparation and once again it can be a one-person attempt or it can be a more orchestrated actually approach. I think finally something I think is very important to say which I think also the other presenters have said before had is that even a one-man band as such is a bit of perhaps a little bit of a misnomer because behind that person you've got the methodological support stuff which are helping with the materials and the activities, you've got the technical support stuff and you've got the people actually running the platforms. So there's quite actually a lot of work actually going on to do this and the only thing I would like to I think defend about this whole different ways of actually preparing these courses is that to try and encourage the freedom and the educational experimentation because sometimes we get locked into the same way of doing things and teaching things and that is a bit of a shame and by doing courses like MOOC and different approaches to actually preparing them then we can explore the conceptual space of possibilities with our students and with our colleagues and come up with different kinds of courses that are actually quite interesting and beneficial for the people and I'd encourage you to go and have a look at our course catalogue and the interesting thing is that even though it doesn't have this official rigid structure of course preparation then the majority of the courses are actually the same. Quite why this is the case, I don't know if it's an educational phenomenon, psychological or just a social cultural phenomenon that living and working together we end up doing things in a similar way but okay that's what I particularly wanted to add there. So what I'm going to do now is to... Tim, sorry. Yeah, no just to say that we have a few questions and especially one I think a difficult one for you. We have some music questions that we can take care of and a difficult one for you. Can we start with a difficult one? By all means, I'll stop sharing screen. So you've been following the questions so perhaps you can present them to the people you think are most relevant to answering them. Yeah, well I think it makes sense to answer by voice as well so that we... I mean, everybody can hear them. So the question by Lian Muray is what is the typical average cost of a MOOC, including design, investment and ongoing delivery? That's a very interesting question and it's considerably less than what your commercial platforms like to charge for them and it really depends on how you actually come up with the bottom line of the cost of a particular course. I've heard people talk about tens of thousands of euros for the cost of a MOOC but what you have to consider is the context in which the course has been developed and used and the classic story which I particularly like about this was that perhaps I shouldn't talk about the actual institution name for political reasons but let's just say one of the big open educational universities in Europe had a problem, I don't know, 10 years ago or 15 years ago because what they were doing in the realms of open education, I mean they weren't calling the MOOCs at the time but it's essentially open courses were being if you like frowned upon by the managers of the university who as usual were concerned about cutting costs. They didn't want to have money spent on this and then the guy who was the director of the program who's a friend of mine, what he did to close this conversation once and forever was to make a click-through to actually say okay if I put a half-page advert for the official training courses at my university in a prestigious broadsheet newspaper, I know how many thousands of euros that's going to cost. Now if I do this particular online course and set up a similar kind of click-through mechanism then you can actually measure and show that you get more click-through from the people actually doing the courses than from the ones reading the newspaper so the actual cost benefit is there in a particular way because I mean Liam if I don't know exactly where you're coming from to ask me that question but for example if you wanted to have your own MOOC on a platform you don't actually have the platform then there's a whole set of skills and resources you'd need to actually be able to do that. You need to be thinking about the instruction of design for example you're going to call in somebody to actually help you with that you've got somebody in your team or your educational context to actually help you with that then when you're thinking about putting the course online do you want it branded with your particular institution or would you be happy to put it on one of the many open platforms that are actually around so if our form model for example whether it depends I mean typically we don't pay the people who are moderating the forms if they're in the forms it's because they're doing it for a range of issues just that they be there for the entire six weeks but that doesn't mean that they necessarily need to work in the entire time I mean if you've got a group you have to see okay how many people have I got who want to be the facilitators for a particular course let's think about how they're going to engage with the students I particularly interested in one in one module or one particular subject and we can divide them up and then give them some support for the for the teachers because from my experience they are usually for example in our case tutors from UNED or people who are actually working with students and they are heavily motivated in helping working with with their students for example in the last summer usmas plus project we did moonlight we did a language MOOCs for social inclusion and it was the we contacted the the non-governmental organizations that were in an around majority I think we got in contact with 20 of them in the end and got them to to help us actually do this course and then we found the people who were actually working with the refugees and migrants were volunteering to be the forum moderators and were particularly happy to to do that I mean if you actually do this in a pure economic terms then it can be quite expensive for sure if you're if you're doing it but really if you if you contextualize it in a in a broader more socially integrated environment then people are usually quite happy to participate and actually support the course if anybody wants to add anything to that just adding I don't know if some of you are familiar with the recently finished MMA project it was a european project I think it was a coming from the research side of the commission and the objective there was to co-develop MOOCs I mean not not just to to build the platform like the classical platforms MOOC platforms where you have MOOCs from different universities the platform is just displaying them but that was really the idea was to provide a platform for automatic translation and also some cultural collaboration around different kind of MOOCs and there the main issue apart from the classical sustainability of these kinds of project was the cost involved because as you say if you want to do a MOOC by engaging more than one professor in our case we are supported by european project but the work simply multiplies I mean you can go from a very limited cost you can use one of those platforms out there and just put your stuff there but if you want to integrate different views and different things it can it can become expensive especially in terms of times yeah I I quite agree I don't say I think I should feel quite fortunate in the sense that our institution actually provides us with a platform for us to use and support mechanism and technical staff at zero cost but I mean obviously there's no such thing as as free so that cost has been absorbed as part of the overall budget for online teaching and dissemination and different kinds of activities and just beginning on some of the the comments in the chat because as before I could naturally see them it was interesting what I saw about Marianne Lesanche about a specific MOOC for female entrepreneurs with such a high success rate I think experience also shows that with courses that are specifically focused and prepared for a direct collective then you can have a better success rate but it also depends not just on the things we've spoken before but doing it in such a way that it's actually it meets the expectations of the students who are doing this course now one particular example I could think of in for this point which is not really to do with MOOCs although it's for an online course and I think it's directly relevant is that a colleague was trying to prepare some online training for for women in Palestine and the what they had in common these these ladies would they they they suffered an awful lot of you know bad treatment by typically by males etc in in that particular region for the reasons you can probably imagine and they were freely very alienated and it was really quite interesting because the typical dynamic is you could have a study session and you might have a group of maybe 15 ladies connected with their web cameras on although they'd be wearing the the traditional clothes but they still be active and participating but the second that one male connected to that particular environment who'd be a member of the teaching team they'd all switch the cameras off immediately and the the participation would actually go down enormously so sometimes when you're thinking about preparing a particular kind of course it's important to to make sure that the people who are appearing on the courses are from the same social social group and sometimes you can talk about educational proxies there to to actually enable that connection to to take place but I mean if we take your question out of context Liam for example let's imagine for the sake of arguing you're not in any institution that would actually support you I think these days there's plenty of of online tools I don't want to particularly want to mention one tech provider over another but if you actually search around there's there's plenty of ways of doing MOOCs without a MOOC platform or doing a course without any platform at all and that's actually provides us with a very rich and varied way of integrating our activity with people's desires to fill in okay Fabio where are there other questions maybe maybe for the members of the yeah I think there is a question on on assessment within our MOOC and also which goes from peer assessment to anything we use it in this I think Bea you can answer maybe this question and then there is another strategic question for you team on if there is a model that can be used to cost and justify the creation of a MOOC to a higher education institution to a university so how to advocate for a new MOOC with with the proof that it will not cost so much so that's okay but maybe Bea you can briefly explain how the assessment will work in the MOOC yes apart from the self evaluation and the with a test and also taking that they are no the videos and and readers are being visualized we as we said at the beginning we have collaborative activities and a final activity that is the final project where they the participants will have to put all they have learned in practice following a very simple template that we designed and they will receive feedback from the team it's true that this particular collaborative activity I get they can do individually or collaborative between them but these activities are presented like optional or voluntary because you know you you can't force everyone to to do all the all the activities in the MOOC but yes we were trying not to offer only the typical no self evaluation test but other kind of activities where they can also reflect and and those who are more motivating or have more time can can do it and we will provide feedback to those anything else Fabio no just a comment on this which is also connected to the cost I think Liam was also commenting on the cost of moderators and facilitators during the MOOC of course we took that into account and also there the idea is to use the the project partners some people from the project partners to help so that we can have a multinational and multilingual group but of course that's an important cost especially when you have big number we have no clue about how many people will will participate in the first edition of the MOOC so there we actually in that case you still don't know the you don't know the cost until you launch a MOOC because if you want to do it with some interactivity you need to to to include the cost or the time of some some facilitators and people and so in that case it's a it's a moving target they would say the the final cost of the single course yeah I agree assessment has always been a big a big issue in in MOOCs because typically it's very hard for if you've got a ratio of maybe a one or two teachers to I don't know 5,000 10,000 students which you can have or more then obviously it's very difficult to do any meaningful assessment because if you put a some kind of closed closed test on a course then you can show that the students have been following the material and they've understood what's actually being shown to them but it's not really demonstrating that they can apply the apply the knowledge of actually learning the good news is I think we've moved on from these sorts of problems and I've got colleagues like Maria Dolores Castanillo if you search for her online who has done a lot of work in this area and shown that you can actually develop if you like rubric sets of instructions to enable students to correct each other's work and that way you can actually set up peer evaluation in a particular course which obviously it's not the same as having expert evaluation but it's it's pretty good and can actually enable us to to actually provide some degree of assessment to actually going on should you should you play the game with padlet yeah I might not I think I can play yeah okay so as I'm not going to share the screen but we'll talk while it's some going oh do we have some likes we can start thinking about that now perhaps so anyone's been able to connect to it okay so the way this is structured if you're across on the page is that we have the maybe I should just share the screen briefly hang on a second let me go back here and then thanks man and then I can make some life easier thank you that way you can see even if you're on the on the on the page okay so what we've done here is what they has done is actually set the different types of MOOCs up with the pros and cons the things that you might appear might think are easier with one kind of MOOC than the the other so for example I mean it occurs to me that for example a one-man MOOC what's what's easy is this the the coordination because basically you can decide just with yourself what you're actually going to do on a particular course where it's with the orchestrated version you have to actually you have to actually have to you know sell your ideas to your colleagues and discuss with them and and and you know go through the the points you want to do in a more in a more democratic fashion so that's actually like one of the advantages of the one-man MOOC okay so I can see Fabio is adding to the the page is needed here okay the cons if you like in a way is things that you know at a time that you're having to spend on your own to do the to do the the course maybe I should actually add something here as well do this page so and I mean we we don't want experts opinion so feel free to put there what you think I mean things maybe it's not it's not necessarily true that you need more time to develop a collaborative MOOC but just we just would like to get your views so just if you just double click the the post it will appear and you can just put it whatever you want just to to see if this distinction we made makes sense or it's maybe too artificial or if we have some some different opinions and also if if you've any particular experience of doing either one of these MOOCs and you can give us some feedback on this as well that would be interesting to to know because sometimes you can find MOOCs at one particular stage start off if you like by being a one-man affair and then as they start to get developed then they attract the attention of people who may be initially thinking of participating as facilitators and then as the development process goes on they get more and more involved and end up being a member of the teaching team or perhaps that can happen over subsequent editions of of MOOCs or maybe you think in fact that the separation between one man and orchestrated is not quite so dichotic it's not black and white there might actually be a whole spectrum of possibilities thank you Leo if I'd invite you for a coffee if that were possible to say thanks as the first person who's added something to the yeah I think you're completely right you can you can write comments as well I see people writing comments so it's perfectly fine yeah yeah the comments are coming up yeah thanks Vlad see when you're written on there actually the the secret narrative of this webinar is to see if this approach to MOOCs can reply to some of the critics to some of them at least can be smoothened by possibly a collaborative approach of course that would be at least let's say less one way let's top down let's say north south for example so it's there is a secret meaning there I think that's very very true so it's a question of of mutual support and mutual control because when you're you're doing things on your your own you're applying at the end of the day your own criteria and that can be good but it's also very good to have other people as a sounding board and to provide I'm not going to say more rigor but additional if you like ways of actually thinking you know have you really thought about this and what about if if you take into account something else or maybe they have particular experience and they can share that with you as part of the development I mean obviously I suppose we're implicit in what we're talking about is that these sorts of courses we're talking about are typically more ex MOOC pointed than CMOOC because CMOOC by themselves are typically have an emergent structure so I think would maybe tend to be more well I think perhaps you could argue that both aspects because on the one hand they're one man in because somebody has to start them off but then when they actually take life they actually get cracking then they become more orchestrated because then they feel like the students or participants on the course are deciding what they're what they actually want to learn and so it's a self-fulfilling prophecy in a way. Yeah and another comment which was made is to do with the sustainability of MOOCs so we know that MOOCs or online courses tend to get old pretty soon depending of course also on the subject but I mean in the case of a decentralized MOOC with different heads on it and maybe even with different languages I guess that if you take part on it then you are you will be thinking of translating it also for your for your your language so in that case one MOOC can become more different MOOCs that can take different shapes also in the different in the different universities that are using it so it's I think that that was a good point I don't know if it was glad to make it but I think it's it's very good. Absolutely I think that's a good point there's a question raised here by Anonymous about student retention and I'm not necessarily sure that the retention figures are different between the two the two kinds of courses and I'm sure you can find examples of woman courses which are very good as the one that Fabio provided earlier and on the other hand you can find examples of orchestrated MOOCs that aren't terribly good so I think there's no necessary guarantee of a particular approach for providing a specific kind of course and student retention I think that can be a problem and it's helped really by the quality of your facilitators and also in the interest that the group actually have for the course are actually running the when we've done specifically very vertical very well focused MOOCs for a particular group of students who we know will be extremely interested in them then we do find that the retention lasts a lot longer but this is something which can be taken for granted I don't think you can ever sit back and just think okay this course is well designed it's well structured the material is good I'll just sit back and let people carry on because I mean I think most of us have done a MOOC at one stage or another and what do we find where we find that reality comes along real life and our best intentions of the time we were going to be able to spend on a particular course drifts away as the initial enthusiasm for the course actually goes across one other thing we haven't actually mentioned which I think is actually quite important to to cover is that quite often on official courses you have a like a sequential structure on the course it's almost if you like a like a carpet rolling out as the course goes along or book where the chapters appear as you move along but typically on most MOOCs you find the entire course structure open from the very beginning and that can provide difficulties for the students who actually undertake in their course and also it can be complicated in the sense that you come along you go into the the course you jump around between the modules you find the things that you find are quite interesting and then you step back and then may not you may miss the actual flow of the of the course okay we've got some great comments coming here on the on the paddle I'm pleased to see that everyone's a lot of people are actually participating with this one man concept work overload I think that's a very good very good comment one thing I forgot to mention I don't know Bea if you also agree is the the learning dynamics within the team developing the MOOCs I mean we shouldn't forget that there should be some professional development also for the educators or for the ones working in the book and in this specific case I wasn't I'm not super proficient in the in the field so I had to work a lot and they were experts of course in the in the team so actually I had the feeling that it was a genuine learning process for many of the people also thinking a bit outside our classical themes which is good it's refreshing so I think it's a lets you discover also interesting new perspectives on things it's a good idea Beth you got a comment on that yes yes yes I was going to say I was reading the comments at the same time sorry yeah I totally agree because normally we it's true that we had in the it was very interesting that we had in the team we had experts in the topic and we had experts on developing MOOC so we have we had to do a very close collaborative work and and kind of understanding to to put all all together I don't know yeah that was really interesting yeah so do we comment some of the yeah please do let's see because I like the one of I agree actually I like collaborative MOOCs but I I agree with the one of somebody anonymous putting the one MOOC MOOCs the coherence yeah because actually it's not that a collaborative work is not coherence or consistent but you have to put a lot of effort to make it coherence more than if you are doing alone but it but as as you said also at the beginning Fabio I mean to put other perspectives and and ways to to see the things and ways to present materials and pedagogical approach is also important so that should be taken into account that maybe one man MOOC has more only one approach so it is richer but the coherence is yes it's a good point what else do you want to comment anything Fabio Cristina yeah maybe just to comment on what Leo was saying that what we call now D MOOCs or MOOCs are close to the C MOOCs so the original connectivist or more collaborative MOOCs mainly because there is more space for co-development and reflection also with the learners with the learners so in our in our specific MOOC we for example in every module has at least a couple of reflection moments apart from the self-assessment but a student should just you know do something look at it for example at a tool for a civic participation and then reflect on this tool in a pilot or in the forum in different places and and and co-develop something mini posts and even at the end if they want an optional mini project so I would say that the more the MOOC is was open or the online course as Alfredo is saying it's the same thing the more it has been designed with an open mind in this case a group is forcing you to have an open mind otherwise you don't you cannot proceed the more then you can take this open approach into account so co-creation and this sort of thing so I think it's a it's a good parallel or between orchestrated and C MOOCs I think it's in that direction at least absolutely I quite agree it's an interesting comment by anonymous the about the one of the cons for one man MOOCs is the difficulty to answer students questions I think it also comes back to the point I was making earlier about the role of facilitators in the course because it's not typically the teachers who actually answer the questions is the facilitators so for example I can remember one man MOOCs for example who smart enough to have a group of however dozen facilitators who actually stepped up to the plate quite nicely and provided that level of advancing the other thing which I think is an important factor when doing a course is the homogeneity of the student population because we I mean for example if you're if you're doing an official course on something at a university for example it's the first year degree course then the chances are the majority of the people who are doing that course are in a certain range age range and have a certain amount of knowledge which is typically less than the people who are running the course about that subject now since MOOCs are actually open you quite often find that some of the people who participate in the MOOC are in fact people who know an awful lot about it and they're participating because they like it and they find it very interesting and they want to support the initiative in which case what you actually find is that you actually get students answering each other's questions and this is this is actually a wonderfully positive when it happens although you have to be careful obviously you have to you have to be careful and keep an eye on the sorts of answers that are being provided so you can get some very nice dynamics set up in these courses. Well this is looking really good this padlock it's really really come alive we got a lot of really good comments coming on here. Christina you've been quiet for a while would you like to pick any of these comments on the padlock and give us your opinion on it? Yeah well I just wanted to add about the positive aspects of the collaboration that also the working methods have been an interesting aspect of the collaborative production of the MOOC in addition to all the intercultural things that have been mentioned already also learning from the other people involved in the production in terms of working methods and which tool they use for producing contents for planning activities I think this has been also a positive aspect of the orchestrated MOOC. Yep absolutely I think that's a very good comment to to make okay I think this has been been and has has proven to be an interesting activity I think people are providing a lot of comments on these these courses so okay let's end it perhaps in the few minutes we have left let's go around the table and let me ask you the question that if you now had to do a new MOOC at your own institution not necessarily related to a specific project or anything to do with the topics that we've been talking about in nexus which of the two approaches would you undertake and and why okay so I'm gonna gonna go around and ask each of you Fabio can you can you start please with this well I can start I would say egoistically I would go for the collaborative one because again I have to say I learned a lot and I enjoyed it so in many cases when it happened to me to have to write content for a course or to design a course actually on my themes on the themes of my research you feel that I mean your own motivation sometimes is a bit it's a bit stagnant because it's you have the more of the same syndrome why is if you if you explore something in a group of course new ideas and normally comes up and it's so in terms of enjoyment and self-learning I have to say I would definitely go for the collaborative one but then we have also an efficiency issue and in terms of efficiency now we had the luxury of being in a european collaboration project where collaboration is part of the game and we had time of course to do this and that is how it should be now I'm thinking of a of a professor that has to put online a course for example six months ago or even more when the pandemic started so that does not have the luxury of taking a few months to do this so in terms of efficiency probably some sort of a mixed way maybe with this with not the one man but I would say a quartet thinking in musical terms a quartet at the very beginning and that and then an orchestra joining in a second moment would be maybe maybe better so you have deficiency in the first part but actually if I think of our process I think most of the time was really in the initial part of the design it was really brainstorming what do we mean by this and then when we had the clear the number of issues the whole process was very much streamlined actually and as Cristina was saying I think it was like working in presence in the same office so that was that was very very good but I think for to to make more efficient the first part maybe we'll start with a trio or a quartet and then move to a full-fledged opera orchestra I totally agree I mean if I had to do it again maybe now I will say no please let's do one man MOOC or one institution MOOC but then I will realize that the because this is the second experience with collaborative MOOCs as you know team before with moonlight and the best results are always when you collaborate with you know people that are expertise or with the or are involved with stakeholders like the case studies experts or our partners from KIC or ICAS that are real experts on the topic so I think I will do the same way I would do collaboratively okay thank you Cristina well yes I would do collaboratively because we have it from a lot of fun but also a reference back to the question in the chat before about how we could advocate in higher educational institutions for those institutions to prioritize the production of MOOCs those collaborative MOOCs could be seen also as a way to put in practice many of the memorandum of understanding for international cooperation that we have so those type of collaborative activities can be a way to collaborate with other institutions to internationalize the curriculum but in a real case scenario so really working together and at the end and also within the institution with people from different faculty I think in at the end it's a way to set up efficient working groups and teams and there may be also an added value for the institutions and for the higher education collaborations absolutely I agree with all of you I think really the added value comes from the diversity of the people participating in the orchestra if you like I mean if we were all violinists or we all played the the drum for example then I don't think the collaboration would work quite so well I mean it's a fact we're all bringing something different to the table I also think that each part of the approach doesn't necessarily exclude the other because there are sometimes when you want to be working individually or perhaps in a small key core team and other times when you want to widen up for participation because I mean in our particular case now we're really the clock is running down we're coming towards the end of this development period and it's just very very creative and wonderful like we had a project meeting today for example and as we start talking about it and lots of things come up and the thing is we have to move towards closure on this particular issue so therefore I think it's good to have a small set of people maybe our case is going to be just just us four and we're going to be the pragmatists and we're going to be taking the decisions the executive decisions that have to be taken but the nice thing about the whole MOOC life cycle is since it's iterative yet the first second versions of the course then it's not the end of the world if you decide not to do something for a first edition or even if you decide to do something for a first particular edition of a course because it's always something you can try again in future versions of the course so in general I'd encourage everyone to get collaborating in MOOCs and other online open educational initiatives I mean MOOCs are not by any stretch of the imagine the only way to attack this particular project at the end of the day I think added value comes from the collaboration so we've got a couple of minutes left to us now I mean I'm sorry we're not really paying a lot of attention to all the comments that are coming seems some old colleagues and friends came in the chat it's really lovely to see you here and I'm pleased you've actually enjoyed that one thing at the last thing I'm going to do whether it has come up in the in the the chat is flip back to the presentation to please encourage you to come to tomorrow's event in the Eden 2020-21 OEW series the webinar on supporting teaching and learning in schools through open education resources lessons from the the pandemic was moderated by my colleague and friend Antonio Atesieta and I think that looks that promises to be a very interesting event so if you you can find the link to this now to register here you got a shortened address and also on the Eden online website so I'll just leave this here for a little while for you to see thank you all very much for having come along and for having participated so well I mean whenever you set up a padlet you can never really tell how it's going to work out and I think we're we're very grateful so I'd like to say thank you to everyone who's participated in this thank you to my panelists thank you to the wonderful technical support we've had from from Eden from Diana and for everyone who's actually come along today I look forward to meeting you face to face in the in the future thank you very much and bye