 Thank you very much to the webinar hosted by Eden and Eden Knapp, the network of academic professionals, and this webinar today is about the quality and technology and learning on micro, meso and macro level, and who are the stakeholders and why and how do they have an interest. So this is the first webinar hosted by this special interest group on technology-enabled learning and quality-enabled until and quality in learning. So we have Eleonora and we have Salta and Ferens from Eden and we are waiting for some more people dropping in during this hour. We will see how it goes. But very much welcome to all of you. So this webinar, the facilitators in this webinar is myself and I am an Eden EC member and also I'm coming from the Swedish Association for Distance Education in Sweden. I am also coordinating this special interest group about technology-enabled learning and quality enhancement. Together with me is Mark Nichols who also is in the Eden EC from the Opel University in the United Kingdom. And then we have our president from Eden and we now belong to a visionary from the Otters Magnus University in Lithuania. So it will be the three of us and we will take different parts here in this hour on the different levels of micro, meso and macro level about TEL. So this Eden special interest group have been discussed for a while in Eden EC and in June when we had the annual conference in Sweden in Jönköping. We launched officially the special interest group on TEL on quality enhancement and we did that first with a presentation by our president, Irina, at the welcoming address and then we had each of the days we had different seminars or workshops on the teams of the Reginald and the Action Plan for this Eden special interest group and about renewing the quality agenda and about quality and TEL at micro, meso and macro level and about stakeholders and then also about innovation for quality leadership. And we had a very interesting discussions during all the workshops during that week and people had a lot of interest and we have also now launched a special page on Eden web page for this special interest group. So please welcome to have a look at that and to contribute. So far we have a group, a core group, as I said myself I'm leading this group and then we have Sandak Kristina Softik also from Eden EC, Mark Nicolás and Irina Volongoviciana. I have already mentioned who will be here today. We have Antonella Portchev, she is the share of the Eden app and she is in our group because this is a collaboration with Eden app. We have Antonio Teixeira, the former president of Eden, we have Ulf Elis from Germany who actually will host the next webinar in the beginning of December, the 7th of December. Then we have Jakke Kouami and we have Jakov Katz and so we will also have Fred De Bruyere. So we are welcoming many Eden members to take part in this special interest group and to collaborate about this very important topic on quality and tell. So first briefly the term technology and learning is maybe not always so easy to define as it has a lot of different interpretations and that is also both time. Time and context dependent. So there is not just one definition and during this webinar we will discuss this a lot. However, the concept is more and more used to be talking about enabling learning instead of enhancing learning and this is from work by Keyfoot and Pryce in a report from Commonwealth of Learning. So what does it tell about? In overall it is to enhance and to enable possibilities for learning and that is what such a term is more used than enhancing because it is rather difficult to enhance learning in that way. But it also has some synonyms as we will hear. Sometimes it is just e-learning or hybrid learning or blended learning can also be called. So it is not just about the technology as such but using digital media and digital technologies. So this seminar is about quality and again also the topic quality and the term quality is rather difficult to define and to interpret because it is time dependent, it is context dependent but most of all it is a question of quality is in the eye of the beholder. So what's in it for me? And in this webinar we will hear a lot of different kind of perspectives about quality enabled learning. So by that I will leave the floor to Mark to continue. So please Mark. Great, thank you. I forgot to mention before you start. Sorry, I thought they were all uploaded previously. Krisha is it possible to get my slide set up please? Lovely, thank you. This webinar is recorded and you can have the link afterwards and also you who participate in this webinar will get a badge afterwards. That is important. So thank you. So please Mark. Wonderful, thanks Eva and thanks Krisha for getting the slides up. Well good afternoon everyone. My presentation today is concerned with different levels of quality at the levels of micro, meso and macro. And these are very interesting to mention to how we think about, ensure and measure quality in TEL. At the Open University we've had several discussions in our TEL team about how to measure quality and as you'd expect it became a difficult thing to talk about because the term quality is a very tricky one. We all think we know what it means but when we talk to others it becomes clear that not everyone understands quality in quite the same way we do. To be clear I'm not ultimately responsible for the quality systems at the Open University but as a TEL practitioner I have a special interest in the area as it applies to TEL. Let me just start with a quick poll. You'll see the options come up here now. So poll number one, how much do you understand about quality in TEL? And poll number two, how easy do you think it is to establish quality in TEL? I'll just open one and two, thanks Krisha. So how much do you understand about quality in TEL? The first one on the left. And how easy do you think it is to establish quality in TEL? That's the poll in the middle. Okay there's not too many of us in the webinar today but good to see some responses come through. Okay you're clearly very experienced quality people. Okay interesting to see how easy you think it is to establish quality in TEL. Very difficult, four out of five of you. Not too difficult for one of you. We all think we know about quality and that's actually a big problem. I'd like to start with just a few slides showing how complex any talk about quality really is. Quality is not something we will all agree on, not just in terms of how we measure it, but also how we think about how quality should apply to our work. So to begin with agile project methodologies interpret quality as something that emerges over time. Agile methodologies challenge what we mean by quality because what is good enough for now is thought to be sufficient. And this photo really illustrates the difference quite well. Historically quality was something built into a product before its single one and only market launch. The product reflected all of the quality decisions that were made as the product was reduced because once a product was made and marketed there was little opportunity to change it. Now things online are very easy to update so we can often settle for just good enough for now. We're more used to bug fixes being released after a product is in use and this changes how we think about quality because we can always fix what we did once we know what users think of it. We simply provide an update. This is something I think we need to consider quite carefully and tell because the principles of agile production fit very well with how we might choose to teach students. So if we think we know about quality, well perhaps there's a bit more to it. Consider too some different forms of quality. There are very different goals for tell quality and quite different approaches to measuring it. So as far as goals are concerned you may want to achieve some sort of certification for your institution such as an E-excellence quality benchmark. Certification is something you can use in marketing to help reassure students and other stakeholders that the quality of your work is recognized by an independent body. You could also benchmark yourself against other institutions using agreed criteria. If you do an internet search for the terms at the top of the slide in red you'll find two tell quality benchmarks and I'm most familiar with the A code one of those two. The benefit of benchmarking is that you can learn from other institutions about their way of ensuring quality. Benchmarking also requires you to think about your own performance and how you can improve. Accreditation might be another objective and here the goal is to assure an agency that your tell function complies with particular standards. These are usually national in scope but various professional bodies also insist on specific quality standards. So there are a whole host of things we might measure the name of quality. We might be concerned with the entirety of the technologies used for tell whether they're well looked after, well supported and make student needs. We could also measure quality based on how tell is used across different courses and we could consider quality in terms of tutorial sessions and teaching quality so we could consider quality to include all or only some of these. There are also very different ways of measuring quality. Surveys and evaluations will provide good data of what people experienced or thought about things and statistics and analytics provide different forms of data. Well you may want to participate in a benchmarking exercise which takes a lot of effort because valuable insight into the quality of what you're doing. There are also different times for measuring quality so before something is provided to students a systems audit might be done so that you know students will get a quality experience before they begin to study. Two options for measuring quality after a course is either developed or studied include checklists and evaluations. So as with the previous slide all of these approaches might be desirable. So what at first seems quite simple becomes very complex. In the very good 2015 report quality models and online and open education around the globe it said that there's probably no topic in education which is so discussed and generates as much controversy as quality and I think from those previous slides you can see why that's the case. You can find the report by doing a web search again using the terms in red at the top of the slide. Part of the reason for the controversy is that we all practice TEL from different perspectives. Some of you may be VLE administrators, other may be course designers some of you might be TEL designers or senior managers of TEL or perhaps so you might be responsible for all aspects of quality in your university. Some of you might be in universities that teach face to face and apply TEL to support lectures. Others might be in universities such as my one where a distance model is used. In all of these examples our understanding of quality and what's most important will be different. What I want to do for the remainder of this presentation is focus on a way of looking at quality that will be useful to TEL staff. At least it's very helpful to me as I consider what quality might look like in our TEL practice at the Open University. You'll have heard of these levels of micro, meso and macro before both from the Eden TEL Seq and the upcoming 2018 Eden Conference in Genoa. From my perspective considering quality from the micro, meso and macro levels makes a lot of sense and it leads to making good TEL decisions right across the university. Micro, meso and macro give us another set of options for measuring quality. The macro level is the highest possible level of detail the micro the smallest and the meso fits between the two. It's possible to measure TEL quality at all three levels and I actually think it's best to measure the quality of TEL across all three. So what do these levels actually represent? I think there are two ways in which people tend to think about micro, meso and macro. In the ICDE report I mentioned earlier the perspective taken is that on the right that of a quality manager. On the right you'll see that the relationships across micro, meso and macro are determined by the relative number of the personnel involved either a single individual, a group of people across the university or an entire nation. Clearly there are different quality interests that each of these will have. My preferred way of thinking about micro, meso and macro is shown on the left. At the open university we have less individual practice and more of a team focus on how we approach TEL. Each member of the team is responsible for the elements that make up a TEL based course or module. At the meso level these elements combine into a course experience leading to a qualification and all of this takes place in an overall context of practice some determined by the university and others by our external stakeholders. As you can see from the arrows each level influences the ones above and below it and of course there is some crossover. For the remainder of this presentation I'm going to describe more about quality from the perspective of the TEL practitioner so on the left of the slide. Before I get on with that I just another quick poll just the third question there down the bottom. Which level of TEL are you usually concerned with in your work? So three options there across the university at the level of courses or designing individual activities. So you just quickly indicate there. Okay so I can see there's some quite different levels here. We've got three of you across the university level one at the level of courses and one at designing individual learning activities. Okay it's very interesting as we proceed because I think you'll find this version of micro meso macro will be quite useful to you. So let's start with the overview of all of them. It helps to think of micro level quality as being concerned with individual assets, features and benefits which are like the parts that make up the whole. The whole is the meso level where all of the micro elements combine into a single product. These meso products or services take place in an overall context which is the macro level. So I'll tell you a bit more about what those mean as I proceed. But as an example you might consider this presentation. The micro elements include things such as your internet connection, our use of Adobe Connect, my slides and what I'm saying. All of these can be judged by different quality criteria. For example what I say should be at the right pace or speed. My voice should be clear. I should use words that are easy to understand. What I say should make sense. My slides should be clear. I shouldn't use too many. Colours shouldn't be too overstated. So at the micro level you can see that every asset or feature might have its own quality considerations. At the meso level all of these micro features combine into this overall presentation experience. So the entire presentation experience you're having right now is at the meso level. You can see that the quality criteria applied for this presentation overall would be a bit different than for each component. So here the quality criteria might include whether I go on for too long, whether you felt the session was worthwhile and your overall experience. The macro level is the context in which this presentation takes place. So it's a webinar for the Eden Tell SIG. Here the quality criteria might be considered differently. Eden want this webinar to be of benefit to its members, good for its profile and to generate interest for later. So to finish this presentation let me just pick up on each of these three in more detail using different examples as I continue just to make it clear. So I've introduced the micro level as the one where individual assets, features and benefits are considered and you'll see that I've applied two examples here. Your phone has a number of individual assets, features and benefits including its amount of storage, the number of megapixels to your camera and the number of apps available in your app store. Each of these must be of good quality in themselves. Similarly your car consists of a number of components that must be of high quality. In the same way and tell, each element of a course we produce in each online tool that we use should be recognisable as being of quality in themselves. So this includes all elements of a course from discussion for home exercises through to assignments and video assets. The ICDE reports quality models in online and open education around the globe mentioned earlier points out that most quality systems are designed for the MISO and macro levels. So it'd be a good idea to ask specialists at your own university to help develop micro quality standards based on their own experience and some research into media and activity design. At the MISO level the micro elements we've just looked at combine into a single deliverable be it a product or service. So at the MISO level we consider entire modules, courses or qualifications. There are many tools available for assessing quality at the course level but you may find that the best come from instructional design sources. So if you do a web search for open university learning design iceberg you'll find some work by OU colleagues that lends itself nicely to a set of MISO level quality criteria. It's a bringing together of a whole lot of lessons to do with instructional design which apply very well into telepractice. Finally at the macro level you deal with the level of practice and overall compliance. So this is the area best served by existing telequality models. It is also where telepractice meets overall institutional quality codes. From our extended examples you'll see that phones must comply with technical standards including battery regulations, file type standards and network protocols. Each car needs to comply with safety standards and highway codes. In the same way we're not usually free to practice tele with no responsibility to the university. Some accrediting bodies, for example here in England the British Psychological Society also set standards that we must comply with at the macro level. One of the standards set by the British Psychological Society is that there must be a minimum staff student ratio of 1 to 20. This is part of the compliance that determines how tele might be used. Your country's quality assurance agency for higher education will have other very wide reaching requirements that your telefunction must comply with and you'll also have institutional policies and strategies that will shape your practice. There are many tools available for determining telequality at the macro level and I'll refer you back to the ICD quality models report using the terms at the top of the slide for a web search to get an overview of them. So where does this leave us? Well a lot depends on what your role is in your university. In general I suggest that you do these things as telepractitioners. Firstly, take an interest across all three areas of quality because all of them are interrelated and so all are relevant to what you do. At the macro level I suggest you learn as much as you can about your university's quality systems and obligations. Not just the tele ones but the overall ones. As a further step I suggest you take a look at the ICD report mentioned earlier and select a quality model that you might use to certify or else benchmark your telefunction. This will give you confidence about your practice and will likely lead to areas of improvement. I also suggest that you give some thought as to tele at the meso level. I suggest that the Open University's iceberg report is providing some great research based ideas for applying telecourses in modules. You might also want to research your own quality standards at this level. Finally don't forget the level of micro assets because each activity a student does each video they watch, each paragraph they read should also reflect quality. It's a useful exercise to consider what criteria you might develop to ensure that each asset is produced to a good consistent standard. So there's a lot to consider when it comes to quality and tele and by considering it from a micro, meso and macro point of view you'll be able to address all quality matters. So do take an interest in all three levels. I hope the presentation has been helpful and I do welcome any questions. Thank you so much for this very interesting presentation and the overview about the different levels about micro, meso and macro level. Are there any urgent questions right now in case we can take one question otherwise we leave the questions after all the presentations. So please if you have a question you're writing in the chat. I can't see that anyone is writing so we continue and take the questions in the end. Thank you. So Mark also talked very much about that quality can be looked at different kind of levels and so what's in it for me. And he also mentioned the report from ICD which I'm also the research leader for. We looked at, I will also mention this report because it is a report which is really up to date and it has a global perspective about quality in open online learning including technology-enabled learning and also about exactly about micro and meso and macro levels. We have written a lot about that. And what we did was that we studied over 40 plus quality models as you see on OER MOOCs, eLearning, online learning, TEL and those models were representative from all over the globe so from more or less all continents. And what we did was also to discuss a lot as I said about micro, meso and macro level for those quality models. And as Mark was saying maybe most of the models are on macro and meso level and not so much about on the course level or the small models in the course for example. But we also looked very much about norm based or process based quality enhancement because that is also a huge difference. So from all those 40 models we did a quality matrix. What kind of model were all those 40? Were it more like benchmarking as Mark was mentioning? Was it more about accreditation? Was it more about a guide or a framework? Or was it about certification? And that is also a huge difference and one have to consider for what purpose are we discussing quality and what will we do with it when we are going through the process? For example if we are going through process with benchmarking. It is very much about internal work and to make things visible within an institution and also if you benchmark with other institutions to get things on the table so to say. But if you are going more for accreditation it is often a national or maybe even international body who are setting the criteria and also the standards. So that is a huge difference. And it also is very much about how material institution is. So we also discussed quite a lot about the stakeholders. Who have an interest in this? Because as I said and also Mark was pointed out quality is in the eye of the beholder. So it depends on who is deciding what kind of criteria we are looking at. So you are responding and when? Is it immediately after a course for example? Or is it after one year? Or is it maybe measuring how people are getting employed after courses? And then we also came up with a lot of recommendations. Maybe not a lot but I think there were some 12 or 13 recommendations for different kind of stakeholders both for institutions but also for organizations like ICD and also for quality assurance agencies. So the report is also mentioned available at ICD's webpage and on my slides I have a direct link as well. So you can just take that link and you can see what it is about. And the report is in two parts. The foot report, the summary for some six pages if you don't like to read a whole report and then the appendix where all the models are described. So I would also like to argue that it is very helpful to look at both micro level, meso level and macro level. And of course different kind of stakeholders have different kind of interest on which level they are looking at. But I will argue that all levels have to be considered because the saying quality is not as strong as the weakest link I will say is very true. Maybe you have very good strategies for example, maybe you have very good everything is in place in macro level but then on micro level for the learners for the students, everything is a mess. So then you can't say, you can't measure that you have good quality. So it is really important to have a look at all three levels and it is really helpful I will say if you also divide it into this because then you get a better overview. But it is as Mark also was stressing that you need to see how they are interrelated or maybe where there is a gap. Because then you can start to work where there is a gap or you can enhance where you have your strengths. More about this is written in the report as I said but also in the work by Capewood and Price from Commonwealth Learning and you also have the links on this slide so you can go directly and have a look. I mentioned that we also talked about in the report about norm based versus process based and that is another way of looking at it. Is it more norm based accreditation or quality enhancement? It is more about an accreditation from an authority, often an authority from above so to say, national level, international level, sector specific area for example. Or is it more that you are looking for processes and it's more like a benchmarking, a self-evolution. And I would really and strongly recommend if you haven't gone to none of the processes start with the benchmarking and start with a self-evolution because then you can see where you are at. And there are several models where you can just do an example, the Exelance model. You can do a self-evolution online. You can do it by yourself, you can do it with colleagues in a team and then you can see where you are at and where you have to work further on. Certification is something more in between. It is often by associations like ICDE, like EIDTU for example, like sector specific organizations who are setting the criteria. So again it is very much due to what you like to achieve and what shall you do your quality work with, how you are going to use it and what will you do about it. Then we of course have the stakeholders and those maybe have a different kind of perspectives. We have those stakeholders that are used to be mentioned, the learners, the academics, the faculty, institutional, region, national, the country and at international level. So of course all those stakeholders have different kind of perspectives and different kind of demands and needs when it comes to quality in TAL. And also as was mentioned both by myself but also by Mark that they also have interest in different levels. That is why it is very wise to think about and to go into all levels, micro, meso and macro. Because learners maybe have more interest in micro and meso and at international and country level maybe they have more interest about the macro level. So stakeholders need to be involved and need to have an opinion about it. I will briefly mention some very good references. This one by Duvala Sharps and Sutterland, Technology Enhanced Learning, which came this year. The other one I have already mentioned and that is Welcome to Learning by Kate Wood and Pryce. I have also the links directly here in my slides and I would like to have a further look at it. I have some questions but I will save them till the end. And here are my contact details. So I would like to now move on to Arina's presentation. So welcome aboard Arina. Hello everyone, I hope you can hear me well and if not and if the quality starts decreasing I will switch off my camera to improve the broadcast. Okay, thanks Ma. So my decision today is to talk about the quality enhancement of Technology Enhanced Learning on micro and meso levels. And my concern actually is the links. The links that help to identify sometimes quality criteria groups, sometimes responsibilities within an organization who should be discussing. Of course I understand that all stakeholders are discussing all three levels, micro, micro and the middle one, the meso level. But it is useful to address all of them in detail and that is why I will introduce to you the micro and meso levels but in the context of the links among all three of them. Actually we have been doing research on quality issues in even professional community and also in Widotas Magnus University. Our first research in the format of dissertations came out in 2008 but we continued and currently we feel the need to help organizations who want to uptake Technology Enhanced Learning in broader scope in mainstreaming to help them to get oriented and to help them to prepare for Technology Enhanced Learning services. So there was a good discussion going on among actually several researchers who discussed what is responsible and responsive integration of Technology Enhanced Learning within an organization. And this is definitely the macro level and I am not going to stop on that but I will use it in the form of introduction. So the seven areas of organization activity have been identified that you can see in the very middle of the model which is strategy and management, IT infrastructure, telecurriculum and programs, staff continuous professional development, support systems, quality assurance and marketing and business. And these areas seem to be at a very macro level. However when we come to discuss them we see clear links with establishing the preconditions and the same way as preconditions are usually established within a country or a region in terms of the policy of education, in terms of infrastructure, internet, mobile applications, other things, tell demand. So the same way we see that the infrastructure in the organization affects very clearly micro and meso levels of quality as well. So the colleagues that made introduction into the macro level just confirmed that I think. So if we go to the seven areas, the seven key areas of organization activity that you just saw on the slide in the previous framework, I think might be the good start to discuss which of them have direct links with the course level, that would be like meso level from the reference in Mark's slides and which of them would go to micro or activity or a resource level. And definitely we will find several links here and I will present them shortly in the slides that will follow. If we pick up only one activity area which is quality assurance and I think universities have dedicated units, departments who deal with quality assurance issues. It would be for both traditional or online technology enhanced learning. One thing we all are obliged to do in Europe is to work following quality assurance standards and the network of quality assurance agencies usually good reference bodies for that. So whenever we start discussing them we already situate ourselves in the vertical dimension of quality issues and we go deeper inside. So first of all each organization that has education organization which is education provider Intel actually needs to have regulations on organization of online studies or technology enhanced learning in the organization. So quality starts here. As an example it could be like quality assurance group established that agrees upon the criteria for the meso level objects for the courses. It could be also modules as Mark suggested. Therefore the group may wish to establish or to pick up existing quality criteria, quality grids and adapt them for the purpose of an organization. It might also be the case and I personally go always for this process of quality enhancement and process of quality assurance instead of assessment to have the group that is organizing peer reviewing and peer reviewing is helpful not only for the teachers who are doing peer reviewing and other colleagues who are involved in this process but actually peer reviewing I would say following my experience is one of the best instruments to develop community in understanding quality culture and developing quality culture within an organization. So actually if you recall previous slides and if we have the activity of quality assurance as one of the key activities within an organization of education providers we are already addressing I would say macro but very often also meso levels. I don't know how clearly you can see actually the scheme inside but it says that we have always challenges in discussing quality assurance especially that when we start considering subject matters which are sometimes too specific for one or another area of learning or area of studies then we have of course teaching and learning situations that occur during teaching and learning process and those who are already experienced in TEL we know that we probably never prepare for them. We experience new situations every time that is why when we speak about technology enhanced learning curriculum designing and when we discuss the dimensions and criteria quality dimensions and quality criteria we always think that we leave a big percentage of unpreparedness in front of us. However what we may prepare for and what we should discuss is actually the consistency of learning strategy dimensions so that learning outcomes or learning objectives are consistent with teaching and learning methods and then assessment strategies and learning assessment of learning achievements. Then we should ensure experimental validity and creation of real-time context learning situations because we already hear quite often that theory and practice go apart so we need actually to link this together. At the same time you see if you are able to see the further dimensions say that epistemological, cultural, philosophical and psychological dimensions should be also taken into consideration while considering quality assurance factors and quality assurance criteria that we might wish to accept in our organizations. Metacognitive learning dimension which actually goes as direct support for our learners it's very nice that recently we have a lot of suggestions and ideas how different tools and virtual learning environments may support us in that, for example applying learning analytics and of course technological dimension which leads to application of correct technologies in order to enhance learning instead of bring obstacles for teacher and learner. We have different examples of criteria grids that have been developed, that have been experimented and validated also with an Eden community. You have here the reference from one of lifelong learning projects very practical one for everyone can be reviving with practices. Again if you look at the title it says that we usually are inclined to do our peer review and self-assessment applying these criteria so that we target at improvement and development of what is already available to better meet the needs of our learners. And there are grids developed that should be as far as I understand should be based on the dimensions quality dimensions agreed. So whatever dimensions are chosen by an organization or researchers working within an organization they should be very consistently described in terms of criteria and suggested in a very simple and clear way so that actually teachers who develop technology and enhance learning curriculum could apply them. And that is I think already common truth that if we develop sets of quality criteria in the format of grids or other formats we can easily identify the developments themselves. We may want to insert new criteria we may want to initiate discussion on existing quality criteria we may want to review them and be open towards their development. Yes learning objectives or competencies and learning outcomes in higher education and that presentation of theoretical material all these details matter a lot actually. We have them I think on both micro and also meso level. Sometimes we get annoyed or dissatisfied with some of the characteristics that we identify in the course. We might think this is a micro level. However this criteria described on a meso level on the course level will help us to prepare better for these smaller pieces and fragments that we should identify now as micro level. So actually the examples that I am showing to you right now you may found at the website you see the reference here and they are adaptable they are editable they are published under creative commons licenses for non-commercial share alike and attribution license and they already are widespread in different projects in Indian projects and other projects all over Europe. They actually enhance peer reviewing among institutions and teachers for quality enhancement and whenever we start cooperation with other organizations we want to compare quality criteria for meso levels. We want to compare also criteria for micro levels and to see how they are integrated into our common understanding on quality culture and how we may prepare to be more open and organize peer reviewing between and among education organizations. So different deductical methodologies have been analyzed and research has been done until we came up with the selection of quality criteria. However on micro level we are always prepared if we are detailed enough in meso level quality criteria and then when we have activities we may also find that consistency is in there when we present quality criteria for activity or assignment description or when we introduce quality criteria to teachers on how to design learning objects and learning resources and open educational resources then we definitely continue developing also quality and enhancing quality on meso level. So this is the link that I wanted to demonstrate to you between micro and meso level. Also I wanted to share with you information that quality criteria, the sets of quality criteria exist. Also quality is discussed in terms of criteria but also in terms of factors that might affect quality in teaching and learning. Different methodological examples exist but again one of the key messages is that actually we design curriculum for technology enhanced learning but then when we teach especially in technology enhanced learning environments we experience unexpected situations and I think it is also one of the important area to be discussed quality assurance criteria for teaching in TEL not only designing for TEL. And of course if we have different initiatives in Europe we need to adapt, we need to sit down and discuss if we may apply quality criteria, quality factors to new phenomena equally well and I think not. Each innovation, each phenomenon should be reviewed in terms of scenarios that it goes through. Now I'm showing to you an example, for example of virtual mobility quality handbook which would be different if we have different scenarios for teachers, students, multilateral and for the scenarios of implementation of virtual mobility. So now I see that we are coming already short of time so apologies for taking one or more maybe minutes longer than expected but I complete here and give the floor back to you Eber. So thank you so much Irina for your very interesting presentation also. Yes we are a bit short of time but I will still invite questions because we have some minutes left. Maybe in overall do you think it is helpful to have this distinction between meso, micro and macro level and to see the links and to see the gaps. Can it be helpful when we discuss quality as we have all the three of us pointed out that quality is such a complex concept as well as tell us what a complex concept. I see that the Eleonora and Irina are typing. I'd like to respond to that question Eber. I think all three are very, very important. I like the way that Irina just demonstrated how all three can be combined together into one overall quality approach. I think the point was made several times that different stakeholders have different expectations of quality and considering all three levels I think you do get the broadest possible view as to how our quality is working within the organisation. Yes I will also very much agree on that because different stakeholders have different kind of approach to quality and also approach to different kind of levels. So that is the other dimension which comes in. I'm trying to see what Eden is saying. Thank you Eleonora for your feedback. You were asking about MOOCs. For example for MOOCs there are, yes to you, the European Association for the Strategic University have come out with a very nice both self-evolution but also benchmarking model for MOOCs and that is the only one at least in Europe which is available now and I think you can have a quick look at that and to see what it is about. I think there are seven features especially I'm looking at and that gives an helpful overview how you can look at MOOCs for example in your case. Well I think all of us are sitting with a lot of questions maybe after this webinar and there's a lot more to discuss. We will continue with a special interest group of talent, quality and hand spent in different ways. We have our web page. We will see if we can maybe have some kind of activities to that maybe like blog posts or whatever. We have also more webinars. We have the next webinar, the 7th of December and at that webinar we will more discuss. I think you can get into that discussion also. There are new things coming up all the time and we need to measure quality at the current state of the art so to say. We can't measure quality with old models. So we need to have some kind of agile method to do that. And then we also have a tweet chat the 3rd of December and we in the core group we invite more people more colleagues to be involved in this area which is really an important area at universities and educational organizations. And together we can do a lot I think. So I'm sure we will continue in spring as well I see some suggestions about more that maybe some teachers or academics can see how they are looking at quality and their particular course and to give more concrete examples because I think for today it was more like an overview of how to approach quality-enabled learning intel. I see Renata is coming in as well. So for all of you who have registered the link for this webinar will be available and sent to you and you will also get a badge for your participation and the link will also be provided to the web page for a later web page. So by that I will thank you very much all of you who have participated all of you who had contributed and of course very much to Mark and to Irina who has also contributed and to Eden who have helped a lot with the dissemination of this webinar and the technical aspects. Thanks a lot. So have a nice afternoon.