 I sent my presentation to Cristina, it is updated now, so, hello everyone, okay, okay, wait a second for Cristina, we are uploading the introductory presentation, and then I leave the floor, thank you so much Cristina, because I changed it, so let me know when you want me to start, okay, so thank you. Thank you all for being here, we are starting this Eden NAP webinar series today, and I just wanted to introduce the Eden NAP Network of Academics and Professional, which is a group, I would say, a community, in the community of Eden, European Distance and the Learning Network. My name is Antonina Lapocha and I chair the steering committee within the Eden NAP group. What is NAP? I think it's important to make this short introduction, Network of Academics and Professionals, it is a network that within the association supports networking of individual members, it supports communication interaction among professionals and academics interested in distance education as a wide concept, it is coordinated of course by steering committee that I chair as I told you. We work on providing information for professional actions, we help and we wish we could help more actually members to create a personal portfolio, we promote communication as we said, and we also help finding partners within our membership, the Eden membership. This is the group so that you can have those faces to the names of the people you might find online on the Eden website, but also on social media. Alfredo Soeiro, Elba Tornar, Elga Tornar, Wendy Chawi, Don Alcott, Elspeth Corgard Sorensen, Alice Greelman are all members of the NAP steering committee and we all work together to support communication and interaction among the group. Here you have also the website, the link to the website area where you can interact with members, get information about other colleagues and join the community. Which sort of services and benefits the membership could get in being part of the NAP group, of course being part of the community, access this database of institutional and individual members and partners. As a member institution you can delegate up to 30 individuals in the NAP. You can attend the conferences with a slightly reduced fee, establishing partnership, get access to the proceedings, use Eden logo, you know, having relevant newsletters access, and of course have the possibility to access with discounted price to our publications. Again, this is a slide that shows you, you know, how to access and what is present on NAP members area. Saying that, of course, we work as I said from a communication point of view, so we support and we try to use social media to improve such an interaction as you can see through Eden chat and webinars like today. We support professional development, we improve, we try and we wish we could improve more and more our website. And we start and we promote webinars of common interest. We try to listen to our members ideas and so that to support the interaction I was mentioning in order to promote new projects, new possibilities for research. Here you have some links to our channels, communication channels, and as I was saying, today we start this webinar series and today we start with very important, I would say, I'm sorry I'm missing one of my slides, I don't know why. Anyway, we are starting with a very interesting and relevant contribution from our colleagues from Vives. The organization is promoting actually and will be the place where we are going to have our annual conference in June, so this is a way to support you participating in our annual conference in June in Bruges. I will give you just a brief introduction related to our hosts today, just a few words related to their background. We are going to talk about evaluation and which is a really central issue in distance education programs and they have been working both of them on technology in education for a good time now. But anyway, let me introduce you, Andrew Cook, I don't know if I pronounced correctly your name, but maybe you can support me later. So it started at Vives as a lecturer in mathematics in 2004 in teacher training for the bachelor primary education program. Besides teaching is also an educational technologies to support teachers in the use of educational technology in their teaching practice. It's currently part of the Vives educational technology team, part of the Department of Educational Policy, which gives concrete form to the vision on educational technology for the entire University of Applied Sciences. From the latter position, he started to specialize himself for more and more in digital tools, including specifically the Edumatic Assessment Tool, which has now become Assessment Q. Cohen Verrest is working at Vives since 2000, originally as a teacher ICT and multimedia in teacher training for the bachelor primary education program. He has been working at the Education Policy Department since 2014. The goal in 2014 was to increase the flexibility for distance learning students by establishing the examination center together with the development of square and implementing proctor exam as a tool for remote examinations. His role in this was mainly to design and develop the square tool. Today is responsible for the management of square in addition to the management of a number of other tools. So I thank you both for being here with us today and I leave you the floor for your presentation. I hope participants can write down their questions or comments in the chat area so that in the end if we have questions we can ask them to you and support the interaction we are looking for. Thank you very much. Thank you for the introduction, Antonella. I don't know if it's possible to change to the PowerPoint that we have given right now. Okay. Thank you very much. So, hi. Welcome everybody. My name is Hendrik Kukker. It was almost pronounced correct. And this is my colleague, Cohen Verrest. So, as you said, we are both member of educational policy department in Viva since 2014 and especially in distance education, we work together and use some specific tools. Today we want to talk about evaluation in distance education and as you may be able to see, but we only have six slides because we want to show the tools themselves with some examples and how we work with those tools in Vivas. First of all, before we do that, I would like to make a short introduction about how distance education in Vivas has developed. Actually, we started over 20 years ago in 1997 with the first form of distance education in Belgium. We were pioneered with Vivas on that and it was actually for about seven students who wanted to become a teacher in primary school. So, the difference with the other students from 18, 19 or 20 years old was that those students were older. They actually had a job yet. They actually got a family and they live far away from the town where the courses are normally given. And in Belgium, 50 kilometers is very far, so we wanted to look for other tools, other ways of organization for those students because they can't be guided in the same way as other students are in their program. Now, today, over 20 years later, we have about 30 programs in distance education. They are divided over five study areas as healthcare, applied social studies and education and so on. And that sets a count of about 2,300 students and that's quite a big part of our student population. As you see, about 15% of the total Vivas student population follows any course by distance education. So, for those students, we want to think about the organization in general, the tools we use to give courses, the tools we use to organize assignments and of course evaluation as the focus for today. So, there were some things during the growth of Vivas on distance education, some problems that we still have. And the one is that in a lot of courses, students who follow a distance education program are still expected to get through the campus town at some fixed times. For instance, to take exams because in day programs, sometimes in January, in January, you get two moments that a student can take an exam. But students who want to study with distance education, why can't we let them choose a date as they like? So, if they want to come next Thursday morning and take a specific exam, then why can't we give the opportunity to the students, why can't we give that? So, that's one goal that we have, a problem that we have and that we try to solve. And the second, when a student lives 100 kilometers, 50 kilometers, 200 even abroad, because we have some students who live abroad, can't we be able to let them take the exam remotely? So, when they stay at home and they take the exam. As a fact, about 10, 12 years ago, we started experiencing with that. I don't know if it's said rightly, but we use this tool Adobe Connect where we are right now. But we started looking for other tools and so on. So today, we want to show some tools that actually helped in the organization of the evaluation for distance learning students. But in about, I guess, 20 minutes, I'll show the tool, the assessment tool that we use to organize that. But first of all, I want to give Kun the possibility to say something about Square and the examination center that we have here in Vives. Thank you. We will actually change the image because we have to share our screen. So I go to this environment. Can everyone see this? Yes, okay. Thank you. Good afternoon. So now I will take you to Square. This is, obviously, right now it's in Dutch, but I can change this into English if we want. We'll have to log on. Okay, so Square was, we started developing Square in the year 2014-15. And as Hendrik has been saying, we wanted to gain flexibility and we had the assignment to start up an examination center. Vives counts several campuses and on each campus before 2014, we had some programs that offered distance learning. But of course, each campus, each city from Vives had to organize examination moments, which was quite a lot of work. And so we've started up an examination center here in Vives-Kortrijk, where we put all the students that follow distance learning together. I hope I'm being clear, but I'll show you the view of a student that will make it somehow more understandable, I think. So what I'm going to do is I'm going to open the profile in Square in our software. There we go. So if a student logs in into Square, this is what he gets to see. It's quite easy. There aren't a lot of buttons. We've made it according to the KISS principle. Keep it simple, stupid, so it's not very hard to learn to work with Square. The student has one main button he can click on and that shows him the program of all the courses that he is taking. Of course, this is our test student, so there's only one course that is showing. And within that course, we see all the assignments and all the exams the student has to take for this course. For instance, we see here an example of a written exam, a digital exam, two oral exams, one and two, and then we have three assignments that the student has to upload, has to deliver in Square. We can also see immediately that there is some feedback. Currently, assignment one has been delivered and it's been evaluated by the teacher of that assignment. So the student doesn't get to see his actual score, but he sees a letter and if he hovers over it, he can see what that letter stands for. So in this case, in this example, our student hasn't done his work quite well. He scores a 7.49 on 20 or less. So that's not so very good of the student. If we open this assignment, I still see as a student what the assignment was, what I have uploaded, and just underneath that I can see the written feedback that I've gotten from my teacher if he has given any. So I'll return to the written exam. A student can subscribe for an exam or unsubscribe for an exam up to five calendar days before the exam moment itself. So what I'm going to do to show you, I'll unsubscribe in this case and then being that same student, I can start subscribing for my written exam. I have two buttons. The first button shows me when the exam can be taken. So you see in this case there are a lot of dates and times. We have two categories. You'll see it if I get back. So this can be taken in the examination center or this can be taken remotely. I'll get back and I'll make my subscription once again. There we go. I'll make my subscription and let me say I'm not going to take the exam on distance remotely. I'll take it in the examination center so I'll click that and then I can really subscribe. If I click on this button, I get to see when this exam is being organized. So you see there are a lot of moments in this example. This is a choice that the teacher or the curriculum responsible is going to make. How many moments are we going to open up for our students to take the exam? So in this case, being the student, I would like to take my exam on the 30th of March and so I click a green block. Everything that is green is clickable, is usable for me. And so right now I can see I've registered, I've subscribed for my exam on the 30th of March 930. What happens at this time is that I get an email confirming my subscription so I can be sure that I'm being well registered. And if I get out of the view of the student right now, I will show you or I'll try to show you our exam room, the examination center. And I'll open one date here, let's say for instance the 19th of January. This is a Saturday as well so I'll zoom out a little bit. This is the whole room, the whole examination room. And so what we see is that all the red chairs have been taken. Our students that have been registering to take one of their exams. We also see some blue chairs right here. Those students have special facilities to take their exam. For instance they have the right to work 20 minutes longer on their exam. That can be something or they can use a certain type of software to help them take the exam if they want to. The red chairs in the back, the orange seating places are reserved for digital exams. Those seats have a computer so they can take a digital exam. And here once again we have another color, purple. That shows us that this is an exam that will be taken on a geometric or assessment view as we call it nowadays. This will be explained and demoed by Hendrik within a few minutes. What I wanted to show here, so if I open a certain seating place, we will be able to see, it takes a little time, that this student has a certain chair that has been given her. We see what exam she comes to take, what version of exam she's going to take, how long the exam can take maximum so she can work for three hours. And what's very important in Square is that the software itself will be placing the students. So if I'm a student and I register for course A, exam mathematics let's say, and Square places me on the seat I-10, well next to me, right before me, right behind me, no one can take place, will be placed there who takes the same exam. So that is something that's a part of the intelligence that's built in Square to help us make sure that the exam, the examinations will work without any possibility to fraud the exam. So that's quite important. Next to that we have a clock that we can set, but of course this is a date that's in the past. The two people that are responsible for this examination room can start the exam time and so Square will help them to say, okay, we've been working for one hour and a half. The seats G-13, E-12 and C-11 have to hand in their exam because they only can work for one hour and a half on their exam. And so the clock will be following automatically which students have to turn in their exams. So that's quite a big help for the people that do the proctoring in this examination center. What we also have in Square that is built in is that every exam that has been taken is logged. Of course, that's quite logically, but more important is that there's a version management system built in. So if a student should fail for an examination and he has to take it again, let's say in August or we don't know when, the software will be checking, okay, the first time the student took this course, this exam, we have given him version two, let's say. So next time he takes this exam, there is no way that he can get version two again. He will get version six or version 12 or any version that is available. So the software lets the teachers upload their exam versions if ever there's a version needed because a student takes it for the second time and there's only a version available that he's already taken. And Square will email this teacher, will ask him to upload a new version by a certain date, a certain time. That is in short what Square is doing for us. So we have the seating plan, we have the intelligence not to place students with the same exam on places which are one near another and of course the diversion management. We also get the assignments that are posted, that are turned in by our students and we get the possibility as a teacher to give some feedback. I'll give the floor back to Hendrik who will now show a second tool being idiomatic assessment queue. Okay, thank you. Are you all still there? Okay, I'll try to tell you something about idiomatic. Maybe I'll try to go back to... I don't know where I need to... Okay, I go back to the PowerPoint. So here idiomatic assessment queue. It was said earlier, idiomatic is the old name of the tool, assessment queue is the new name because they are rebuilding the system with a lot of new tools but it's also more improved too. First of all, just like other systems to give digital exercise to teachers, idiomatic is an online system where you can build as a teacher a database with all sorts of exercises, feedback possibilities and so on. There's a possibility of a various range from about 20 different question types. Of course the logical ones like multiple choice and so on but there are some powerful question types too that I would like to show you later on. So next to a database where you can put exercises in, you can make from those databases assessments, exams that are being generated by the system. And you've seen it earlier in Square. A student has the possibility for instance to choose between 100 or more different moments in a year to take one specific exam. So for the teacher it's very important that he will be able to generate more equivalent versions of a specific exam. So when he built a scenario, a predefined scenario in idiomatic and assessment queue, the system can generate an exam based on that scenario. That can be for instance giving you three exercises from that folder to from that folder and so on. Secondly, the online assessment of course results in some reports of the exam result so you can make your exam better. As we can speak maybe there's a question that wasn't very good so we can update the question for the next time. But what's also been created last year is that there's a link between Square and Proctor Exams. Proctor is for later on but there's a connection between those systems. Now when a student subscribes for a specific exam on a specific day, on a specific moment, and it's a digital exam that has been made up in assessment queue, there will be automatically being scheduled an exam in idiomatic for that student. So there is no teacher that needs to plan a moment for the student. The system by the connection between the system does it automatically. So I'll show you idiomatic and I'll show you idiomatic from the side of a student. So I go back to the next and I start sharing my screen. In fact, I think we lost connection or Andrew actually seems to have lost connection. In the meantime, I can, yes it will be back shortly. In the meantime, I can start announcing, first of all as I was saying, the conference in Bruges, where I hope you all will be, that you all, I hope you will be attending. During the conference, we are going to have a workshop devoted to the possibility for different groups to interact. Here is the slide introducing our conference in Bruges, to be held in Bruges in June 1619. The title connecting through educational technology. So being that the title, but you know, it is very consistent with what we are going to do during the conference as an app, as a network for academics and professionals. We're going to offer a devoted workshop where different groups from different institutions, from different countries will have the opportunity to meet actually to introduce their own work, their own research interests. We will organize the meeting as a sort of speed dating event. So every group will have the opportunity to meet with other different groups, different research interests and find connections and find the possibility to start new researchers, new developments that maybe could hand in a further presentation in the next Eden conferences, events scheduled for the following time. Now, I don't know what happened to Andrew and the previous group. Maybe the secretariat can help me, but something might have happened with their connection, I think, because our own is working perfectly. So I don't know if Christina can give me any information. I have seen that there is also, there are some questions there, there are some questions there. They are related to the kind of assessment items that are used in the system presented by the previous colleagues. And when they are back, we are going to ask them. As a group in Rome as well, we are working on assessment and actually technologically supported assessment, which is a major issue, not only for, you know, reliability considerations, but also for validity consideration. We don't have such technological developments as regards open-ended questions and this is a major issue, especially in such a high-level teaching and learning environment like the one we deal with at university. There are some skills that needs to be developed, needs to be supported, that are not accessible, actually that might not be assessed through closed, only closed items. And that's why open-ended questions are needed, especially while you want to develop cross-sectional skills like critical thinking or other skills like, for instance, argumentation and so on and so forth. What we can do now in the meantime, while our colleagues from Vivas will be connected again, is to try to have a little discussion among participants related to your normal way of assessment. What kind of exam system do you use in your environment? Do you use only closed items? Which kind of items? Do you use which kind of platforms are the environments that you use normally? Let's see if any of you want to give us, you know, contribution related to these aspects. Maria is saying that we are still very traditional paper-based, only paper-based. So in our university, especially in the department where I'm working, which is the Department of Education, where in Roma Tri-University, in Rome, we're having lots of students. Actually, in our department we have more than 6,000 students. We have actually to work on computer-based assessment because, of course, of these large, large numbers. And we used to have a paper-based system, but then we had to skip to the computer-based one in order to solve the issue of, you know, the numbers, the huge numbers we have. And, of course, the issue also of actually marking, of course, exams. I see Natasha, you use Mudo as the same platform we are using in our university. I think it is widespread. The only problem, as I was saying, that we wish we could overcome is to find an automatic system for open-ended questions, which is still far from being available. We are trying to work on research on that. So if you wish to participate in some research to gain, to reach this same, please let me know because we would like to start new projects on automatic assessment in open-ended questioning. So, again, traditional paper-based assessment marking of the paper-based examination. Yes, this is a good question. I think they read and mark the papers. Am I right? By hard-working, but that is truly hard work. That is why we skip to computer-based. Actually, until last year we had, you know, a sort of medium-sized automatic assessment using paper-based and machine that was reading automatically the papers. I don't know if you have the same, Marielle. I don't know what happened to our colleagues in Vivas because they are not answering. Yes, Marielle, I was asking if you, what do you do when you collect the paper-based exams? Do you use a system to read the paper-based or you just mark it with, you know, human raters? Marielle is typing. So, computer for scrap cards, okay? Computer for my question. Open questions by real people. In fact. In fact, that is the issue, the problem of open-ended questions is a major one. We, as I said, we are trying to develop an automatic, I see, it's very, very time consuming. We are trying, as I was saying, but if we meet in Bruges and hopefully we will on the workshop I was telling you devoted to interaction, communication, exchange of ideas, we will have surely the time to introduce you to what we are doing in our department to develop a system, an automatic system to assess open-ended questions. It is very difficult because it is almost impossible to replicate the human brain, but there are already some levels of possibility, at least to reach certain objectives. It will be interesting to have an interaction on that. Oh, so maybe they are back. But anyway, this was a good opportunity to have this exchange on assessment, open-ended questions, and common sharing of ideas on the possibilities of development in automatic assessment. So, here you are, Andrew, you are back, finally. Have you seen the automatic tool, the question types or not? Have you seen everything? If you start from there, it's good, from the questions. I'll go right on, so I go here. Kuhn will enter the room too, to check if we are still online or not. So actually, I go back, I just go very fast through the questions. So this is the environment, where we can set up different questions. And a lot of questions are very known. Also, in other systems, you have questions like multiple choices, like this one. So, where you just answer one right question. And the system can give feedback, give correction, if you want to ask to. Of course, in an exam, those buttons will be invisible, of course. But I'll show you around eight questions, very fast question types, but there are about 20 different question types. But I'll show you some that I think in other systems are not already there, or not in that quite of manner, like in idiomatic. A new question type, you see, I just reset the question, is drag and drop in a column, like for instance, you have some sentences and they need to be dropped in the right column. For instance, will your system come tomorrow? That's a questioning sentence, so I drag it here tomorrow and then sit still is a commanding sentence. And then I can check, okay, that's correct. So far so good. Not very new, I suppose. Also, drop down in text like this. So, you can choose the right answer from a drop down. Also, in other systems, you see those kinds of question types. This one is a question type where you have to answer, where multiple answers can be right. So, for instance, in which countries is the euro used to pay? In Portugal, Slovenia and Austria we do. So, I can correct like this. So, that's very simple. As you can see above, I can flag a question. When I think as a student that the question is quite difficult, I can flag it. So, I see in the menu that that's a question I need to look at at the end of the exam, maybe to check if I'm certain that it's correct or not. Then this one is quite a beautiful one, I guess, a crossword in some other systems you have there too. Actually, when you click, for instance, on capital of Latvia, it will show where you need to answer. So, in this case, it's Riga, then capital of Turkey, and I can just type and the system fills in all the letters. Croatia, that would be Zagreb. So, we see that in four capital of Belgium, that's Brussels and so on. Fill in the last one, capital of Belarus is Minsk. So, and you can, every letter can have a specific score, but you can also make one word a specific score. In this case, when I correct, you see I get six out of six. I get the six. Oh, it should be five out of five. I don't know why it's mentioned at six. I don't know. That would be a fault from that I made while I set up the question before. So, that's a crossword you can add. Now, this is something that our teachers, our language teachers have asked to create those question types. Actually, when you get a text, in this case, I have two sentences, and I would like to know if the students can indicate all the adjectives in the text, all the nouns, all the personal pronouns and so on. It would be great if we can do that by marking the text of those words in the text. You can work to a maximum of five categories in each question. So, in this case, we have three. And now, as a student, if I want to mark all the personal pronouns, I just click on the marking button and then I can mark every word that is a personal pronoun. Every noun, I can also double-click, as you see. And then, the adjective, I can like this. And then I can ask for the correction, and you see I get three out of three. In this case, we made one point for each category, but you can also give one point to each word, that word that has been marked. So, those are things that are very powerful, I guess, that I personally haven't seen in other systems right now. So, marking the right words. Another one is the possibility for the student to answer with a specific mathematical symbol or structure. You see, now, the teacher has only given the possibility to answer with a fraction. But there are several possibilities, like difficult ones, like integrals and some other structures like this. So, when a student needs to answer this formula, this addition, so he can click the button over here and then click on the structure to open the numerator and the denominator, and then he can fill the right numbers in the cells. The system can check and that's correct. So, those are also possibilities in idiomatic to create your questions with. And the last one I want to show is the one that also has been made, that had been asked by a teacher, a geography, who wants to be able to, I just reset the question, who wants to be able to have a question type that I can drop text on a map, but in that way that it is being handled very correctly. In this case, for instance, the lease is a river in Belgium and that's the river over here, you see, that's the one. In a lot of systems where you have drag and drop, when you have hotspot questions, the problem is that around the river you need to make a rectangle like this, where the whole river is in the rectangle, but when a student places the word over here, it's still in the rectangle, but it's way too far from the river not as correct. So we have been able to let that question be made and then I can show it, so this is over here, Bruges is of course the city where even conference will be in June, so it's at the north of our province, Liege is a town over here, the Kampen that's a region in the northern of Belgium and Lisse is also a river, but over here. So when I ask for the correction, it will be set as correct, but I do otherwise, I place Bruges right, I place Kampen right, Lisse also, Liege also, but Lisse, I will place it over here and if I do correction, you'll see that this one is being set as incorrect. I can view the solution where Lisse should be placed, should be being placed. Okay, that's how the question is shown for students. I'll show you right now how a teacher can make those question types and you see it over here, he can make what we call drop zone on the image and over here you see that the drop zone is a polygon around the right answer so you can be more specific in working with images. Sorry for the fast explanation, I'll let work now if it's okay to Queen that can tell the last thing about Proctor Exam. There we go. I'll get back to the slide first. Right here. And on the middle of the plane. Put me up, use it. Stop sharing. Okay, so sharing one here. Okay, so now we're back and we'll take our last slide. Along with the startup of our examination center in 2014, we were also looking into systems to be able to organize exams on distance remotely, as we called them. And after searching and comparing a lot of tools, we've ended our search with Proctor Exam, the Dutch enterprise that has become a bigger player during the years. We've met the people of Proctor Exam when they were very, very small. They were just a startup with one or two persons that were involved. How did we come up with a Proctor Exam or why specifically did we choose at the end for a Proctor Exam? In 2014, when we started our search, in fact, Proctor Exam was the only European enterprise because there were a lot of American players, but given the data protection laws, etc., we felt more comfortable with a European enterprise. So they were the only enterprise who used two cameras, not only the frontal webcam that's built in the computer, but also using an app on a smart device, smartphone or iPad or something like that to have a second view, to be able to create a 360-degree view of the room where a student is taking his exam. So we've come up with Proctor Exam. We are now about four years later and so far 4,000 students or 4,000 exams have been taken remotely. So we see that this number is growing during the years. I'll explain in short what the procedure is like. So a student first goes to Square, which I show in a short time ago, where they register for an exam. They don't choose the exam centre as a location, but they'll choose the remote possibility where they can choose a day and a time to take their exam. And the advantage is that Proctor Exam is directly linked to Square. So if a student registers for an exam remotely, immediately the exam is being planned in Proctor Exam, together with automatically planned in Edumatic or Assessment Q if it's an Assessment Q exam. So there are very few, very little mistakes that are being made in the planning. Nobody has to make a planning manually, no teachers, so that works quite well. Once the exam is being planned in Proctor Exam, the student first will get an email asking him to take a technical test. So what is being tested? First of all, the screen sharing. Does that work? Does the microphone in his computer work and register any sounds? Do the speakers work? Does the webcam work? Is the internet connection strong enough? Very important. And last but not least, a student will be asked to test whether the app of Proctor Exam on his phone or smart device is working and is able to send video footage through the exam. So if that test goes wrong, if something doesn't work, the people of Proctor Exam will help our student. If the test is being easy and everything works, the student will get a second email from Proctor Exam with the link to the actual exam. That exam is, of course, only available on the moment that the student has signed up for. That's quite logical. At the start of the exam, there is a whole starting procedure. The student will first be asked to start screen sharing. He has to check if the webcam works. He will have to make sure the microphone works, the speakers work. He will be asked to start up the app of Proctor Exam on the smart device, phone or something else, tablet. And then a whole procedure is being started. First of all, with the smart device, the student will have to show, using the camera of the smart device, the environment of the computer. What's behind the computer? What's next to the computer? How is the desk organized? He will be asked to show the bottom of the table to see if there aren't any papers that are being held there. He will be asked to show the ceiling, all the corners of the room, the entire room. He will be asked to take a close-up with his smart device of the ears. Isn't there any Bluetooth connection that is hidden within the ears behind the hair somewhere? So all that is being checked and if this start-up procedure is being processed, the student can start his exam. During the exam, he will be asked to place the smartphone or tablet behind him so that we have a 360-degree view on the room where he takes the exam. All images, all footage. So we have a lot of footage on that moment. The footage from the smart device, footage from the frontal webcam of the computer, the footage of what is happening on the screen, the screen-sharing, the recording of the microphone, all that is being recorded, is being saved. So at the moment of the exam, there is nobody on the other side looking if the student isn't cheating. So everything is being recorded and reviewed afterwards. Right now, this happens by the people of Proctor Exam, after an exam has been taken. In the near future, this will be done, this reviewing work, based on artificial intelligence. So in the near future, this will change and we will be able to have artificial intelligence checking the images for us. So that will be a big change during next school year. If we ask students, why do they take exams remotely? The most logical answer is that they don't have to make any travelling to come and take an exam. They don't have to take the car or the train or anything to come to the exam centre. So that's a great advantage. First, they have the freedom of space where they take the exam. What is also quite helpful is that nobody has to be searching for a parking space nearby because that's a huge problem here on our campus. In addition, a student doesn't have to be stressed out looking for in what room he has to go to take his exam and so on and so on. But most of all, and that's what I'll tell as a last, I will be stopping the presentation in here, is that the students mainly name as a great advantage the fact that they're taking the exam in their own very, very familiar environment. So they're behind their kitchen table or in their sofa to take the exam. They don't get any sounds or noises from other students being around them in the same exam room and that's really, really something that's a great advantage. So strong even that we have students that live only 10 kilometres away from the exam centre but that take all their exams from home. And I think we will end this here. If that is okay, I suppose there will be perhaps some questions. Are there any questions? You have to look in the chat screen, I suppose. I saw one question post here. Yes, I wanted to thank you, Andriq, first of all and all the comments are very positive and people really much impressed with what you're doing. I have a first question. I don't know, I think we don't have so much time because we had that break and so we have just very few minutes I would say a couple of minutes just to have these questions on and it's related to what you were mentioning regarding artificial intelligence because what I was telling everyone while we had that first break was that in Rome we are trying to develop a system an automatic system for open-ended business and so using artificial intelligence of course. I know it's a difficult issue and I wanted to know more from you in which is your direction in this sense. Can you tell us more? Is this about the exams that are being taken remotely? Yeah, yeah, yeah. So right now we have the Dutch Enterprise Proctor exam that is doing the review work for us so they're doing this and in fact it's normal people that do that job so Proctor reviewer will be watching eight exams simultaneously and those eight exams where he sees all the feeds both the webcam feed, the smartphone feed, the screen capture feed will be played at a higher speed of course so that's quite a lot of information for one human being to process so very soon artificial intelligence will be taking this over and what's very important is that the eye movement will be monitored the typing behavior the speed and behavior of how the letters are being typed will be monitored as well as head movements if a student flips the head in any direction a lot that will be marked as being perhaps suspicious but much more than this I can't tell you at this moment I'm waiting quite, quite soon to get a full demo of what the artificial intelligence is going to change within Proctor exam platform and I really hope we'll have the chance to talk about it during the BRUGE conference as I was telling before so I think, yes, Mariali is mentioning have you considered plagiarism when taking exams at a distance but I think you already answered but anyway, if you want to add something regarding this point and then we'll have to hand our webinar please, Andrew, give me Mark is referring to plagiarism we didn't really get the question we've lost connection, I think, for a short... As soon as it was related to plagiarism if you have considered the issue of plagiarism when you deal with exams at a distance there is a tool turned in it's in the learning environment, too little that we can use for that but I guess for papers or something like that I suppose you mean, no? For papers, plagiarism? I know at the end we actually had the chance to use it, of course, of course but anyway, it's an issue that needs to be discussed further and so I really hope we'll have the opportunity to do it when we meet in Bruges I thank you so much for being with us I also thank you for having the opportunity to come back after the forced break with this problem with the connection and of course I apologize with all the attendees I don't know if Cristina can help me with that slide related to the next opportunity to have a webinar an Eden webinar together the date is April the 3rd so please do not miss it maybe, Cristina, exactly so it's not a slide but it's an advertisement that you can read here in the NAP area and anyway, it will be advertised on the Eden website on the Eden website you can already find the program for this term you will also have the opportunity if you are on Twitter tonight at 6 to have another opportunity for interaction with Palita Di Risinga from the Leicester University UK Department of Education will be chatting with us regarding digital skills so I really hope you will be on Twitter later less than two hours, actually less than one hour and I hope to meet you all back and please disseminate our events with your colleagues and we'll meet on April the 3rd on our next webinar thank you for being with us, thank you all, bye-bye