 Heel meneers van dit konferent, vrouw en vrouw, we begon een paar minuten later, omdat het probleem is om een paar van jou te doen, maar ek denk dat we, we kunnen starten. As president van Vives, ik ben honderd om jou te welkom in dit vernieuwde Edenkongres. Eerst wil ek die Edenkongres, om ons de kans te honderd om dit event te doen. Het voet perfect in een vernieuwde team, gered door technologie, maar meer van dat later. We zijn heel gelukkig dat u hier om hier te honderd, en zo'n groot nummers. Bruus is vaak de Venis van het Noord gehaald. Hopelijk gaan jullie het kans, en tussen dit Edenkongres, we gaan dit historieel behoorlijke stad honderd, om in het typische koos van kleine steden te honderd, en niet om te honderd, dit middelige stad is Unesco World Heritage. Vives, de universiteit van about 13.000 studenten en 1.300 staafmembers hebben 5 verschillende locaties, die al over de provincie van West-Vlanders gesproken hebben. Besef 4 andere universiteiten van gesproken siense en Universiteit van Leuven, we geloven met de associatie van K.U. Leuven, die is de grootste associatie van de landers. Besef van studenten die al werken, maar wilde het studiën gaan, Vives begon met een vernieuwde studie in 1997, in het studieveld van educatie. Sinkt het de oudere studenten die school die het siense goede children had, die het siense geloven, die het siense geloven, die het siense geloven, die het siense geloven, die het siense geloven, en was verpleemd in Broder in Vives. Sinds 14.15 van de academic year, we ontvond de forum versterking. Deze manier wou ons volgen, ondra wat ek die stelings, dat ditensleerling ons eindig was, dat die vervolg is, vervolg ek dit. Mevrouw, ons verhaal, 30 stelings via ditensleerling en meer dan 2000 studenten volgen ek dit om te vir een professionele bachelor. Selkels die jaren, ons versterk om technologie te gebruiken in een verpleeg en innovatieke manier. The needs of these evening students are different from the needs of regular students. We had to make the necessary adjustments of these, certain little to limit the high drop out. For example we had to optimize the administrative and logistical process that are typical in this form of education and we have adjusted our assessment section. We could not evaluate according the usual way. We had to look for possibilities that met the demand of distance learning students. Flexible exam schedule, faster feedback and so on. We work with an exam center where students are offered the opportunity to take exams at different times. Students can choose themselves when they take an exam of a certain course. These can be done at the central location in the exam center or even simply from home or abroad. Top athletes for instance can now also use these options. We have gradually become the market leader in Flanders with distance learning and we want to continue to focus on this focus in the future. I already told you driven by technology is now the focus of our university college. Driven by technology cannot in your that the fact that everything we do must be inspired by education, by students and by the future of the students. Technology is not a goal on itself for us, but it means to ensure that we can do even more the things that are needed to provide quality higher education with technology as a leverage. Our vision is twofold. How do we as a university prepare students for the new and renewed professions for the future and how can we use technology to increase productivity within our education. We can no longer use technology and digital evolution solely in our technology directions. As we have many, the technological and digital evolution affects all courses in order to respond to the students of tomorrow. With the prospect of making all courses innovative and future proof, we started this academic year by strengthening all courses by offering artificial intelligence, visual representations, technological trends, augmented reality and virtual reality. And we are trying to connect technology, research and education. Innovation in this province, a province with a strong industrial fabric is on the agenda more than ever. Our partnerships and our contribution to the development of this region are strongly digital and technological oriented. Together with the university, we focus on mechatronics, new materials, food, care and educational technology within the province. Together we are fully and fully damaged to acquire our digital and technological position that we also deserve. Because we find that we need to be able to delve even further into this, we will also deepen the team further next year. As a university college, we must continue to look for new technologies to design innovative learning environments. A few things we plan to tackle next academic year are more remote meetings, further experimentation with the virtual classroom and we also realize that these will provide a different interpretation of the teacher's position and that we will also have to invest in that. I would like to thank everyone who participates in this congress. The colleagues from Eden, but also the colleagues from Vives themselves. I would like to thank also to our sponsors in particular Tilevik who will bet the next keynote. We look forward to their contribution. I also want to make a warm appeal for the demonstration that we will give on Thursday about evaluating distance learning, but also for the tour in the pop-up classroom that is also on the program. I certainly recommend it. I wish you all a hugely inspiring congress with a lot of rich insights. I hope you will be able to connect to educational technology, which is in fact the team of this conference. I thank you. Now I give the floor to Mrs. Irina. Thank you very much, President Hendrix. Dear colleagues, dear friends, dear Eden family, our global partners, fellows. It's very nice to see you here coming by Texas to the late minute and the traffic reaching out for our wonderful keynotes that are waiting until we list all other important facts for the opening of the 28th annual conference in Bruges. Eden has organized conferences in 22 countries already in Europe in the last 27 years. And now in Bruges we are so happy to come with 310 conference delegates from 44 countries. 20% of delegates are coming from outside Europe, from other continents. Highest number of conference delegates of course comes from Bruges, followed by the UK, Spain, South Africa and other countries. We are so happy to have people from Ethiopia, from Canada, from the States and other continents. At the conference we will have 4 plenary sessions, 37 parallel sessions including many papers, posters, workshops, training sessions, demonstrations. And actually we have already started conference with the pre-conference events that are of such great importance. As the living president of Eden, I would like to highlight that we celebrate collaborative leadership in this organization. Collaborative leadership allows us to see better the whole picture. The achievements may be summarized over the last three years in two areas, consolidating of our community, as well as revisiting, reshaping, creating new practices, services, European policies and education. And these steps are supported by Eden constitution from the very beginning. Despite of the fact that we stand on the shoulders of giants, we have the fellows, we have the senior fellows who just established the council of senior fellows. We have many challenges in front. We had, we had and we will have. Starting with Brexit referendum results in June 2016, the efforts to reshape Europe and optimize European high education arena by member states and many others. However, Eden senior colleagues knew very well and always that Eden family is with us. On shopping we started and in Barcelona we established the council of Eden fellows. Consolidation of Eden research in revival of Eden PhD symposium after several years are only two of major milestones reached in our association. Several initiatives and task forces were established and fostered Eden through these years, a mandate in ED 2020 walking group DELTA digital education learning teaching and assessment. Eden special interest group for technology enabled learning and quality enhancement, international and global partnership with global organizations. Finally, a great change in the development of terms of recognition of academic and professional merits, but also Eden aiming to serve as a platform to develop, discuss and improve recognition activities through new instruments and recognition schemes in digital credentialization. Of course we have things to be improved and this is for the board continuing these difficult tasks of the governance of Eden. But collaborative leadership is here. We are easy taking, scalable, manageable, always negotiated and thoughtful approach by great colleagues of mine. I would like to thank very much all of you with whom we collaborated for these three years. I hope we will continue and also past presidents, also the secretariat, executive committee, council of Eden fellows, PhD symposium chair and the committee. All of you gave me that strong confidence in my every step. I walk the walk with you, with each of you to reach the new peaks of Eden. So I look forward to the self expanding network as Manuel Castel would say. And now I would like to invite the president of Eden, whom was endorsed by annual general meeting yesterday. Sandra Kucina Softik, Eden president elected yesterday who is the head of e-learning office at University of Zagreb Croatia to speak on behalf of Eden presidency. Please welcome Sandra. Good morning everyone. Sorry for being a little late. I almost said not the kingdom for the horse, but the presidency for the taxi this morning. But maybe we made more tourism in Bruges and maybe we made the cities and cities aware of need for more taxes. So increase in this business. So thank you very much for joining the Eden annual conference this morning. Thank you for the Bruges and for the Vivas University for accompanying us, for hosting us. It's a very nice city and it's a very nice university and I'm very happy that we are here. And we have really, really huge support from the hosts, which was very, very nice in the organization. And I hope you will all enjoy all three days of the conference here. As you have seen the title of the conference is Connecting with Educational Technologies. Connecting is very important today because with the numbers of information we are all acquiring, it's very difficult to find out what are the right information and which information to choose. And by connecting jointly and collaborating we can see what we find is most important to us. I'm taking the presidency from Irina, my dear friend and the colleague. And I'm very happy that she was before me because she left the Eden in a very solid and strong state. And I hope to take steps following her and make Eden even broader and a more wider institution. But with all of your help, I cannot do it alone. We are organizations, we are people, people are the most important. And only with you, with the members, with the network of academics and professionals, with Eden fellow council, with secretariat, we can make Eden organization place where we can do more, where we can push the boundaries, where we can aim for some changes on the benefit of education and society as a whole. So I wish you all very, very good and fruitful work these three days. And looking forward to all your presentations and sessions. And do collaborate, do talk, do think about how we can make this world better for all of us. Thank you. On behalf of Eden, I would like to express a series of gratitudes. And I would like to start by expressing special thanks to Vivius University of Applied Science for the hosting of our conference. By name, we are most grateful to Silke van der Kruus. I don't know where Silke is. Hello. By Lynn Tees and other colleagues who started walking a year ago and continuously, supportively, very collaboratively came to the result of this brilliant hosting that we see today. They have been most valuable partners from the beginning of the organization in the program development, supported Eden secretariat staff with advice, careful elaboration, many important details, and ensured generous support to offer the best conference experience and quality for the delegates. I would like to invite President Hendricks to accept our award of honour on behalf of Eden for hosting the conference in Bruges. Yesterday evening, we already started enjoying the generosity of conference organizers who of course worked together with the conference sponsors. We would like to give our special thanks to the principal conference sponsor, Tel Avik, and I would like to invite Tel Avik representative to the stage to hand over certificate appreciation of principal conference sponsor if available. Later in the day, we are really grateful for the principal conference sponsor, as well as to Blackboard 47's signpost, simulated standard walking handle, conference publication sponsor, and the City of Bruges conference dinner sponsor. Please applause the sponsors of the conference. Finally, I would like to thank on behalf of Eden the program committee. Thank you to Eden fellows gathered to the important meeting in Bruges for today in the afternoon. Please don't forget. Thank you to the evaluator panel of the Ph.D. symposium by name Professor Joseph Maria Duarte. Thank you to Eden network of academics and professionals, members and the Eden fellow, senior fellows. Thank you to all delegates, but particularly to those taking active role. Speakers, authors, chairs, moderators and presenters in the workshop. Thank you all very much. And now we start one of the most interesting part of this conference, where we have keynote speeches by two important intellectuals. We will have presentations of 30 minutes followed by the interaction with the delegates. The first keynote we underestimate the testing effect and how technology can help is Bert Willing, e-learning and E-assessment expert, K.U. Loven Belgium. Keynote sponsored by Televig. Bert has both an academic as well as business profile. He works at K.U. Loven since 1993. Since 1996, he led the Education Innovation Center. In 1997, he co-wrote and published the first Dutch language book about the integration of Internet in the classroom and education. 1997. In 2001, he founded a K.U. Loven University spin-off company developing and servicing educational technology projects. In 2008, this company became the mergers, the current Televig education today. Bert Willing is responsible for the e-content projects, e-assessment and e-learning in research. He focuses on assessment, Q and online testing tool for learning. Please, Bert, the floor is yours. We'll be standing right here if you don't mind. Won't take the step. Thank you for the opportunity to talk to you. Hopefully discuss at the end as well. Welcome to Beautiful Brush. I hope you enjoyed it already and can enjoy some more. You already got some introduction about me, but I'll briefly repeat because all in all I think my profile pretty well matches this conference theme about connecting through educational technology. I started my career at the University of Loven, actually the campus at Cortraig, where I worked for eight years, specifically in distance education, first in a project for developing distance courses. Then I was supervisor of the Open University Center in Belgium or in Flanders. Some universities host the Open University of Holland. We had one of those study centers and I supervised that. Then for some years I got to lead what we called the Center of Education Innovation. That was already a joint innovation center between Vives, that was still called Cato Den and our university. Together with Vives my role was to help the teachers and assistants in introducing technology in their courses. That was really fun. So fun that in 2001 I founded a spin-off company from that center. That was called Telram, which is abacus in English. That was a company specifically to develop educational technology. It was a spin-off company, if you understand the same what we mean by it. A company where the university also joins at the beginning and then after five years gets out and hopefully the company still lives on. So we did, we grew a little bit, we evolved a little bit and in 2008 that company merged with another company of about the same size. Together we became Tel Avik education that is now the sponsor of this session and of this conference. So today I still combine those two, some academic research or even better trying to implement academic results into real products and to bring them to the market. That's the core of my job, helping to make the best possible product based on scientific research. There are worse jobs in the world, I can promise you. I love my job. I forgot to take, you should have seen this. Here I am now for you in front of this Eden audience in what is called a sponsored keynote session. Probably some of you think horrible sponsored keynote stuff. I'm a company speaker probably a bit beyond the control of the program committee so some of them must always also perhaps be scared of what is he going to tell. But I think what I'm going to tell is really on the crossroads of academics and business. I know also from my own not always best experience that it is not always a perfect combination, academics and business. Many academics are often skeptic when they have to work with companies, we're too commercial, our solutions are never exactly what they want. And many business people think the other way around about academics I must admit. With all their beautiful theories that never come to practice and with all those overfunded projects that compete with our products that never grow beyond the concept design or the proof of concept and so on. The mission of a company like ours is exactly to try to integrate academic research or results or ideas and to develop them into the best possible products. In our case at Tel Avik education we do that for assessment technology, assessment technology in general, language learning and even more specifically translation and interpreter learning. And still after almost 20 years combining both worlds I think it's worth investing more and deeper in such partnerships. I think and I hope I will be able to convince you with a few good examples of how academics and business can work together. Please don't run off, stay here for some more time. Now the topic we choose for today is assessment or eassessment. I know this topic is not an exclusive one to distance education, it applies to all kinds of education. But it certainly applies to distance education. Assessment, eassessment is surely one of the big issues in distance education. What I will do in the first part is take you into the back office or into the kitchen of our company. There where we prepare our menus, where we prepare the foods before we try to serve them to the market. As you probably know most products don't sell by themselves. You have to market them, you have to promote them. I think if you have a project or a research ID you also have to do that. You have to convince people to choose your project, to choose your research ID to get funds for it. That's not different. So what we do as a company is we get together, we call that a marketing meeting and then we try to prepare what I call a story. We try to prepare a story for our customers and our customers that are the ones that are going to pay us. But also the end users that are going to use our product. So we try to think about a story that we can bring that will convince the people. So I will disclose a little bit of our story and then lead you further into my topic for today the e-assessment thing. Just to get things clear what I understand by story is not a fairy tale. It's not about something magic and unreal. A story, what I mean by a story is that a strong set of arguments that we, arguments that are coherent and that are captivating enough to convince the audience. That's what we understand by a story. So it must contain a little bit of a metaphor probably, but also solid really scientific arguments. And it should not be boring because if you tell boring stories people fall asleep. So in our company we try to find a story for both the e-learning world and the e-assessment world because our products can be used for both. In the first part I will focus on the e-learning world. Now for the story about e-learning the main ingredient of our cooking of our story is ebbinghaus. I don't know if you heard about ebbinghaus, well he's old so you should have heard about it and his forgetting curve. This is what we tell about it especially when we have customers that want to learn. Ebbinghaus was a German researcher born in the 19th century. He designed this curve that shows that when you learn something, when we all learn something, by reading something, by hearing something, by listening to someone, by following a e-learning course, you have a very high first knowledge, you get a very high amount of knowledge at first, but in time that knowledge decreases very fast. So what he even found out was that after 30 days more than 80% of what you learn is lost if you don't use it. And that's not just you, that's also me, that's everyone, it's just something that is... So that's this important for getting curve. After 30 days if you don't use your knowledge you lose it and it gets worse than the 80% of course over time, but this is very important. So this metaphor or this fact actually it's not a metaphor, it's a hard fact. We use to tell people that they should invest in learning, learning if preferably, and to help overcome the curve and to make the curve go up again by organizing formative and summative evaluations. Every time you do something, you activate that knowledge, you can go against the decline of the curve and then make the curve go back up again. So what you see here, the first intervention could be an online exercise, a new course, an email you receive with a reminder or something, and that is a formative moment or an exercise, a formative evaluation that gets the curve up again. And also very important, an exam which is also known as the summative evaluation or the summative assessment also gets this curve up. This is something I want to come back on later. So assessments, evaluations, exercises, help against the forgetting curve. And something Ebbinghaus al ready found out very early was this idea of distributed practice or spaced practice. So what he does is he says that the best way to go against the decline of the curve is to give at specific regular moments in time come back and do exercises, spend time on it, activate the knowledge. So the slide you see here is a slide that we use to promote one of the tools that we have which is sending an email on a daily or on a weekly basis with a question. About a specific topic. Well, every one of those moments will bring back some knowledge and will keep the curve high. So this is something that we tell, we apply the knowledge or the scientific insights of Ebbinghaus to what we tell about how people should do e-learning. Now we tell this story for instance to hospitals. To hospitals who want to train their staff and we say look if you spend the course on hand hygiene for instance and you don't talk about it anymore afterwards, that knowledge will really, really disappear afterwards. So you should invest in a tool or in a method that keeps the curve high. So please spend time on activating, reactivating the knowledge. And this is exactly what technology can do. Technology can do that. Technology is an excellent tool to do that distributed practice. To send out on a regular basis exercises, evaluations, activities and so on. Now in my humble opinion this is a very sound and correct story. I mean I don't lie or cheat when I tell this story to people that I want to convince to buy our product or use our product. It's a story from our marketing kitchen but I hope you find with me that our cuisine is pure. It's a modern cuisine, it's not fake and I think it's also fair. Let's go one step further. I said e-assessment was the topic. So let's focus a little bit more on that summative evaluation or that summative assessment. As we said before, there are many words for summative assessments. Assessments, exams, evaluations, certifications and so on. There again there is an important story that we want to tell our customers about exams. Our customers for e-assessment are very, very right customers. There are schools, universities but also professional organizations that want to do certification programs or exams and so on. So the story we use for our e-assessment came to us through a contact with the Welten Institute from the Open University in Holland. More specifically through Dr. Kim Dirks, she visited us not too long ago to discuss with us and with our customers about the e-assessment. About the effectiveness of testing. And what she came to tell us actually was what we call the testing effect. And this is really what my talk should be about or what I wanted to talk about, the testing effect. Simply put, the testing effect is this. It is that research confirms that evaluating and examining have several advantages. But the most important one is the fact that testing and evaluation itself has a major and very positive influence on learning or memorizing. So from doing a test, from performing, from passing an exam, you learn. That is called the testing effect. I hope that is clear. So actually, simply put, I could say exams are fun. That is the other way to say the same thing. Exams are good for you. And I know I go a bit fast now. I think I am a bit too optimistic. Say exams are good for you, exams are fun. It happens, we say that my own daughter is having an exam in this building somewhere here right now. And she started at 8.30. So I don't think she thinks exams are fun. I'm sure she doesn't. But actually it is, the findings that they did are very interesting. And I cite a few names. Rudiger is a very interesting starting point if you want to read about it. But Carrier and Pashler already researched that very thoroughly in 92. Testing does not just provide an additional practice opportunity. It reduces better results than other forms of studying. So doing exams is an excellent way to learn. To study even. That's a very interesting insight. And that really helps our story to convince people to do assessment and e-assessment. So a lesson that we all should learn and you all should learn from this is that we as universities, schools and so on, we should promote testing, evaluating and exams as a positive thing. I think we don't do that enough. I think we scare people with exams instead of telling them that it's useful, that it's fruitful to do exams, that they learn from it, they get better from it. Exams are good for us. There is some further research that is very interesting. I'll skip a little bit because time is short. But what they say is that an immediate test is the best. Immediately after receiving information, you should have a test. If you do that, then your results will be better than if you don't have that test. So immediate assessment, immediate exams are better for the longer term. And that is that one other thing that they found out is that intermediate tests are also very interesting. So that idea of distributed practice also goes for distributed evaluation. The learning curve gets much higher, stays much higher if you do distributed evaluations. This is just one you should read, I won't talk about it, but it's a fun one, I think, instead of getting information into the heads. If you're interested in this, in my humble opinion, osmosis.org has the best videos explaining this testing effect and distributed practice effect. They are really interesting videos. You can find them on YouTube if you want. Now, the most interesting part of the research from the Welten Institute was for me this. They said that if you do exams and exams are good for you, what we should do as a school, as a university, as a training institute doing exams is that we should include feedback in the exams. The results, the learning results are much better if you include the feedback in the test, not after the test and especially not long after the test or in some situations never. Because in some situations you perform an exam and you get a result, a score, a note, but you never ever get feedback. So they found out that it's very important to integrate immediate feedback in the testing. Immediate feedback, that's already just let them know if it's correct or wrong. In some cases you could add a score and in the best case you can also add qualitative feedback explaining why this is right or wrong. So in this simple example screen here you get a question and then when you answer and you submit your answer at the bottom you can see whether it's correct or wrong, you can see your score and then there is a small explanation on why it is right or wrong. This is in the test. Now, again, technology can do that and I think only technology can do that and this is where the conference topic is so important. This is where technology really brings a help that is impossible without technology. Paper cannot do that. So this is just a very subtle thing that I show you but in our tool there is a very simple option that says always show feedback before moving to the next question. That means that even if they skip a question they will see the feedback, we force the feedback because they learn from it. Let me show you one online demo. I know my colleagues always laugh with me because I always keep doing online demos and often that goes wrong but there we go. Imagine this is an exam. I have three questions in my exam. This question, I'm not sure. I go to the next question. This question, okay I know this is a physical one, this is a psychosocial one and so on. Now I think that first question, I know, I know. It's a yes or it's a no and I submit my answer. I get immediate feedback. I'm in my exam, I get immediate feedback and I cannot change my answer anymore. As long as I don't submit I can change my answer. I could have checked yes, no but when I submit I get the feedback. So I learn here. That's not the same as just clicking and going on. So this is a very simple but very effective way of adding feedback in an exam. The same goes here. If I click here, imagine I finished my question. I get the feedback, I even get to see the correct answer if I want. This is in the test because my score is already fixed now. I cannot change it anymore but I see the feedback. This is impossible in paper and pen. This is something that only technology can do. So technology can do that. I know it's less common but it's perfectly possible to show immediate feedback to the exam. Now to finish, I want to show you one last, a little bit of an extensive example, of where technology helped people in testing and evaluating and where both teachers and students were so, so happy. Try to follow because it's a bit a story but we did it based on an idea of the University of Leuven. We developed a tool and it was used recently by the University of Padua. It's about revising or translation. So this is about translation, students in translation studies and teachers send out texts to translate and then have to revise those translations for an exam. This is how that was done until recently in Word. With all those remarks and comments and so on. Impossible to get overviews or statistics on the kind of errors and so on. And also impossible to be consistent and objective. I mean when you finish the first student you close the Word and you go to the next Word document and you start all over again. Even if they make the same mistake and so on. And I can guarantee you the teachers that we worked with. After the 15th question, their objectivity was not that correct anymore. They often said, I sometimes give a minus two for the same error and so on. So what we did was develop a tool to try and solve this problem. So what you see here is this tool. So the teacher sends out translations, receives them in the platform and then in some way receives the translations from every translator. We make the names of the translators anonymous because the reviser cannot see the name of whose copy he or she is revising. So then you click on one of the translator's texts. On the left side you see the source text. On the right side you see the translated text. And we start with a blank revision. There are no errors at the bottom so far. So what you do then is as a reviser you select an error. So in this case it says, de quoi consiste l'examen. That's bad French. In French you say en quoi consiste l'examen. So you select that and immediately this plus error pops up. If you tick that button the system allows you to enter some extra information about that error. So the error, a correction for the error, you categorize that error. You could say it's a grammar error, it's a punctuation error, whatever. You give a penalty if you want, a minus one, a minus half, a minus five, whatever. You could also say this is a plus one, this is an excellent translation or just a zero, just a comment. And then you give feedback and you give a correct translation. So this is automatically stored into what we call the revision memory of the system. And the system when you submit that error will check all the other translations to try and find the same error. If it finds that error it will ask you to apply that same information, that same penalty if you want to that error. So this is what you see here, this error was also found in Translator 2, the quoi consiste. The student translated this procedure instead of this exam, but it's the same error really. So then you see the error applied, in the error overview you get a minus one. And if you go to the second student you will see that the system already shows that error found and corrected. So it helps in being more objective because everybody gets the exact same feedback and penalty on the same errors. And it is a bit more efficient because you only have to correct an error once. By the way this also works in monolingual texts if you want so you don't have to translate text. So this is how that system works. The end result of all of that is this is what we call an error memory or a revision memory. So all the errors are stored in the memory and you can analyze that. And we call that the big wrong data. You know big data of course, but big wrong data is an immense heap of errors. And I quote a famous recycle company here in Belgium and Holland. They say, afall bestaat niet. There's no such thing as waste. It's true, I mean this is not waste, this is very interesting stuff. From all those errors we can learn. And again demo of an Excel that I prepared. This is a dump, this is life in Excel. This is a dump of all the results from that revision memory of the students in Padova. They had to translate French to English, sorry Italian to French. No French to Italian. On the right side you have some statistics about that revision memory. For instance showing you that here for instance the overall average frequency of an error is 2.4. That means that average frequency of an error. It means that you have 2.4 less time needed to correct because it already finds so many. If you forget the unique errors then the average frequency of the non unique errors is over 6. So it means that there are a lot of unique errors but the non unique errors are very frequent. Students tend to make the same errors over and over again. That's interesting but what is even more interesting is if we go into a pivot table we can change this revision memory into a very efficient look at this. We can see the category of errors and the number of errors and underneath we can see which errors go into that category. Interesting, I can drag in the score into that overview and look what I see here. Within a category of errors I can see the score distribution. I can see which penalties were given within the same category of errors for which errors. You can see a score distribution of the revision behavior of a teacher. This is something that is really impossible if you work with pen and paper of course. If we make it even more interesting this was very interesting that first thing with the score distribution for the teacher. In this case she found out that she was very inconsistent. Sometimes we found the same error and once she gave it a minus one, once minus two and so on. So that was very helpful for her. The other thing is that and then we dragged in the names of the students as well but I make them anonymous here. But if we go into this pivot table you can see that with this system you can make a profile for every individual student and show which categories of errors they are in our queen and so on. The other way around if I drag the categories over the students I can see that in the punctuation category well this person here made most errors. And this is something you can tell the students, you can give them a profile. You can tell them look this is your profile as a translator. This is where you should work on this and so on. Now this is an excellent example where after some time everybody was happy doing exams because they learned from it. The results really helps them for the teacher to give better courses because they know where they have to focus on. In this case it was on punctuation. She found lots of punctuation errors. The students got a personal profile of their translation behavior and they knew what they should be working on for the next year or the next text and so on. This was a really good example of where technology was used to do evaluation and that has an immense added value. To finish, three takeaways, just very simple takeaways. I hope you are with me in these takeaways. The first one, as a company we are your friends. So don't be afraid to do partnerships with companies as academics. I think those two worlds go very well together. All the examples that you have seen are all examples of things or ideas that we have developed and brought to the market but based on things that we have done together with universities. In the product developments that we do, we invite the universities, we invite our customers and listen to them and they really bring good ideas. So we are your friends. Second one, exams are good for you. This is a message for the students or the learners. I think we should convince people more and explain people more about the fact that exams are beneficial, that they are good. Not just a burden, something horrible at the end of the year. And it can also help if you spread the exams more over the year than all at the end. Last takeaway, technology can do that. It was the conference topic connecting through educational technology. I truly believe, of course, that technology can really help a lot in making things better. In doing things that were never possible without the technology and technology can be a real help. One just last thing, just last week we launched a new company, a small company that is called PEP. I think I have the logo here. It's about patient education. What we want to do with that company is look at me here. I got a surgery last week. What we do is when someone goes to the hospital, the hospital can register a patient and say this patient comes for this knee surgery on this date. Based on that thing, our PEP will send an email like 15 days before explaining what is going to happen and with a few questions in it and so on. And then two days before your surgery, think about this and this, when you come, bring this and this. And then two days after, let's start your reeducation and we're going to work on this and this. And then one month later, a survey and so on. So this thing helps educating patients while they are in the hospital or before and after. The basic idea there is very simple and it has been proven scientifically is that the more people know about their treatment, the better they heal. It's so easy, it's so simple. I think we should do the same with students. We should inform them better on why we do exams, on how we do those exams, on why they have to do those exams and then explain after the exams that it is so beneficial. I think we don't do that enough. I think this is something perhaps you can take home with you. So I think I can finish off by just referring to this excellent good practice that will be presented I think tomorrow from the colleagues of Ives. They worked with some of our tools and I think they have a very strong case on assessment and E-assessment. Henrik is al ready referred to it with her assessment center. Excellent stuff so please join them. Thank you. Thank you very much, but as agreed the questions will go in the end during the interaction with the audience. Therefore allow me to move on with the next keynote who is Steve Wheeler. We know Steve as a former associate professor of learning and technologist at the Bleemouth Institute of Education and learning innovation consultant where he led the group of learning futures and computing and science education teams. Author of over 150 scholarly articles with more than 7000 academic citations. He continues to research into technology supported learning distance education with the emphasis on pedagogy underlying the use of social media and that to zero in mobile learning and cyber cultures. A negative blogger. Learning with ease is a regular commentary on the social and cultural impact of disruptive technologies. Steve served six years as Eden executive committee as chair of the network of academics and professionals. Here is Steve Wheeler learning innovation consultant on the keynote connected pedagogy. Learning and teaching in the digital age. The time is yours Steve. Thank you Irina. Thank you so much. It's lovely to be here and it's I think a real honor and privilege to be invited to speak at such an incredible event as Eden. For me Eden is the most I think it's the leading organization for technology supported learning worldwide. I don't say that lightly. I've been involved in it many years. So Alina and Sandra and Andrews and all of the team thank you for inviting me to speak here today. What I'm going to try and do today is not lecture you or try and educate you because you're a very savvy very sophisticated group of people. You I think you know a lot about distance education and about technology and about learning in particular. So I'm not going to attempt to educate you today. I don't think I could. But what I will do is try and give you three challenges. So right at the end of this session you'll get three challenges on the screen to go away and think about. And leading up to that I'm hopefully going to try and throw a lot of ideas at you which you will connect together in your own head and make sense of in your own way. And from a different perspective and each of you will have different experiences. So before that I just want to do this show you a few things about Eden. This was my first Eden conference back in 1996. Anyone there Poitier. Does anyone remember this Futuroscope Poitier. This was the hall in which I spoke and there were probably that many people in the hall when I spoke actually is probably empty. But it wasn't that empty because I shared the stage with two people who are now very valuable colleagues to me. That's Sally Reynolds and Julie Salmon. So we were the try and virate that day and that was the session we did Poitier. But then later on I can get this clicker to move. Who's that young fellow there standing next to Andrews. Remember this from Eden in 2005 Helsinki. And that was probably one of the first time is when I started to get to grips with the idea of what distance education and what technology supported education. And what social media at the time which was evolving was all about. And then I move on again. Here's me in Lisbon with this gentleman here who you probably recognize as Alan Tate who was the president at the time. And I received my fellowship award then in that year. And that was that was a great honor for me. And that for me that kind of launched me into into doing other things with technology supported learning that I'd never really thought of before encouraged me. So all of the those of you who are fellows and senior fellows of Eden I think you can take that on board and move move ahead with that. But then later on. Oh here we are two years ago in Jan Scherping you can recognize some of those people there on the left is my my lovely wife who sat down with me. In the front row there go and say hello to her afterwards and embarrass her. And she's called Dawn and that's that was me receiving my or just after receiving my senior fellowship award which I was even more thrilled about. But I just wanted to show you those because for me Eden is a family. It's like a kind of an extended family isn't it would you agree. I think those of us who have been involved with Eden for a while think of it as a family and those of you who are here for the first time. You're in for an incredible experience this next three days. You've got so many different things to look forward to. I've been looking through the program from Kahoot to MOOCs. Try saying that after a few Belgian beers. You've got from innovation to in face education if you like. You've got from projects to progress. There's a whole range of things going on this next three days which I'm sure you'll go away with your head brimming full of ideas. But to get you into the mood of this. Here's my experience. Oh there's another one there. Look at this look look at this. Recognize these faces here from 2017. And then another one from back in Dublin in 2011. And another one there from Budapest a few years ago. So it's a family. So welcome to the family. Back in 1970. I was just a teenager back then. Some of you probably weren't even born then but back then in 1970. I was taken by my school to this building here which is just across the border in Eindhoven in the Netherlands. And this was amazing because Philips who were the giant electronics company of the time they built this. What they called the Avallion a flying source of shaped building. And it was a technology. History museum. And you'd start at the top you go up in a glass lift right to the top and then you'd work your way down in a circle. All the way down through the floors. And you would see the history of technology from the first the invention of the wheel and the invention of fire or the discovery of fire. All the way down through. And I remember going into one room with some friends of mine and we were liking around finding what we were going to be looking at and so on. And we discovered that there was a television and a camera and a microphone in there. And then down the other end of the room there was another television and a camera and a microphone. And amazingly they connected with each other. And you could sit in one room and you could go like that and your mate could see you in the other room and you could tell each other stories and so on. And you could hear and see each other and I thought this is amazing 1970. This is Star Trek type stuff. And I thought this is what I want to be involved in. So that inspired me to become involved in in technology right from the start when I was about 16 15 years old. Moving on a little bit. I mean we looked at these kind of technologies and tried to make sense of them. He's looking very interesting isn't he. He's really thrilled I think to be here. But moving on a little bit if I can get this machine to work. There we are. Another thing that happened to me which I think really spurred me on into this world was I took a distance education program. In fact I enrolled with the British Open University in Milton Keyes which incidentally is where next year's conference is going to be. Another plug there. And the interesting thing is that I left school with nothing. I left school with very few qualifications because I was bored stupid. And I was absolutely scared stiff of testing but I didn't want testing. I didn't want exams and I didn't enjoy any of it at all. So I left school with nothing. But when I reached my thirties I thought I've got to do something with my life. And so I went off and I did a degree in psychology at the British Open University. And that launched my career into academia. And the rest as they say is history. And so from then on I've seen all sorts of mega universities like the British Open University like UNED and like the open universities around the world. I've seen them as the universities of the second chance. And that ladies and gentlemen is what you are involved in. You're involved in giving people a second chance in life. And that's awesome. And that is the most important thing I think that we can ever do is give people second and third and fourth chances to succeed in what they want to do. But we have a problem. I'd like you to just read this. This is from a professor at the Open University. And while you're reading that I'll tell you a story. Last night we came back from Bruges. This is the center of the Bruges by taxi. And the taxi was actually on time. It wasn't taxi snail was it? Was it taxi snail? It was another. It was an independent taxi driver and he brought us back. And as we were coming back to the Weinerberger Hotel he said have you seen the Tesla machines outside the Tesla electronics outside for the electric cars? We said yeah. He said well do you realize that Tesla is also providing a download soon so that you can make your vehicle your electric vehicle autonomous. It can drive on its own. He said soon we'll be out of business. I said I don't think so. I said it's like teachers. Teachers will never be necessary. Because as Arthur C. Clarke said. Any teacher who can be replaced by a computer should be. It's the same with taxi. A taxi driver will always be needed. Think of elderly people or people with disabilities, people who want a conversation. We will always need the human element. It's the artificial intelligence element that will actually create better ways of doing mundane and boring and dangerous tasks. Whilst we get on with the real life of humanizing. I think that's going to be the big difference in society over the next 10, 20 years. We have a problem because technology can become both an enabler and a restrictor. It can be both a positive and a negative thing. I'm going to give you a couple of examples of this. Here's a few glimpses into the future. We're looking at things like the quantified self, ubiquitous connections. We're looking at things like augmented reality and vision enhancement. The wearable web. Wearing the web. Actually wearing the devices that connect us to the web. This is a complex and torturous future that we're moving down. And it will create problems. Social problems, emotional problems, relationship problems. Politics. Economics economically. There's a whole range of things that we have to look out for here. One of the innovations that I'm really interested here in is this one here. I don't know if anyone's seen this before, but Waverly Labs have recently released the pilot, which is in any year, but if you like, which allows you to work with your smartphone and it translates simultaneously the language you're listening to into your own language. And it works. And it comes out at the moment at under $150 US. So everyone's going to be buying these things. So the interesting thing is, what will that do for language teachers? Will it make them redundant? I don't think it will. Because the problem is, you can listen to the language and you can understand it, but can you speak it? No. The person would also have to have any device to listen to you speaking as well. So it's not a universal translator like Babelfish or from Hitchhiker's Guide to the Galaxy or anything like that. But it is a step in the direction where we are beginning to see artificial intelligence and smart technologies starting to undermine the role of educators. And we've got to be aware of this. We've got to understand that the technology that is coming in will change the pedagogy that we are currently engaged in. That pedagogy will be changing all the time. Teaching and learning skills and methods that we are currently using will have to adapt, will have to change to meet the new needs of society. And when you start looking at what young people are bringing into the classrooms of them today in their pockets and in their handbags and so on, you have to understand that they are becoming involved in the new digital tribe. The worldwide digital tribe that we are seeing because of this ubiquitous connection. Young people are absolutely wedded to their technology. My daughter, my second daughter, Katie, when she was around about 18 years old she had a brand new iPhone and she left it on the side of our kitchen table. We've got a stone floor and it fell off, she knocked it off and it fell on the floor and smashed. It was absolute disaster for her. She looked at me and she burst into tears. She went into mourning for about five days. She was dressed in black and going around creeping around in mourning. I said, come on, we've got another iPhone coming for you. We've ordered another one, it's insured. She said, but dad, you don't understand. I said, what's wrong? All my friends were in there. I said, no they're not. They're in there and that was her way of connecting with them. Can you see what they do with these technologies? They are a member of that digital tribe. They have that digital cultural capital to kind of use a version of Bourdieu's work, the idea of cultural capital inhabiting being involved in the habitus of that device. It's important to them. How do we actually help that membership of the tribe to become a pedagogical advantage to us? That's a big question for you to think about. And then there's the idea of the socialized aspects of don't buy your cat a mobile phone. They won't use it. They haven't got fingers big enough, alright. And if they did they wouldn't bother responding to you any of because they're too proud, aren't they cats? If you've got a cat, they wouldn't respond to you. But the thing is, students do. Here students taking notes. You may have seen this picture before or one like it. But I see wherever I go in audiences around the world is why I do a lot of travelling. And wherever I speak I see people holding up their cameras at certain points taking pictures of the slides. Because they want to capture the moment and they probably want to share it or do something else with it afterwards. My colleagues used to say to me, Steve that's not learning. I said no, but it's the first step towards learning. Because it's what they do with those images next that is important. And here's another idea for you that relates to that. Woodrow Wilson, formerly a U.S. president. He said this, made this statement here, which I think is poignant. I not only use the brains that I have, but all that I can borrow. There is a principle of connectivism. It's not so much what you know anymore that's important. You need to know so much, but it's about knowing where to find the rest of the knowledge exactly at the point you need it. So schools have to move away from just in case learning where everything is delivered to them and they have to try to remember it all and get tested on it all to the point where they're learning just enough learning or maybe just for me learning, a very personalised form of learning which allows them to cope with the challenges as they meet them. This I think is going to be a shift in pedagogy over the next few years. It's going to have to happen. The idea of community that you're involved with community where it's a group of people that kind of have a common notion of where they are and what they are. But today I think that we're in a place where location doesn't matter anymore. I'm trying to get this to work. In the digital age it's not so important where you are anymore. Yes, we come together in events like this and we come together to socialise and to meet each other and to have fun together and to learn from each other but we don't have to meet together to learn together and to know each other. We can connect through technology, the digital tribes that we become involved in. I just wanted to show you this picture, an illustration of that. I don't know if you can see what it is but it's actually Barack Obama before he became elected as the first black president of the United States. How we miss him, don't we? How we miss him? Before he was elected president he actually spoke in this place which was just outside the Brandenburg gates. I want you to try and see what the people in the background are doing or in the foreground are doing. Can anyone see what they're doing? They're actually doing something. They're all holding up cameras, that's right. There's even a guy up there somewhere if I can find him. He's up there, where is he? There he is there like, he's actually live streaming, he's holding up a laptop and live streaming the event. Which I think is amazing because all of these people, they knew that the whole of the world's media were there. This was being transmitted around the world by a satellite. Everyone was seeing this happening, it was a historic event but they wanted to capture personal mementos to share with people that they really cared about, people that meant something to them. And that should tell you something about why we are connected through technology today. And then it should show you some ideas perhaps or an inkling of an idea about how we might leverage the power of that. And there's a close up. This is about connecting and sharing. This is about this. I don't know if you can see this but if you were to stop the internet in one minute this is what you'd miss. So let me show you some. So up here for instance you've got one million views on Twitch. No, you do. Well done, the only one in the room. There must be another few million people out there doing the same thing. But some of the more common ones up here, 87,000 people tweeting every minute. Instagram nearly 400, well 347,000 scrolls on Instagram over here. Netflix nearly 700,000 hours watched. Google 3.8 million searches. 3 billion Facebook logins over here. Tinder 1.4 million swipes. It doesn't tell you whether they're left or right. But it does tell you what people are doing online. It's incredible what's happening. So in 60 minutes. I wanted to show you this because this is a model from Aito at Al. Which talks about why we connect and the pedagogy behind why we connect. It's an interesting very simple model which has three components to it as you can see there. And the thing that I wanted to point out to you here is that each of those components can be explained. In particular detail as to why they are pedagogically important. So learning occurs when there are strong connections between people with a space. That space can be virtual. And we have to start acknowledging these connections and why they're powerful for the young people that we're actually teaching now. I will make these slides available to you afterwards so you don't have to worry about it too much. The theory behind this is by Gotsky and in its first instance the idea that there are three zones. And the zone of proximal development is the really important one here which is where knowledgeable other people can help those who are less knowledgeable to expand their ability beyond what they'd be doing on their own. But I'd like to put a new spin on this because there is also the idea of technology scaffolding now that is available. The idea that technology can be used to scaffold learners when there is no immediate expert available to them. And I'll give you an example of this. This man here is Julius Jago who was the gold medal winner in the Javelin in the world championships. He also went on to win the silver medal in the Rio de Janeiro Olympics in was it 2018. It's 2018 or 2016 sorry. So this guy has won a gold medal and a silver medal in the world championships and the Olympics. How did he learn to throw the Javelin so well and so far? He learned his technique not by having a coach. He learned his technique by watching YouTube videos. It's well documented. Go and look him up. He learned to watch videos by people like Jan Zalesny and Steve Buckley. The great champions in Javelin at that time and he just simply copied their techniques. This is an example of what I would call digital scaffolding. It's available to all our students and here are the tools that help them to do that. The architecture of participation as various people such as Eugene Bosky have called it. This is a really simple idea that learning 2.0 is generally about creating content and sharing it. Modifying, remixing, repurposing and social media of that kind of nature gives everybody a voice. Even the quieter students get a voice. They want it. It's about well I suppose you could call it storytelling the idea of the narrative pedagogy. The idea that stories actually hold us together. They create within us all and wonder. Stories inform us, stories warn us. Forever we've been telling stories. When you look at the cave dwelling pictures in the caves in France, in southern France, you'll see people telling stories using images. That's exactly what we do as teachers. We tell stories and some of the stories you will see them online. Eden chat, which we set up several years ago, become very successful. It's still going. Who's been involved in Eden chat? One or two of you. A few more I hope in the coming days because Eden chat, if you go through the archive of what's been talked about over the past five or six years, you'll see that Eden chat goes into a lot of depth and a lot of breadth on all the burning issues of the day in terms of technology enhanced learning. Blimage was another one and twisted pair. These are two that I set up several years ago. Blimage is a blog image. Put the two words together, you get a blimage. Basically the idea behind that was to send somebody a picture or an image and then challenge them to write a blog post about learning based on that image. They would then post that up and challenge another person to do the same thing. This went like wildfire over the summer of 2017, I think it was. We had something like 300 blog posts in two weeks. Across the world, many organizations, many countries took part in it. People were saying to me, this is incredible because we've learned so much by just thinking hard about what the image might mean to us, what the metaphor might be and then telling a story about it. These things really work and they are in fact the digital tribes totem. It's what they gather around if you want to look at it in an ethnographic way. That's exactly what it is. I'm not going to go on too much longer with this. I'm just going to talk about communities of learning here. The idea that you can connect someone together with three other people. That is a beginning, but the network comes when it's the people that they connect with who connect with other people that actually becomes the powerful aspect of it. It's exponential. You might be following, I don't know, 20 people on Twitter, maybe 20 people are following you, but one of those people might have a thousand people following them and one of those people might have a 20,000 or 50 or 100 or even a million people following them. You can literally exponentially create content and share it and it can go viral. When students understand this, they get very excited. Some of my students, when I asked them to start blogging as a part of their professional practice, they began to start thinking about how can I disseminate this wider, how can I get bigger audiences for my work. I showed them the power of Twitter and the power of the other social media. Before long they were leveraging this and creating content and sharing it with huge audiences. When you have an audience for your work, it raises your game. They began to be much more careful about the way they were referencing their content and the way they were phrasing their sentences and the way they were constructing their arguments and the way they were being critical improved. You can see that students who had the audience were beginning to leverage the power of this. That was a different type of pedagogy completely. It was very powerful. Another one for you to think about. I was in New Zealand several years ago, a couple of years ago and I was working with a group of nurse educators in one of the universities down there and I had a two-week residency and one of the groups came to me and there were about seven or eight of them in the group and they said to me, look we've got a problem Steve. I said what's the problem? I said three thousand students. They said yes, they're all nurses and they're all medics and they're all midwives. They're coming in to train in all the professions, paramedics, physiotherapists, the lock. I said what do you do with them? They said we break them up into ten cohorts of 300 because the biggest room we've got is 300 and I said so what do you do next? They said we give them a lecture and it's one hour lecture on anatomy and physiology. I said so how many of them come along the second? One week. They said all about half and that's our problem. I said well how many times are you going to do this before you realise it's not working? They said well what's the answer? I said well think about how you can inspire your students to work hard for themselves. Think about how you can engage them without standing up like I'm doing now and lecturing. They said well what can we do? I said well off the top of my head what about an un-Googleable question? I said does that exist? I said well it does now, I've just made it up. An un-Googleable question, what does it mean? Well it means there are things out there that you cannot Google. They said rubbish, there's nothing out there you can't Google, I said yes there is. I said there's loads of stuff that you cannot Google. If it's not in the computer the computer doesn't know about it it's not going to give it to you when you Google it. Quite simple. I said well give us an example then I thought oh my gosh here we go. These seven people here are experts in anatomy and physiology. What am I going to do? What is there exactly five of in the normal human body? Now you're thinking aren't you? What's the five of in the normal human body? So someone said that's easy fingers. I said no wrong, you've got eight fingers, ten digits, 20 if you count your toes. I said so that's wrong. They said oh I've senses. I said no there's actually 32 senses in the human body. Look them up. I said oh yeah you're right. So then they started being much more esoteric about it and then someone said five lumbar vertebrae. That's our own way of describing them. I said there are more vertebrae than just the lumbar vertebrae. So in the end they started to get angry about it and they started to kind of you know they Googled it and they couldn't find anything. I said they are an un Googledable question you know it works. And so they said look what's the answer then? Because they were stumped. Very mind these are seven of the national experts in anatomy and physiology. I said well the simple answer is there are five lobes to the lung. There are three on one side and two on the other. We're asymmetric. And they went ah they were kicking themselves. And so I said but that's a gateway question. That's a question into deeper learning because they need to ask why are we asymmetric. And they said well that's because the heart slightly inclined to the left. I said yes so why is the heart slightly inclined to the left. And it went deeper and deeper and deeper from there. And they went we've got it. We're going to go away and we're going to do this. We're going to create un Googledable questions. We've got to struggle. Because learning is in the struggle. There's a sound bite for you. Learning is in the struggle. Make your students struggle. And they'll learn deeper. I'm going to finish in a moment. I just want to tell you a few more things here. There's some new theories that are emerging and some older theories which are being modified which describe these kind of things that we're talking about. When you're using technology we can talk about connectivism. You know what you know is not just what you know but what the whole network is involved in knowing as well and you can gain access to it. Paragodgy which is a leveling of Weigotskian theory where everyone teaches everyone something because everyone knows something but nobody knows everything. And then there's distributed cognition. You know the idea that we can learn across networks and across cultures and across communities by connecting with each other. And we crowd source our learning by building our personal learning networks by creating these kind of things that Stephen Johnson calls them. The idea of the individuals not being smart enough to solve problems but when the connective gets together it can solve the problem fairly easily because it's connecting together and using all the brain power in the room. I just want to finish by reading something from this book which I published. This is not a plug. I need to read this to you. This is a book I published a couple of months back. And it concerns something called creative destruction. Now the picture you see in front of you is Shiva which is a Hindu god. Now on the border between Switzerland and France you will find the European organization for nuclear research commonly known as CERN. CERN is the home of the large Hadron Collider a very large and expensive technology physicists use to research the origins of the universe. Outside in the forecourt of the main building stands a statue to the Hindu god Shiva. Shiva is in enigma because he is both the god of creativity and the god of destruction. An appropriate personality to stand at the gateway of the esoteric world of particle physics and atom smashing. Shiva is at once dynamic and static young and old some would conclude that he is the god of paradox and ambiguity and this is important as a principle for us to grasp because learning can be paradoxical and there is a great deal of ambiguity in our world to help all of us to come to terms with this we need to support the idea that learning can cope with uncertainty it must cope with uncertainty in short we need to prepare our students for just about anything that life throws at them it's not just about learning it's also about unlearning and relearning it's a constant cycle it's a constant iteration that we're involved in it's not linear it has many junctions and many circuits within it it's a complex world learning and none of us can describe it perfectly psychologists have tried for centuries but it's the most complex thing we do as humans it's a complex world learning and none of us can describe it perfectly and the three challenges that I want to end with on the back of that for you to think about are these the first one what can we teach our students that will never go out of date wow that's a big one isn't it when I was at school I learned that the universe sorry the solar system had nine planets no there's only eight so if I said to my teacher years ago there are only eight planets even if I said wrong there are nine planets they say no wrong there are eight so knowledge changes it's fluid second one how will we capitalize on rather than negate the potential of the students personal devices do we ban them from the classroom or do we say okay let's use them for a purpose let's leverage them where's the balance there only teachers in that context can answer that question and the final one how can we leverage existing connectedness to create new and dynamic learning environments so that students learn how to be digital citizens well I suppose first you've got to ask what is a digital citizen what does that mean and once you've decided on what that is then you try and work with that challenge I'm sorry there's the text is slightly gone over itself there I don't know why that is so there's my three challenges and I just close by saying thank you very much again I hope you enjoyed that I hope the challenges are going to be taken up please come and speak to me afterwards and I think we've got time for some questions so thank you very much thank you very much Steve I think you raised three interesting questions we do have time for interaction we do have to break for coffee at 11 so now the floor is yours the audience and flying microphones around so who can take the first challenge David Bourne from London the question for Bert if I may I think you may just have missed a small trick in there somewhere I know your emphasis was on testing but what I see behind testing is not judging performance it's students doing things students doing work students getting feedback on that in the work that they've done I wonder whether that's an illegitimate or a legitimate step out from what you've said because if I'm right it gives a much more inclusive idea of a productive learning environment a place where students constantly do things and get feedback on them whether it's in the formal context of a test or not we agree I mean evaluation or exams may have been presented here a bit as summative at the end I fully agree that evaluation should be permanent throughout the year throughout all the activities get as much feedback as possible so I think that's a very good point it's over there a question for Mr. Wilin so have you analysed data through predictive models do students errors predict participants performance was it for Steve or for Bart Bart Wilin sorry can you repeat the question please have you analysed any data through predictive models such as multiple regression and do the students errors predict their performance at the end so early performance predicts later performance we don't predict students performance by what we do of course I think you're referring to the tool that I showed at the end what the system does is first gathering all kinds of errors categorising them that's an algorithm if you want based on that what we can do is predict other things not the students performance that's pretty unpredictable but what people can do based on the results of the analysis do interventions on the students performance give specific courses or new exercises based on what they have found on the predictive side what we can do is based on the analysis that we do is say something about the difficulty level of a new text because if you enter a new text into the system based on previous errors we found we could predict the difficulty level of this text we know this and this already generated errors in the past so that we can do but we cannot predict the students performance based on that I don't think we will ever be able to do that thank you very much I think we can predict students performance through measurable tests so for example if I will talk from my own perspective because I'm an educationalist and I'm working with children's morphological and phonological awareness this is a linguistic issue so I'm testing children and then I see where their children's performance at the beginning of the test at the beginning of the period predicts their actual performance in reading and spelling after 8 months or 10 months so I can see where their weaknesses are related to their achievement at the end through multiple regression and I'm controlling for variables that relate to the two variables to the predictive variable and the outcome variable that's why I'm just asking if you have analyzed any data through these models just to know where their students errors actually relate to their performance very interesting it makes me think about the new research that we started with Professor Isabel de Lara from the University of Leuven in which we will try and see if we can predict try and find the best interventions needed to make the performance better I think this touches a little bit what you say we will try and see if we can find the best ways to help the students perform better but that's not predicting the performance itself but we will try to influence the performance of course based on what we found I cannot see very well but there is a lady in yellow in the middle of the stairs hello I'm from the University of Derby I'd like to bring both of your thoughts together actually testing yes space practice putting those interventions in place is the future I think personalize an adaptive learning is absolutely part of everybody's future not just in Europe but we see this globally actually I want to aim this question more at Steve to say we know artificial intelligence could be great in those interventions both remedial and stretch at edge cause in february everybody said this is a serious challenge we are not able to address this challenge yet so how far away do you think we are from having AI and remedial and stretch interventions that's a really interesting question because artificial intelligence is quite patchy in the way it's progressing some areas we are moving forward other areas it's stalled and I think we are still in the area of artificial narrow intelligence we haven't yet breached the Turing test gap if that makes sense to you all so we are still at the point where technology is intelligent to a point but it's not able to do the ground things we wanted to do like the forms of learning you are talking about there I think there will be a role for artificial intelligence in learning in a lot of different ways I mean Bert has already talked about things like learning analytics there are other areas that it's also going forward in like chat box for instance where we are for support for students and for encouragement and so on but I think we are a long way away I don't think I would put a date on it we are quite a distance away from what you are talking about adaptive is not AI we do adaptive things already based on our system based on the profile of a student you can give them a text that is specifically about the errors that they make that's adaptive teaching adaptive learning but if we are honest it's not AI based it's just a simple analytics algorithm that is behind that's not AI yet so way to go for the questions I have one question from sorry over there the last room for Steve what would you do if you were a language teacher what would I do if I was a language teacher wow I wouldn't panic because I think like I said in my speech I think we are still a long way away from perfection in terms of translation through smart technology I think it's very simplex at the moment it's in one direction you'd have to have two different people with the same device and I think we've also got this situation where the technology learns from accents and it learns from phrases and idioms as well so the best will in the world is going to be a long long time before that technology that Waverly has developed is to the point where it's near perfect so I think that there's always going to be a place for language teachers even if it does become perfect because I think people still although they'll have the immediate effect of going out into a marketplace in a foreign country and being able to understand and being able to speak and communicate I still think that they would like the cognitive gain of learning a language as well I think there's something that you achieve when you learn another language and you take joy in being able to speak it and it's a social thing as well so I think that I wouldn't panic over my shoulder because there will be some huge challenges ahead okay, thank you very much we have two things to do first thing is to give recognition to our two keynote speakers thank you very much we learnt a lot, Paragogy for me was new and thank you and please give the big applause the second one is I hope that our principal sponsor representative from Tel Avik is here is that correct so I do invite you on the stage to receive Eden annual conference health and blue appreciation for your very valuable support that we feel from the very beginning of the conference so please again give applause to the principal sponsor and we will issue this now we close our first plenary session please do enjoy a coffee break and then parallel sessions in the day