 謝謝 everyone, my name is Nadav. I'm coming from Israel, the department of science teaching at the white man institute. I'm going to show you today the group. It's an AI tool that we developed to group and personalize students' performance and help teachers be like an assistant for the teachers. We also be sharing this tool with everyone, זה יהיה אופן סורס קצת קצת, איפה קצת שאני שומעת כי זה ככה קצת שמובעת במה שזה עובד זו דיברטמנט אתם יכולים לראות, שבאדגת של גרדינג הוא יותר גרדינג מהגרדינג של רשארט פשוט, כן כשבדגת כל רשארט, אנחנו צריכים לך לגרדינג של גרדינג כשאנחנו לא גרנינג, אז אנחנו עובדים רשתות ודברות, את האחר שאני אעשה לך, אנחנו גם עבירים פרופורטי פרופסייניים לסיינסתים של המדעים, ואנחנו הרבה פעולים לסיינסתים, נשאציה, ולמה פעולים לסיינסתים. These are the different disciplines. We are responsible for, and one of our major platform based on Moodle, but heavily customized, is called Pettel. It's a combination I showed it a few years ago in one of the conferences, so maybe some of you know it already. It's a combination of OER catalogue on one hand, it's a Moodle plugin, and also a social network, another Moodle plugin. The OER catalogue is a little bit similar to the Moodle net, but for more activities, and you can preview activities, copy activities, share different stuff, I will show you later. And the social network, another way for teachers to engage with each other and share activities. Why it is so important, again for you maybe taking this tool back to your institutions. The base of this AI working is depending on the data coming from a special activity that is duplicated many times and used by many teachers. If you have different activities, this will not work, so consider it when you thinking how you wish to implement this in your institution. So, for a long time everything was peaceful, and then from time to time researchers come into the department, and Tanya came into the department and wanted to break everything up, talk to the teachers, how can we improve, how can we help you, Giora is supervising, is the PI supervising Giora's research, and I'm just trying not to break anything, and like any good research also looking at what Martin is doing, we try to use AI, we don't just think with our own brains, we try to use AI, and we use Dali, everybody probably knows Dali, you try to describe some kind of scenario and you get a picture, and we ask Dali, this open AI, please draw us a picture, sorry, please draw us pictures of research asking teachers what's their greatest challenge in teaching students. As you can see from the nice pictures, telepathically try to direct us, you should probably go to different groups of teachers and ask this, not just a few of them, and asking this question to the teachers, we got a few responses, but one of them that really was interesting and related to something we can solve with AI, teacher wanted to help students understand the gaps early on with misunderstanding or misconceptions, and this is what gave us the idea that we should identify students' knowledge, put them in profiles, all this in real time, so it will help teachers immediately when they are testing or diagnosing kids, students in classes, and then also provide the teachers tailored responses for each profile. This is to scare you a little bit, but I think if you are teaching you should probably know this, this is the metrics of grades you see in the quiz, a lot of questions probably and all those students, how can you tell if two students got 72 or I don't know, other grades, how can you tell if they have misconceptions or misunderstanding on the same subject? You actually don't, you need to go very carefully and try to put together little clusters of students' performance and try to understand what's the main characteristic of this kind of group. This is where we try to partner the teacher and AI and give a solution, so Tania and Giora took some time and did a lot of research and finally discovered or came up with ideas and algorithms how to use machine learning, machine learning as you know is the basic part of AI. The deep learning is the one that you use with a lot of data, but machine learning is a little bit simpler. The whole process was very easy because I wasn't involved, I just got the bottom line, this is what you should do. Then started the real challenge, we took developers, UX, UI experts and the teachers as you can see in this picture and we tried to develop the interface, so it would be very easy for the teachers daily to use it because sometimes it's hard to understand what the AI is giving us as output and this is the interface, actually this is the tool in Moodle that eventually we came up with. You choose an activity, a quiz in your course and it immediately clusters, group together different students according to their profile, some of them are failing at something, some of them are successful in some other questions but they have similar properties and something also interesting for teachers that want to know how we came up, how the AI, some kind of explainability, how the AI came with those groups, we showed them the entire data set, it's a nationwide data set of the same quiz giving to hundreds of teachers with thousands of students and you can see for each cluster, for each group, it's the columns if you want to see and the questions, sorry for the Hebrew, I couldn't translate it and the questions you can see some are failing, some are partially failing, some are successful in some areas so teacher can a little bit understand and relate to the groups, the AI because there is sometimes a lot of rejection from the teachers, they know the students I'm not sure why these students is in group A or B, it is not logical because of this we also enable them to drag and drop students if they think they know the student better than the AI, another thing and when the teacher is clicking or hovering over the I icon you can see a little bit more explanation for the teachers so they know all the students in this group, in this cluster they probably have some difficulties, this is physics so they have some difficulties applying Newton's law, Newton's third law and another thing, interpreting data from figures so this gives the teacher some idea of what's the common property dominators of these students now after diagnosing those students, we want to take it further we want to give them some follow up activities, if they are very good we want to give them something more advanced, if they have some difficulties we want to get them training again so we have three options, one of them I don't like because it enables the teacher to take all the list of the students offline and give them some activity I don't like it because I don't have the data and if I don't have the data I'm not happy, I cannot do anything but teachers like it, they have some magic voodoo and they know how to teach them whatever and the second option is recommendations by the other teachers I will immediately show it to you and the last one is the teacher can get recommendations from the OER catalog if you remember the first slides we have an OER catalog a lot of metadata we're using one of Mike Churchwood's metadata plugin amazing plugin so it took us a long time to add metadata to OER objects and this is how we can recommend the AI can recommend the teachers what could be the follow up activity and let's see that this is a small snapshot of the OER catalog you can see the whole syllabus all the filters teacher can just pick whatever it first recommend what it should probably give the students as follow up but teachers can freely do whatever they like after the teacher chooses the activity they can also choose from their own course if they have something that is not on the OER catalog they can... what? 10 more hours I see, okay good sorry after a teacher is choosing the activity it also being asked to describe it a little bit for the following two teachers let's... oh, so different teachers did it and once the teachers explain why they choose to use that follow up activity other teachers coming to the same activity can see a list of teachers they know with their explanation and they can say okay let's do what they did earlier and allocate this activity it goes immediately in the course flow in the same unit just after the activity that they failed or misunderstood or knew better and okay, that was the explanation how the thing is working how the tool is working a little bit of statistics and response feedback from the teachers all the positives the negatives one we just didn't show you so, you know, you get the thing and we published it in a few places so if you want to know all the research behind it all the data, the raw data it's all out there what next? this was only for closed questions because the AI need numbers it cannot analyze abstract things relatively the next thing we are now also pushing into production is the ability to get open text open questions, use NLP it's a little bit difficult because it's Hebrew NLP you see it in English but it's Hebrew NLP and it, you know, gives meaning to parts of the text and this is based on deep learning not just machine learning and it goes and can tell the teacher what rubrics what rubrics the categories the students was using in I hope I'm explaining myself correctly were using in the answer it's very, very interesting a little bit scary for teachers to see that the machine can understand what the student is actually writing about the answer but also, like everybody else said before this is an assistant for the teachers it's only recommending the teacher what he should probably give grade to the students for that's it some roadmap for the future basically we would love everybody to use it we'll get a lot of data everybody will be happy not just us the bigger the data accurate the calculations this is the roadmap in very, very short you will probably see there a lot of people were involved credit to everybody it's not just only a few of us and I hope no time for questions good, okay you know where I'm standing outside everybody can ask any questions