 Head of E-Concordia, affiliated with Concordia University, and with me today, we have Annick de Saint-Hilaire, who's our chief academic officer, and Yamna Ittes, our director for academic innovation. We're going to switch because we all need to access a computer at some point. I think it would be good like that. Thank you. Okay, so most people don't know about Concordia University. So here we are in Barcelona. If we make a quick trip to Montreal, this is where we are located in Montreal, Canada. We existed since about 2001, and our mission is to build all online courses, credited courses for Concordia University. We currently offer, we have an enrollment of online courses of 40,000 per year, and all our courses are developed in Moodle. We also work in partnership with other institution worldwide, and we create not only for Concordia University, but other universities as well in Canada, different type of learning experiences using Moodle, adding on top of the different type of technologies. So in our case, what we wanted to showcase today is about artificial intelligence. We've been working on that for quite a while. Actually, we have four use case that we want to present to you, but they're not based on the latest version. We're working now on projects for next year, but what we use in the use case are ChatGPT 3.5, because that was available at the time of the project, and IBM Watson Assistance Service. So all the use case are based on this different type of technology, and I hope the sound is going to work for later. I have a 30 second video to show you, and just a quick reminder that we use both platforms, ChatGPT 3.5 and IBM Watson Assistance Service at the lowest cost possible. So we didn't go with the full-fledged version at the time. It was free for a certain period of time, but also with a limited number of users for Watson. So what we've done is only use low-cost technology, not going full-fledged, because we wanted to assess how could we use it in different contexts, and also to see what the future will bring us. So if I go on with the first use case that we have, I will let Annick actually go on and explain what was our first use case. Thank you, Robert. Hello, everyone. I'm very happy and excited to be with you today. I'm very impressed also with having you all here. So as Robert was explaining, I will be presenting the first use case when we started exploring the idea of integrating some AI in our project. So I'll be presenting the Chatbot for student support. So there we go. So it will go like that. I will explain a little bit the challenge first, and after that, the technology that we use, and the result. So the challenge. Robert was telling you that we have 4,000 enrollment per year. So you can imagine the number 40,000 enrollment per year, and you can imagine the number of ticket coming from student to our support team. So and all those tickets, often, or email are often around the same topics, you know, especially at the beginning of the semester, so about the enrollment, about payment, different things. So our team was very busy answering those questions, and it could take some time, you know, for them to respond to the student. Sometimes it was kind of five minutes that they can have the time to respond, or it could take a day. So for sure, we put in place different things, you know, it has a FAQ and different strategy, but at the end, you know, often the student were sending some email to our support team. So we decided to explore the idea of implementing the AI so we can maybe help our student to have a quicker answer, and also reduce the load for our support team. So the technology we decided to use, so we explore different ideas, but we end up using IBM Watson Assistant, combined with Zendesk. So that way we could use, you know, already the question and answer that we were receiving, put in depth in the system, and with Zendesk we were able to have also the ticketing system. So at the end, the result, so we were very happy integrating that in our platform, so you can see here the landing page for the student. They can find the icon just there in the bottom, so we call it ACME for Econcordia AMI, AMI means friend. So student can just select the topic that they're interested in having an answer, and after that, so you have different screenshots, we decided just to present some screenshots like that, but so they can, with question, yes or no, and with the topic, have the answer. If at the end they're not satisfied with the answer, they can always come and chat with a live agent. So we put that in place, and the result was very amazing because after a year, we started that a year ago, so after a year, the support ticket was reduced by 35%. So it helps really much our support team answering the student, and we did also internal survey with our student that shows that 46% prefer resolving issues via the chatbot, and 30% prefer looking at the answer themselves in the health center. So, and as I was saying, they can all the time connect to a live agent. So for the second use case, I will let Yamna talk about it. Thank you so much. Thank you, Annika and Robert. Hello everybody. So I'm working at the company as a model expert. So of course, I'm working a lot on model and supporting the team using model and the professor from Concordia and faculty members. So the second use case and the third use case I'm going to present, they are going to, we are going to present chatbots with IBM Watson Assistant. I use the free version of IBM Watson Assistant. It means you can use it also and you can try it for free and you can integrate it to your model courses, MOOCs and trainings. I hope I am going to be able to inspire you to create wonderful chatbots and well, as we are saying today, we would like artificial intelligence to collaborate with profs, with all of us to give a better experience for the student. So the first use case, the second actually use case, the first one for me, I am going to talk about the challenge, the technologies used and the results. So everybody knows model and okay, model is full of nice components, a lot of graded and ungraded activities. In addition to that, we can add permission, delay permission, play with the settings. So we can create different kinds of activities with the same component for different scenario. So for the team, it was a little bit difficult to choose which component we are going to use, with which settings to be able to answer a question from the prof or to create a new scenario. So we're thinking it's going to be easy if we create a decision tree or a map like this one. And we started with a single question, is it a group assignment? The answer are going to be yes or no. After that, we ask it another question, does it need peer grading? And answers are yes or no. Of course, you know, all of you model assignment components and those questions are usually asked. But reality is more complex than that. So we ended up creating this map and this is the first one. We have seven like that. Even me, I was working on this map and I was kind of lost. Oh my God, what is this? Where is the component? Which one is that? So with setting, we added notes and I ended up adding numbers to those components to be able to figure out where is each one. So of course, you are going to tell me this is not possible to share this kind of diagrams with teams. They will be confused for sure. And I will end up getting more question from the team. So, okay, what are we going to do? So the idea was to create a chatbot with something called in IBM Watson Assistant action skills. And action skills is going to help us to create those kinds of scenario. It means you have question and answers and each time you have an answer, yes or no. You add another question and like that you create the whole scenario. And thanks to Moodle, we were able to add all the documentation and steps and settings. You know that. So the results are, of course, our chatbot called activities recommender and a Moodle course with all the documentation needed for the team. So as you can see here, we grouped all the recommendation in four main categories, manually graded assessment, automatically graded assessment and graded knowledge checks and graded interactive elements. So the map you saw, it's now embedded in this chatbot. So we don't need to go through all those branching scenario. Just we go there, look for the category we need, answer question, yes or no. After that, the chatbot will provide the recommendation with the link to the documentation. As you can see, the link is here. Oh, okay, you can see that here. And this is the documentation, of course. Thanks to Moodle. So the first, the second, my second use case is about using a chatbot, which is called Professor Support. But this is just an example. It was a prototype we created initially to explore the feature of IBM Watson Assistant before the development of ACME. So I hope this is going to inspire you. So I'm going to talk about the challenge. So at Econcordia during the pandemic, we were supporting professors to develop online courses with Moodle, of course. But the more we go with the development of courses with the pros, the more we understand that pros, they don't have all the skills, all the knowledge to develop their own courses. So we created a Moodle course. It was a training, having all the documentation with links, with text, with videos, with everything. And the space was growing, growing, growing. And we ended up saying, okay, this is too complex now for the prof to be used. And it is hard to navigate by the end of the day. So what we can do to help them to navigate our documentation, to answer their question, and to motivate them to see what are the other features in Moodle they can use to inspire them with scenario and with examples. So we developed the chatbot with IBM Watson Assistant with each dialogue skill, which is a basic feature in IBM Watson Assistant. It means you can create your chatbot, like let's say after reading the documentation on IBM website, after a day, you will be able to create your chatbot. You don't need coding skills, just basic skills in HTML if you want to add links or videos, but not more than that, it's easy. And it's going to make the courses more interactive and engaging. So the results are here. So this is our professor support. So as you can see in the chatbot itself, we can have text, we can have option, and the users, they can select one of the options. We can display images, we can have links, we can have videos, as you can see here. I'm asking about group assignments in Moodle, so if the prof did that, so we have the answer with the video, with the text, and with the link to the documentation in Moodle to follow the steps to create a group assignment. So now we are going to move to something else, not IBM Watson Assistant, and we will see that with Robert. Thank you. Thank you, so I'm not as good as you. I'm going to use technology to impress you. So basically the first use cases we showed you are meant to help either our development team in how to pick the right type of activities within Moodle and the other is how to use it for a faculty member who wants to use Moodle with their own pedagogy and the way that they would teach. In this case, we're addressing more the students. So we try to combine two different types of technologies, VR and AI. It was interesting for us to see the example exactly how could we link an AI agent into a VR avatar. We've been working with VR for quite a long time now and even XR, I'll give you a little teaser of what we're working on and maybe we'll present that next year if we're invited. But let's say that we wanted to test the interaction between the students and a VR environment, an immersive environment and use the fully immersive power of VR to recreate a meeting in time and space. And the scenario is for a physics course actually and it's a physics 101 and it's a what if scenario if you could have a direct interaction with a character. So what we've done is we put that in a blended course. What we've used here is chatGPT 3.5 once again that we've put together with text to speech and speech to text environment, unity and unreal that you know I'm sure about creating VR and Microsoft Azure that was used also as that I did. I would say the infrastructure behind it. But all I did was are can I interact in real voice? So not a prom but basically talking and getting an answer from the avatar within the VR environment and what we've decided to do here is to have I hope the sound will work is to have a conversation with Sir Isaac Newton about gravity. Okay, so we're not going to have sound not the first time this happens actually. And so basically what we've done is we put in a virtual environment and you can have then we put the full caption there. So it's basically the conversation we have with Sir Isaac Newton where he's going to explain true questions so it's actually live interaction using the prompt as in a speech to text interaction and then you will have Isaac Newton explaining exactly what is the law of gravitation. You can have the notes, you can actually ask the questions on anything you would like whatever is on your mind and try to test because the interface is actually chat GPT. So you can have real interaction with an avatar. What you're saying right now it was like film directly through the avatar, not that the avatar but the VR headset. So you can even try most students what they do is they try to ask questions to that the avatar wouldn't answer. So what do you think about the internet? Or in this case please say hi to all the friends currently attending the Moodle Moodle conference in Barcelona. And of course Sir Isaac Newton will say hi to and hello to all the people attending here in Barcelona. So we've done it through the immersive environment. You can do it without the immersive environment and actually go oops, sorry about that. I'm going to go to the next one. So you can actually do it the same way with the video version. The video version actually use a different version of the avatar much more realistic. It's interesting because at the time when we decided to use an avatar to create interaction we had to send to chat GPT the example of the avatar so it doesn't look too human. Because it's very important that we don't have an avatar that would actually be considered as human. So this is about the best that we can get. So we're working a lot with both VR and AI. Thank you. And maybe just to give you a small glimpse of the other things that yes, thank you Sir Isaac. We can't hear you. Okay, so the results. Users at the beginning were a bit reluctant to interact with the model after the initial use. They were really engaged. They considered this almost like gamification within the course and there is a huge engagement in the course that they were attending. Of course you have factual errors. I mean chat GPT is far from being perfect although you try to put constraints and limitations on the answers you could get. This is simply impossible to control it totally. So this is something you have to take into consideration. And once again it was a huge engagement from the students when we tried it and tested with them. So other explorations very quickly that we are currently working on is we're currently producing with actually a Paris University, Paris-Sakley University. We're working also an immersive environment on the complete operation room. So this is basically to train young surgeons on different things they will do. This is a totally immersive environment once again that will use AI and also the replication and deeply immersive replication of an operation room that will be embedded in some of the med school that we're doing. We're trying to use VR and AI as much as we can. We do have lessons learned and I will conclude on that. It will take me 37 seconds. So first an interest trick and fine tune AI to very, very specific domain. And it's one of the reason we use Watson actually. Second, we didn't explore more of the data privacy issue. This is something that should also be considered because we're charging PT. There is definitely an issue about that. It's crucial to have a very clear on this understanding of why do we use that? It's not simply because it's nice and it looks good. What is the pedagogical goal that you want to pursue if you want to use such an immersive technology? And then AI system requires high validation and data accuracy. Most people, they would actually under evaluate the effort required in the validation of the data that you want to input in an AI system. SSAI for the furnace transparency and ethical implication and finally don't improvise. It needs to be part of a complete strategy that either the institution, the department, the university and the college needs to have in this evolving landscape of AI and to measure the potential impact it will have on teaching and learning. I'm done. Are we okay? On time, thank you. We do have a couple of minutes before it's done. Yep, do we have any questions? How do you integrate those VR technology in Moodle? Is it using as a SCORM package? It's connected to the data? Exactly, everything we do is used as a SCORM package and that's it. We just, most of the time we have to do it with a video of the VR environment because the VR headsets are not available everywhere. Okay, thank you. Okay, oh, it's over there. Thank you very much for the presentation. How did you check the quality of the output produced by the system in the several use case that you explained and you demonstrated before? We're still testing it. I mean, what I showed you as use cases are tests, they are not totally, I mean, they're completed, but we are still working with faculty members and see where it will be used and how can we implement it on the campus. So I don't have the, I cannot give you the results right now because it's a work in progress. Okay. Just something to add. This is about chat GPT, but for IBM Watson, the answers are added by the developer. It means you decide to give whatever answer to whatever question. So there is basically nothing to test because you add the information and it's accurate. Can you, yep. And it's also the limitation. One more. Is there a huge difference between chat GPT and Watson in its prompting behavior or results? Well, there are two very different technologies. Chat GPT is a generative agent where it will go and find the information everywhere. In Watson, you have to decide manually what type of information you want to put in the database and it will be limited to that. So it's also, it's very strong because the more information you get in there, it can be quite complete. On the other hand, it will be limited to what you have in the database. So it's two different technology. One, you control totally the context and I would say the content, which is very good. But then you're limited to that. And the other one, you will use a generative agent that will pick whatever they find is interesting, make it look good and put it back to you. So two different approaches. And this is a large debate internally. We have Ms. Watson here and you have Mr. Chat GPT here. And then Nick is in between. Are we okay? Thank you so much. Thank you.