 Thank you all for registering we had about a hundred and something register everything to this and then with the zoom issues and with the with campus Wi-Fi being what it is We're expecting maybe some more people will come through over the next little while today session is around about an hour and a half and we're going to be talking specifically about Chatbots and them even more specifically chatbots in teaching and learning I Am coming from from UBC Vancouver today, which is located on the traditional ancestral and unseeded territories of the Musqueam people There is a worksheet which you can follow along There is a QR code there or hopefully someone can just pop this in the chat for me and you can all follow along I think we sent most people who received the The the reminder email we sent you a link to a chatbot which we have Created specifically for this session on Poe Hopefully for those people that did receive the email to we asked you to sign up for an account on Poe.com So Poe.com if you don't have one, please by all means go and go and grab one now If you don't feel comfortable creating an account on Poe.com that's that's fine Later on today. We'll actually be putting you into breakout rooms with sort of three or four Of you and hopefully at least one of you will have an account on Poe.com by that point I'm so you can sort of get carry along with the activity so that's the worksheet for today and We're gonna jump straight in to talk essentially about definitions because I don't know How some of you folks feel but if you've been around the JNAI space for the last sort of 1215 months You'd have seen a lot of words and terms that seem really familiar Like you've seen some of these words and you're like well Why is my own language being co-opted? Why is now? Why is now something mean? Something else entirely different. What's happening? And it's actually why we wrote a glossary of terms on our website which Is proving fairly popular And we've written I think 20 or so different terms one of which is agents and one of which further down here is chatbots Both of which will actually be talking about today in detail So I'm gonna start with a definition because I think in this space at the moment actually making sure we're all we all know What we're talking about is actually really quite important So the two specific things we're gonna be talking about are chatbots or JNAI chatbots and agents So a chatbot is probably some things that you have Used or at least seen many times over the last at least in the last 12 months, but probably over the last The last number of years We're gonna be talking specifically today about JNAI chatbots Which have only really become a thing since the advent of JNAI around about 16 to 18 months ago And then we're gonna be talking about agents. So let's first let's get on to what chatbots I'd start with Chatbots have existed since the 60s, which might sound a bit strange considering they've only really been here for It's sort of at least front and center for the last 18 months and the first one was called Eliza who Was made it MIT in 1966 and it was actually a psychotherapist Scripted response bot. So what that means is the bot was looking for very specific phrases in something that you might My type and then would reply in a very scripted way based on those those responses And things evolved over the next sort of decade ish through parry and jabberwacky and dr. Spatzo and Alice and they were all actually Along the lines of mental health work. I don't really know what sort of drove that but Harry and Jabberwacky and dr. Spatzo were all basically focused on Walking specifically about mental health and I think it might be because some of the language used for these things Is very prescriptive as in it could be literally coded for with specific responses and specific things to look for specific words At least to look for And then again as time went on we're going through the 90s now and in the early aughts We eventually got to sort of 2010 when Siri was developed by Apple And it became the first widely adopted personal assistant, but it could use natural language both for Receiving and for giving out so it could actually understand natural language and it could respond in something that Was at least pseudo natural language and if any of you view Siri over the last 14 years We'll probably just as frustrated as you were 14 years ago with it But we're told everything's gonna be better later this year. So we'll say And then a little bit later so seven years later actually in 2017 things changed and We got to what I think is a seminal paper by folks at DeepMind and the paper was titled attention is all you need and I think going forward in in a few years time Maybe maybe I don't know half a decade a decade. I think this particular research paper will be I Genuinely believe will be a turning point in terms of Where we where we see not just chatbots, but I think where Portions of society are going to change and I think it actually comes from this paper this paper Changed everything when it comes to machine learning and certainly when it comes to generative AI and chatbots I highly encourage you to give it a read actually. It's a it's a remarkably readable paper For something that is so deep and so complex It is it is well well worth well worth the read So that is kind of the history of chatbots, but it doesn't really define them what what our chatbots Chatbots really are where you as a human can type and you expect a response a helpful useful response normally traditionally actually prior to 18 months ago you would get a Very fixed answer so it would be looking for certain keywords that you would type in and you would get a very scripted response Since the advent of generative AI those responses have become more human like and that is the goal of Generative AI to create human like content whether it be text in the form of chatbots or images or video or audio but it's all something that is at least human like in content and Generative AI has really helped both the Understanding of natural language as in natural language processing in terms of being able to listen or understand I'm using very much inverted commas when I say that The words that you are using when you're asking questions or making statements and then also in the responses So you're able to have more deep meaningful conversations with these chatbots now because of the deep rooted again understanding of natural language So that's what we're going to say as a chatbot a chatbot is something that you can talk to normally via text you can't there some of them you can actually Speak to now audibly and get a response either in text or audio But that's what we're going to define as a gen AI chatbot It's going to be something that you can speak to in natural language and expect to receive a response in natural language The difference them primary difference between a chatbot that you'll have seen sort of over the last ten years and something that is that is happening right now is Underneath there is a there is a large language model those large language models basically allows us allow us to actually have a half decent understanding of the language itself and it can interpret very specific phrases or parts of text and Actually output real language that a human might use agents which is the second part of this Talk today are Slightly different in terms of an agent you can think of as a chatbot, but it is a very focused Chatbot that has a very specific skill set Agents can be deployed standalone. So what that means is they can be deployed on their own Just like a regular chatbot would be and that that that chatbot in that regard will be very good at one specific thing but they're much more useful when they're allowed to communicate with each other and Are given a task as a whole that they can autonomously accomplish So it goes from taking a single chatbot and you having a conversation with that chatbot back and forth To having several chatbots, which you give a goal or a task to and they communicate with each other And then spit something out whether that be Anything essentially and we'll see a few demonstrations later of what those might be but it all happens autonomously without your without your Input during the process unless you want to in which case you can actually have some input in the process But you give it a task and each one of those independent agents which are given a specific goal as in they are specifically good at one thing They communicate with each other and produce something Okay, great. So I'm back And I just want to quickly introduce myself and then I'm gonna jump in so I'm Lucas right I'm a senior educational consultant at the CTLT and I've been working here on generative AI for about the last year and a half And I think I'm hoping to share Some of the knowledge I've gained from research as well as from doing a lot of workshops around this area And I think through those workshops. It's such a privilege to get to learn from other folks So let's talk a little bit about chatbots And I'm gonna go through this stuff with you for about the next 15 minutes and then we have some Interactions and today what we're talking about is the idea of customized chatbots as rich as eloquently introduced and so in a few minutes, I'll share some examples of customized chatbots, but The idea of these chatbots that we can customize with our own data and with our own prompts Right away offers some interesting opportunities The way that we've been starting to see these chatbots are Being used as homework and study assistants. I'm gonna show you an example of that in a minute Personalized learning tools so we can really start personalizing the learning that students do beyond the large language models So beyond what chat GPT can do we can start personalizing with our own content with our own course content Skill development tools, I'm gonna demo one of those in a few minutes But again these ideas that we can help customize students experience learning specific skills Virtual teaching assistants, so I think this does bring up some interesting equity issues But the idea of tuning and creating chatbots that can be used in a course as a teaching assistant Course assistants almost acting like a syllabus assistant So they can help guide students through the specific courses and as I'll demo in a little bit an advising chatbot So chatbots that can advise students about enrollments in the case of the example. I'll share science advising chatbot and I wanted to share this quote from The Microsoft future of work report So I've linked this in the document that I've shared with you But I think I find this really fascinating is that you know as we're thinking about all the efficient things that chatbots can help us with I think there's also a larger movement going on and I really like how Microsoft's put this The digital knowledge is starting to move from document to dialogues So knowledge is no longer just embedded in Spreadsheets and text we're thinking about what knowledge looks like when it's embedded in conversation So, you know as you think of your teaching or your design practice I think it's worth thinking in the near term how it can change how we design a course How it can change how we teach but also thinking in the far term How is it going to change the whole idea of a course and the whole idea of what teaching might be? Some value of custom chatbots We can customize them for specific teaching contexts and for specific domains I think one of the challenges we're running into now with using chat GPT using Bing AI is It's really hard to define the content that students are getting and sometimes they're getting answers out of context We can incorporate validated data. We know that these models hallucinate We know that the data they share isn't always the data we want We can think about creating chatbots now that can share very specific data We can also use them as a way to reduce bias again Go into chat GPT search typical Canadian family and ask for a screenshot And you're gonna right away see the bias that pulls up in these tools How can we use custom chatbots to tune them ourselves and reduce some of that bias and kind of that larger piece is how can we personalize teaching and learning but I think more than anything I've ever talked about or any workshops I've ever done is the ethical issues and limitations with generative AI are so big that it's a very big It's very difficult to reconcile these and I want to talk about a couple specific issues and Limitations that we're running into with custom chatbots One is when we start thinking about building a chatbot that can be used by our students without supervision we run into the issue of hallucinations and Thinking about responsibility and there's a term for these generative AI They're a hundred percent accurate, but maybe 70 or 80 percent confident and this creates some really challenging issues Air Canada ran into this recently. I don't think they were using generative AI for this chatbot It was a more traditional chatbot But they were have found legally liable for the advice that the chatbot gave a customer And this this was there was a court case around this so we know these tools hallucinate They knew we know they don't always provide correct advice And we are responsible for this so what does this mean in the academic context? context we know that students if they're using it as a tutoring tool getting incorrect answers may affect their academic performance and How do we think around this if they're using it for application processes a custom chatbot that gives them the wrong information May make the difference between them being accepted or not and how do we deal with? Responsibility around this and if they're using a chatbot for career guidance What if they're misguided in their career and again? How can we work with them to help reduce those issues? Also help prepare students perhaps that there may be Issues when we create them and just overall how do we deal with that responsibility? number two is similar to generative AI which These tools are quite leaky right now in terms of privacy yet We know that there's lots of institutional data going into these tools and recently with chat GPT. I don't know if anyone saw this Hack that happened is someone asked chat GPT to repeat the word poem forever Which I find totally fascinating that this worked instead of writing the word poem forever It started releasing personal information from its database so similar to how the large generative models are leaky we also know that the chatbots the custom chatbox say GPT's also have been leaky and researchers have been able to Find out the data that they were being fed Find out the prompts that there were that were used to create them Just by using front-end commands so kind of two levels of privacy one these chatbots do Leverage large language models. We know that they're leaky and secondly the chatbots themselves have privacy implications The third example is a little bit more future-facing but it's the idea of go students and This is happening in the admissions area quite a bit now in the states where we're having bots go through the admission process To receive Awards etc. We know that this happens now with Taylor Swift tickets where bots are Flooding Ticketmaster and buying up all the Taylor Swift tickets But what does it mean when a bot might be able to enroll in a university course past the course? Completely online course without people realizing it's a bot and how do we deal with this sort of challenge? You know, there are other issues to talk about such as bias within these tools But I think it's worth keeping these issues in mind as we go through and I now want to share a couple examples with you of What these custom bots look like and then we'll do an activity around that where you get to try a couple of the examples? So the first bot I wanted to share with you is a Customized bot made by Tony Bates. Could I get a thumbs up in The using your reactions if you've heard of Tony Bates or read any of Tony Bates's work Just give everyone a second to do that Great, so we have one or two folks have Wonderful so Tony used to be the run the distance and learning program at UBC and Kind of to experiment with bots Tony created his own Tony bot and he used his website to tune this bot and to train it so that it calls on the data from the website I think this is an example I'll get you to play with it in a moment of what these bots can be so I'm just sharing the Tony bot and I'm gonna say is it okay or Should a faculty member do course design with a team and If you know Tony Bates's work, he really believes in using teams to develop courses Rather than having individual faculty do it and let's see what we get from here Yeah, so he does say that we should start working on a team because by using a team It's going to give us his benefits and if you've read his blog You'll know that this is important for Tony not to have kind of a lone ranger approach to course design But allow a team to do that for him the next example of a chat bot I wanted to share with you is called AI tutor pro and this was developed by contact north, which is a governmental organization in Ontario that works around teaching and learning and the goal of this bot is to help around Learning new topics. So their example here is safe driving. So I'm going to keep using this topic and If I click safe driving, you'll see I can do this in At a different level so professional high school or elementary level Let's try it for professional level and I can also decide on the response language So what it's going to start doing now is it's going to start quizzing me on my skills about driving So when you're driving you on overtake the car in front of you Which mirror should you check first before you start to pass and what should you do after checking the mirror? I'm going to say side view My side view mirror is broken right now. So I don't know if that applies, but let's give that a try and It's going to give me an answer So now we're getting into this idea of a custom bot that can be used for tutoring and when we think of Teaching and learning and education if you have a chance to read Bloom six Sigma problem Which is a 90 a paper he wrote in the 1980s that compares classroom teaching to tutoring We know that tutoring tends to be a more efficient way of learning So what does it mean to be able to create scalable tutoring bots that are customized to our particular situation So I'll give you a couple more examples and then I'll give you a chance to play with these bots as well The idea of a tutoring chat bot I think this is one of the most powerful examples Although we can use tools like chat GPT for tutoring it takes a fair bit of prompting So by building a custom chat bot we can have more specific answers. This screenshot is from con mingo It's not available in the in Canada yet But it's available in the US and what that is is it's a con Academy tutor that allows students to Get tutoring on different math subjects as well. It's been customized I think using chat GPT in the back end But it allows for a tutoring experience the second example I wanted to share you share with you is out of Stanford This is a virtual course assistant chat bot in canvas and what they've done for this chat bot is they fed it with the course syllabus And with course information so rather than students rely on the syllabus now They're able to query the chat bot to get specific information about the course the third example is from the UBC kick project kick is a is a Partnership between Amazon and UBC and this is a science advising chat bot and if you take a look at the screen here, you'll see that it's able it's been trained with specific data around the science program at UBC and Students are able to get information about specializations About programs and about approaches by querying the bot and what's interesting about this bot Is it's dealt with privacy quite a bit by using the Amazon cloud? So we're not sharing it into Chat GPT, etc. And it's also been specifically trained with data in this area Rather than giving generic answers So that's a couple examples and what I'd like you to do now is if you can go to the worksheet and Take a look at activity one called try a chat bot I've put a link to the Tony Bates chat bot ask the Tony chat bot about anything about online learning you might even ask some personal information about Tony see what happens and Also try the AI tutor chat bot just to get a feeling for it So I'm going to give you about four minutes to give those chat bots a try and once you do that I'd like folks to share what this could mean for teaching what it could mean for course design and What I want to look at now with you is GPT's and GPT's are a Type of custom bot that we can create Using chat GPT. I think one of the challenges here and I didn't talk about this Before is it does start bringing equity issues into play because GPT's are a play to pay Sorry pay to play my pay 20 Canadian a month or sorry 20 American a month to use them and So not all of our students will have access to them They're also not open so I can't just share a GPT with you so that you can use it So I'll demo a couple of these and then we're going to move on to the poll platform and then have you use that to develop a bot so GPT's are Ways a custom versions of chat GPT and dolly that use the large language model that open AI created but you can also customize the prompts that they're created with as Well as add documents or websites to them so that they can use that information in their responses So I'm going to demo a couple of them a couple. I wanted to highlight for you One is tutor me which is created by the Khan Academy again And it's kind of a mini version of Khan Mingo which has been set up for tutoring The other one is scholar GPT which is fed with scholarly research So it's able to answer questions based on scholarship rather than the current GPT model only So I'm just going to demo a couple for you, but could I get a thumbs up if you've used different GPT's in the past Let me just kind of get a show of thumbs here Great. So I see Leah has Looks like a couple folks have and can I also get a thumbs up if you've created one before Okay, I don't see too many there yet So just a couple I wanted to kind of bring to your attention. Yeah, Flynn Flynn gave us a thumbs up as well what's nice about the GPT's and What I've noticed recently is they've really expanded in terms of the offerings that they have So a couple that I'll mention to you. This is tutor me and if I want to start on a GPT I can just open it like this and I'm gonna give it ask for 10 practice problems What subject would I like to practice? Let's say calculus It's probably gonna ask me for a level And again, this is going to be customized by Khan Academy Stalling for me a little bit here. We'll see if it goes or if we just need to move on I'll give it another few seconds and then we can just jump on to another GPT All right, let's just keep going on to other ones here now. That didn't seem to work for us One month great So another couple that I wanted to mention to you is Wolfram Alpha and I think this is an important one to understand How to use but also that it is there if we're thinking about academic integrity and our students and what they can produce Wolfram Wolfram Alpha is a computational engine and One of the challenges of chat GPT is that it can have real weak spots around math around graphing, etc But by using Wolfram Alpha, it's going to be calling on this computational database to get its answers So I'm gonna have a plot this particular Intersection and you'll see it can write in I'm not sure if it's latex or math ML And it can also call on Wolfram now and it can plot different coordinates solve computational questions, etc So now it's going to create a plot for me and I can link to that on Wolfram So again, these GPTs allow it to have much more complex Functionality the link to other tools just to demo another one. I wanted to create one I made this one just for fun thinking about what a GPT would mean if it was just for an individual assignment so this is an uber urban design tutor and I'll say tell me about a complete street and I prompted this chat bot to Think about Vancouver approaches to urban design Especially ones talked about by Larry Beasley who was the city planner for Vancouver and We'll see if that's going to work for us today So now it's going to tell me about a complete street and it's gonna tell me what it looks like It's gonna ask me a question because I actually had that within the prompt. So again GPTs can be customized based on Data they can also be customized based on prompts the next thing I want to move on to is to pull and I want to move on to this to start setting us up for our next activity Where you are going to be able to create a chat bot so the poll platform is a little bit more of an open platform that allows you to create Chat bots that anyone can access as Long as they've logged in to post so it doesn't it's not a pay to put pay to play They can just log in to po and use these and I'll just demo a couple that I created And then I'll kind of show you the back end of these so the first one that I designed was called a course design bot and The prompt that I used for this called on the research of D Fink to create Backward design tables as well as called on learning objectives I've also asked it to start by asking me what the course topic is. So I'm gonna say introduction Biology and we'll see if it goes through here. Let's try that one more time. I'm having some bad luck Let's try my learning support plot. So I'm a learning strategy tutor Ask me or tell me about an issue or challenge you're having With your studying or learning and I will advise you how to improve. I'm gonna say concentration and I've asked it now to ask me questions one at a time. What are your challenges? I'm gonna say staying focused and After I answer this it should give me some research based strategies around staying focused So again, I'll show you in a second what this looks like in the back end But the goal of this bot was to provide some specific information About learning support and let me just show you the prompt for this bot and This is gonna show you how to build them in a second So you'll see the handle for the bot is a learning support bot the base bot with Po What's really interesting about Po is you can choose different base bots So I've chose chat GPT for this one, but I could have chose clode. I could have chose Gemini Gemini for this one Lama Stable diffusion image generators, etc. So I can choose one base bot then I can generate a prompt for this bot and When you're thinking about creating these bots It's a good chance to work on your prompting skills So for this bot for my prompt, I said you are a learning support expert at the University of British Columbia By using that term the persona you are a support expert I Helped it call on more effective data So by using a specific persona, it generally calls on better data With more than 30 years of experience whenever you are prompted Or ask a question ask questions about the prompt to better understand the needs and goals of the prompt So I've kind of created some interaction within the tool Ask the questions one by one And if you've prompted chat GPT very often of you've created these prompts for bots What they like to do is give you all the questions at the same time And it completely wrecks the one-on-one interaction. So I'm asking you to ask it one at a time When you have a good understanding of the issue Provide a five-point learning strategy The strategy should be evidence-based and incorporate recent understandings of learning science As well as student development literature and student habits Include specifics examples and concrete techniques will improve learning Studying and metacognition And then at the very end I put again start with the first question Because again, it has this these these models tend to give you all the questions at once Scrolling down a little bit You'll see that I could add a knowledge source for the bot that I shared with you. I uploaded A document that had all of the information on the worksheet. I could also link to a specific site there I put a greeting message in there So I'm a learning strategy tutor Ask me or tell me about an issue or challenge you're having with your learning or studying And I will advise you on how to improve. So that's the first message the users are going to see The bot profile here and then I made it public accessible from there. I create the bot And then the bot is available like this and you can share it here as a link So in a moment, I'm going to put you into groups and we're going to do groups of four And what I would like you to do in these groups is I think the best way to learn about how to use these tools And to think about prompting is to play with them So if one person in the group could log in to poll if you didn't do that last night Then I'd like you to work together that person can screen share You're going to click on create bot Think about a bot that you might use in your teaching context or just something for fun I made my kids a a roblox a dot me bot if you played roblox It's a particular game that assess the value of different animals. So you can have fun with this as well Give your bot a title Scroll down decide your base bot Keep in mind that the base bot should be a free version So you're not using your paid version Spend a little bit of time think about those best practices for a prompt prompt create a prompt Let's not do knowledge base this time. Let's focus on prompting the tool add a greeting message Add a bio for your bot and then create it after you've created it I'll get you to grab that link And after you do the activity Let's have you share out a little bit what you've done Thanks, Lucas. Yes, so Before everybody everybody leaves we just have a few questions on the padlet I'm just going to put the link in the chat so then you can all contribute So basically we have three questions for that particular activity Now that you have a better understanding of what the chatbots are You've been able to explore You know what works well what doesn't work really and how you could potentially use them in your teaching So the first question is why are you using jnai agents or custom chatbots? How could you use them? In teaching and learning and what challenges do you already foresee? So Andrew you mentioned you mentioned already a potential issue with the fact that it doesn't save the data Having that sort of a measure And I can see already people contributing but you click the plus button to under each column and then you can just provide an answer And if anybody wants to share, I know we have two minutes left But more than happy to have anybody and mute themselves and just talk about what they just shared But for those already contributing, thank you so much You know as everyone starts completes filling that padlet Um, we will share back The slides the recording for the session as well as the worksheet with you I've also saved the chat So I'll share back some of those chatbots and I want to thank everyone for joining the session I hope you had some fun playing with poh and We managed to get through our technical difficulties at the beginning. So thanks so much for joining