 Good morning, everyone. Welcome to our navigating the A.I. Generative Artificial Intelligencies landscape. I focus on student learning. We really want to focus on student learning. Maybe because this is also exam season, and we will also be looking at how we can design our prompt. And today's session is mainly facilitated by Manuel and Lucas. To begin, I would like to acknowledge that we are situated, Lucas, I am facilitating the workshop from campus on the traditional ancestral and visited territory of the Miss Graham speaking people. And we also recognize some of you may be joining from UBC Okanagan that's located on the territory of the Silicon people. It's important for us to acknowledge where we are and our connections to the land. Especially where we are exploring another new innovations in humanities. The Generative A.I. has been with us for many years that we may not notice or we may notice that it's been around. But last year has been, I would call it an explosion and making it much more accessible to many of us and that we can actually try using it not just as a consumer on one end. As we continue to use it in our everyday life or as we may attempt to bring it into our classroom, I think it's important for us to remember that we should be using this new tool to connect with our students to maybe using the tools as a means so that we can start having more conversation with our students so that the students can have more discussion. And as a result of our discussion and our learning and our teaching, we are working towards making our world, our land, our water, our plants and animals a better place and better for future generations. So we are, this is something new, it's in our society now, it's in our everyday life now. But we need to learn, understand and use it with carrying responsibility when we use it. So that's, I would like to acknowledge that. And somehow skipping, we have a lot of participants here today so please stay muted throughout the session and use the chat to ask questions, share your thoughts and ask questions. We also developed my worksheets and some resources that is in this Google document. I note that Manuel has already shared a link, so use the link to open the document and follow along and you may want to keep the link for future reference. The session is also being recorded and so feel free to turn your camera off and the caption is on so again also you may hide your caption, you may expand it so use it however to make your discussion more accessible to your need. Feel free to follow along, Lucas will be making a lot of demonstration and this is my first time watching Lucas using bank co-pilot, bank chat, also known as co-pilot in our demo session. So if you would like also like to follow along, to use bank chat, you need to use the browser Microsoft Edge. So if you're used to using Firefox or Chrome, you may need to make a switch now, you may need to use Microsoft Edge. If you're like me you're already locked into the UBC Microsoft Enterprise Systems then my UBC lock-in will be visible and you on the very top right corner of your screen you will see this little co-pilot icon click on it then you will get the bank chat. If you'd like for your academic work we recommend using the more precise mode but if you don't know what I'm talking about and you've heard about chat GPT 3.5 or you have the paid version, chat GPT 4, you can still use it or you can use other AI tools that you've been exploring. So it's just we would like to let you know that we may use some of the demo on bank chat. Is it cable, let me see, okay I will try to, I do hear myself rubbing some cable, let me be super stiff. So my agenda, I can still hear you, for the agenda, so we will do some quick refresh, like I said we need to think about our connections to our course and our land. We will go quickly go through some what is gen AI, what do we know, what do we know in the last year and we will have another refresh and activity for you. There will be, we will think about again using gen AI for student learning and then using and also for assessment and also thinking about how to transform some of the classroom assignments using gen AI. So our first activity, we would like you to go through some, I'm not, I'm not touching any button now. So our first activity, we have a few questions for you and we would like to hear your response or see your response from a patlet. Yes, one more slide, I'm not touching anything. Yes, so we would like to hear what do you teach, what do you design, what courses do you design and what courses do you support. What's the goal? Before we play with gen AI, let's really anchor ourselves. What's the goal of our courses? What's the goal of our teaching and perhaps if you work with your industry and what is the significance of gen AI in your discipline in the industry that where your students will be going into or supporting and finally we also would like to know what you hope to get out from this session. So my well has already put in a link in the chat. So that's that will lead you to the patlet and we are going to wait for about I would say 20 ish, I would like to see a full screen of different responses before we move on. So take your time and enter your response or you are someone who would like to think then I hope that you are reflecting on your courses, your teaching goals, what's happening in your industry and what do you like to get out from this workshop. May I just notice someone put on the padlet that they're hoping to get out of this session understanding about ethical issues related to using gen AI and just to mention that this is Lucas speaking. The ethics isn't something we really focused on in this session but just a you know a quick mention it's worth always foregrounding and thinking about the ethics of these tools right now there's a lot of questions around private data being put into these tools about equity and bias of these tools among many other issues including environmental ethics of the impacts around these tools. Maybe one ethical way on a goal of this session is to get all of us on board and help us all learn so that we're better able to have informed conversations about what these tools do and what the ethical considerations are. And I'll also make this as Lucas again one more response on the padlet is about someone's goal is to know what the university policies and practices are with generative AI and maybe Manuel if you get a chance you can share the gen AI website from CTLT where we talk a little bit about overall policies as well as the academic integrity site but currently there's a committee at the university a steering committee looking at generative AI but there isn't a formal policy this is being done on a faculty by faculty basis right now as well as some guidance from the academic integrity office. The rest of your goals really align with the workshop I just wanted to jump in on those two goals while we had a chance. So thank you can you hear me Lucas because I just changed my my microphone headset. So thank you very much for all the contribution I haven't had a chance because I was busy changing my audio system and but Lucas please keep going because the next session is over to you. Wonderful thanks so I'm just gonna kind of get us all onto the same page a little bit and that's a really quick overview about five minutes about what do we mean by generative AI and maybe just to before I jump into that can I get you to put in the chat either I'm gonna get you to hold off for a second and then I'm gonna get you to put a one in the chat if you're brand new to generative AI two if you've been playing with a little bit and three if you've used it quite a bit so just write it in the chat okay so I'm gonna count down one two three great and so we have a lot of twos here which is great and I see we have some threes here not seen too many ones these days so we'll keep this introduction fairly short and it's great to see so many people kind of experimenting plain and learning about these tools so what is generative AI you know in a nutshell it's studying vast amounts of data and this data is scraped from the internet it's using inference to make predictions based on this data and based on those predictions complex predictions about words and grammar it's able to generate new data similar like writing a story images etc and I think when we were first talking about AI there was discussions of these tools being like a stochastic parrot and perhaps just a fancy autocomplete I think most of that's gone away now what we're seeing is some pretty incredible capabilities of creating natural language and doing things like being able to achieve in the 90th percentile of the the standard bar exam being able to write critical reflections that achieve higher than students do and are indistinguishable from student work and I mean if any of you have played with it which it looks like most of you have I think this is far beyond the idea of a stochastic parrot at this point so I wanted to quickly mention about what are these tools trained on and I think this brings up some interesting ethical questions and it's also useful to think about so this is some of the training data that chat gpt 3.5 is trained on unfortunately for chat gpt 4 they've they're not clear they're not transparent about their training data anymore so it's difficult to know what it's being trained on but I put a link in the worksheet to common crawl a lot of chat gpt 3.5 is built off common crawl which is a historical web archive similar to google's index and it's 13 years with petabytes of web data billions of pages over 40 languages and trillions of interconnected hyperlinks secondly a more contentious one is it's trained on book three right now there's a lot of folks on the internet in the copyright era trying to figure out just what's in the book three database but we know that there's a lot of books that are copyright a lot of books that are open search your favorite novel written before 2021 and generally you're going to be able to have it summarize it have it interact with the book as well as the entire wikipedia knowledge base in the english language so that's just a couple of the data points but huge reams of internet data that it's been trained on and the third point I wanted to make and I think again seeing all the threes in the chat we're fairly familiar with this idea is that it's really easy to underestimate these tools the garbage in garbage out metaphor works quite well if we use very generic prompts we get very generic responses and it pulls on very generic data so as faculty as staff and as students it's worthwhile understanding prompts as a way to understand these models better what their capabilities are also for students thinking about this going back to the comment about ethics is if some students are able to use complex prompting to get these models to sing other students don't know much about prompting how are we going to have equitable classrooms so today we're going to be sharing or I'm going to be sharing a lot of prompts with you and inside these prompts I won't go over them but there's prompt patterns so ways of using these prompts often in the field of prompt engineering and you're going to see these patterns used today mostly in examples and models and what I've done is on the worksheet is anytime we've created a prompt I put the prompt on the worksheet and I encourage you to try out the prompt as we go through edit it for your own context a couple of the patterns you're going to see inside these prompts one is the persona pattern act as persona x perform task y so by acting as different people different personas what we're finding is that the data and the generation produced is more accurate this has been researched and I think anecdotally we can also see it secondly we're going to see single shot prompts this is where we train the model with a little bit of data so for example as I'll demonstrate later putting a sample learning objective into the model and then asking it to create learning objectives using this structure we're going to look at refinement prompts for example evaluate this paragraph based on x or act as a cynical faculty member evaluate this workshop based on your understanding of teaching and learning and your experience next is output format we'll talk about different types of formats that we can output in we can output in csvs tables xml etc and I see a question from the chat is there a useful prompting learning website that's recommended on the worksheet we've linked to the ctlt resource for different prompting I also recommend a course if you go to Coursera course engineer prompt engineering there's a great course by jewels white which will walk you through different prompt patterns and ways of prompting I took it a few months ago excellent course it really upped my understanding of prompting and you know just broadly I think these tools are just emerging or these technologies are just emerging and already we're seeing the generative AI can be used in new ways of personalizing learning so thinking about the concept of having scalable tutoring what would it mean if each student could have a tutor for different subjects secondly as we're seeing emerging skill development what skills are needed these days in the different disciplines and areas that we're teaching our students about how are we going to help them learn these skills when things are changing so quickly tutoring I've mentioned already as we're going to demonstrate today these tools these technologies you'll see I'm switching a little bit back and forth between tools and technologies I prefer the term technologies I think these are far beyond tools I've taught so many things around tools I often will say the word tools instead these are very powerful for learning material generation so the ability to generate case studies rubrics questions for quizzes and then ways that we can evaluate things so evaluating outputs evaluating websites I used chat gpt yesterday to take a series of tweets and do semantic analysis on the different tweets in a tweet thread someone just asked for a hyperlink for our slides judy if you don't mind sharing that that would be great and I'm going to turn things over to manual now thank you because so this is myel I'm also part of CTLT and it's a pleasure to see all of you with us today so for this activity we basically are referring to an article by mollik and mollik who have identified seven different ways of using jnai and we thought that was interesting to kind of ask you how you kind of anticipate using jnai in the work that you're doing and in that case you know if you see AI as a tutor basically for increasing knowledge or as a coach for increasing meta cognition so the way we learn basically as a mentor you know to provide balance of ongoing feedback as a colleague basically like with us today to increase you know collaborative intelligence as a tool as a way to extend the performance or even a simulator to help with practice you know depending on the kind of domain that you're teaching so for this we're going to be using the annotation tool on on on zoom so basically if you look on on top of your screen you should see something that says view options so it should indicate that let's say my manually sharing screen you should see something that says view options and if you click on that little button in the drop down menu look for something that says annotation it's always a little bit tricky tricky to find so if you locate the annotation you click on this then you'll have a little ribbon that opens up and then basically to vote and I can see some of you already doing it you can use the stamp it's easier and in the stamp you have different options you have the check mark for instance and wait until I give you the instructions so basically there's no right or wrong answer you can vote for more than one but that's a chance for us to kind of see you know where you are as a group so you can even put the stamp directly on that little check box so even on the right hand side so take a minute or two to kind of give us a sense of where you stand with the use of AI you can use hots you know I don't know if there's a Christmas tree but that would be actually quite nice in this time over here thank you John for voting Yuri as well thank you and feel free to use the right hand side and also you know maybe you know there's an option that you don't see there feel free to use the text and say oh I use jnai as and then you you know indicate that for us as well that's still useful thank you Farid so I kind of see a fair distribution is that like some sort of a gift package or something a fair distribution but mostly I can see that a lot of you consider jnai or use jnai as a tool so for extending performance okay great and maybe can I ask one of you who've been saying you know you use this as a tool maybe use your microphone and tell me more about the actual use of jnai that you make to extend performance anybody or see me later like someone who says you know I use you know jnai as a way to help with practice maybe one of you can just tell me more about you know what you're doing with this no pressure thank you Elizabeth okay yeah I also use it for programming and also sometimes help me to actually come up with you know assignment questions that design of kind of presentation of these things okay thank you thank you and are you generally happy with the the results or do you have to kind of play a little bit back and forth until you get something that you think is acceptable yeah it's pretty good you probably have to go one or two rounds before you get the one you want yeah right yeah and you probably need to change your problem sometimes need to be a more specific about direction you want to go then yeah usually it turns out pretty good okay thanks and that's actually something that you know through the different demos that Lucas is going to do for us today is that you know it's really this sort of iterative process of practice and practice we don't assume you get everything right at the first at the first after the first prompt but it's really working around playing with this and until you get something that we evaluate is you know a fusion for you thank you have sent it for so as a mentor okay Lisa okay yeah Lisa what is stimuli generation so I do a lot of experiments yeah so I do a lot of experiments and sometimes to to generate additional text or actually graphics I'm playing a lot around war with firefly and and trying to do things in photoshop but I used to hire graphic design people to do wow okay and what jnai do you use for that so it's actually integrated that it's currently in beta format but it's integrated into photoshop the one that's actually available through ubc it creates a wonky result sometimes too so it's certainly not perfect but i'm playing around with it now all right i didn't know that okay thanks for letting me know studying thank you okay pbl 10 pay for project so I can see we've got a lot of great answers and thank you for using the chat actually I'm very fun with this gives us a good sense of the actor use that you make of jnai and we have a fair distribution of of usage across you know as a tool simulator even mentor a coach and we we had I think two weeks ago the panel with students and a lot of students in that panel a few of the panelists were actually indicating that they were using jnai you know chat gpt in that case as a tutor so it was interesting to see that not necessary as a tool to basically expect the work to be done automatically by some some AI but mostly like I have a way to spend more time on tasks and really have a discussion with with the AI so I thought that was an interesting perspective from students yes perfect yep that that was the right slide so in that case now the second step of of this session is really designing for learning and you know the idea is really like to approach and you know the design of your course as a process and the intent of this process is really to foster student learning in the context of jnai specifically and using prompting techniques so at this stage courses design is kind of generally a reflection and iterative process very much like you know the kind of use that you've been making with jnai you know not assuming that everything would just be perfect after the first prompt is kind of this sort of back and forth and basically course design is the same type of of work it's kind of something that evolve over time you modify things you apply you change you reflect you actually get student feedback and you and you keep modifying things so it's a lot of back and forth and that's basically what we try to communicate in this session today and in the next slide we've basically given you a chance to kind of approach that course design from different prompting techniques so in that case we're going to be using you know jnai to develop learning objectives you know and Lucas is going to give you a different types of prompts to actually have jnai provide a bunch of objectives and reflect on those and and modify the second step is actually interesting and I've got a graphic about this is the backward design so in learning design I guess we have a variety of frameworks that are used to kind of help you out with the course design process backward design I would say is one of the most popular one developed by Defink and Lucas is going to show you how we could have jnai kind of reproduce the same type of process using the jnai which is quite fascinating I have to say then the third one will be working on course mind maps like mapping up the big items of a course in that case like freedom mind map for first your course and so that's that's going to be one example or even asking the big questions you know create a big question for one and have the jnai come up with something for you so we're going to have different ways of of helping you with the design of your course from using different techniques I guess so as I mentioned just a few minutes ago we have a variety of frameworks that exist that can help you with the design of your course so it's very much a process some of you may be quite familiar we have some people with expertise in education some of you may be less familiar so we just thought we would give you an idea of the Defink model that is quite simple I would say but it's pretty interesting in a way that it works as as a cycle so you basically start with the learning outcomes in that case we oftentimes refer to Bloom's taxonomy with a list of verbs with different levels and then you try to say okay how am I going to assess my learning objectives and then you can you know use different techniques or methods assignments the quiz okay so you have a variety of ways of doing this and then you continue and say okay once you know I have a sense of my outcome how you know how am I going to evaluate it the next step is really how do you put it into practice using activities or how do you actually teach something or develop it through instructions so what you know basically happen in in a in a lecture or even in a in a synchronous session depending on the type of format that you have for your course so that's you know this idea of the backward design that's that's how we call this wonderful thanks Manuel so I'm going to jump in now and just to kind of give you orient you a little bit to where we're going now we're going to be doing three sections now the first section we've started we're going to look at course design space and using prompting and gen AI to enhance and augment our course design then we're going to move into the activity and assessment space and we'll look at how we can use gen AI to augment our assessments you know leverage it to create more assessments better assessments refiner assessments and then we're going to move into the classroom space and talk about how we can integrate gen AI within activities all of these sections are structured in the same way there are a series of demos that I'm going to do and I really encourage you to follow along try things out as I go it looks like all of you have some experience I'm going to alternate between Bing co-pilot and chat GPT for and I'm doing this quite intentionally I think chat GPT for right now outside of Gemini which is coming from Google is the most powerful model that we have to play with unfortunately it is pay to play and I don't think you can subscribe to it right now they've frozen subscriptions but I want to demo some of the things that it can do I'm also going to demo with Bing because this is something that there has been a privacy impact assessment at UBC and what that means is we can recommend Bing for our students with that said what I've been finding with Bing is it can be far more challenging to use than chat GPT and it tends to try to be your kind of little chat buddy is what I find so it will try to I'll ask it the other day I asked Bing using computer vision I said what I'm you know what are you looking at and it said a tennis racket it was actually a Bappenton racket and then it gave me the rules for tennis rather than just describing the images so it kind of goes off the rails so apologize if some of my demos go off the rails so one way that we can use these tools and maybe I'll get a thumbs up now from the group if you've used them in this way is to create learning objectives so could I get a thumbs up if you've used these tools to create learning objectives all right so I see a couple thumbs in the chat or sorry on the interface and that's great and I'm going to use this prompt here right now and create learning objectives with it and a couple things that I've done with this prompt so I'm going into Bing now let's make sure that I'm in there for the screen sharing and a couple things about this one I'm in precise mode what I'm finding and there's some debate online about this is I'm finding the balance mode is terrible it seems to be using I think it has a engine called Prometheus as middleware that Microsoft's developed and it's acting as my little chat buddy whenever I use balance so I don't use balance creative I've found hit and miss for me precise works a little bit better in terms of the prompt you'll see that I'm very specific with it so create three lesson level learning objectives that are measurable and aligned with Bloom's taxonomy one thing that I find fascinating about gen AI especially when we think about copyright is that it does have access to books like Bloom's taxonomy so you can get it to refer to different approaches for learning objectives for exam writing just by naming the author use this learning objective and as an example and this is where I start using a little bit of single shot um what I realized is I didn't put the subject in here so I'm just going to add that aligned with Bloom's taxonomy so let me just say for first year physics so first year physics learning objectives that are measurable and let's see what we get and again you never know what the output will be in these tools and I'm especially uncomfortable with being that it's just going to kind of go off the rails so now it's going to give me specific learning objectives in this area and you can take a look at the measure the learning objectives and see that these are measurable they're specific and they're based roughly on that learning objective that I already added so one way that we can use these tools is to generate lots of learning objectives and rather than going through a process we're always creating the learning objectives start thinking about sorting through these learning objectives adapting them for our course design I'm going to move through the slides without sharing full slides just to make it a little bit easier now for the demos the next example I wanted to give you is this backward design table so what I found is a generative AI is quite good at creating backward design tables as part of the course design process again I think we need to be a little bit careful in that we need to go through this we need to check it to see if it's accurate these tools still aren't 100% accurate it may hallucinate it may give us information that's way out of context but it can seem to understand de-finks approach as well as create these tables so let's try this with Bing first and I'm just going to grab this objective and I'll mention that to create these prompts one thing that I've been doing and I'm not sure if you have in the room is I've been putting a prompt into chat GPT and then I've been saying improve my prompt to get a better output and I'm getting much better prompts by doing this it's actually made my prompts a lot longer so using the following learning objectives create a backward design table based on the work of de-fink so I'm giving it a lot of structure here I've given it the course structure I've given it a couple learning objectives and let's see what sort of output we can get from that again let's try Bing out you never know it could go way off the rails but what I do like about Bing particularly is it gives me these things in a nice table format so let's take a look it's given me formative and summative assessment which is interesting um rather than giving me just assessment which is in Fink's approach to backward design so it's gone through that it's given me the formative assessment and it's given me potential readings that I can use so your turn I've given you a couple examples of ways that we can use these tools in learning design what I would like you to do now is spend let's go five minutes on the activity sheet try the learning objective prompt out try the backward design prompt out or try a prompt that might be related to course design that we haven't talked about at the end of five minutes I'm going to ask you to share the output in the chat and also your reflections how was the quality of the output what was missing what would you do differently in terms of prompting so let's start our five minutes now and I'm quite curious what sort of prompts you're going to be able to generate and what sort of outputs you're going to be able to get so go ahead we'll start the timer now and Manuel mentioned in the chat that he often refers to Angelo and cross 50 classroom assessment techniques to further explore assessment techniques that Gen AI may have generated and it I have found it excellent at classroom assessment techniques another educational framework I use is liberating structures which is open online so it makes sense that it was great but it's able to pull on different liberating structures and generate output based on those so thanks for sharing for that activity and let's keep going and we're going to do something similar but instead of talking about course design now we're going to talk about designing assessments and assignments using Gen AI and again we'll take that same flow what we're going to think about is a couple ways that we can use Gen AI to generate learning materials so first of all is assignments and I have some example prompts around this secondly is creating questions and I think creating exam questions the examples I've heard from faculty I've heard two interesting examples using Gen AI to create exam questions when they need to create an exam for the purposes of accommodation I helped a faculty member do this recently and they're having to you know create a whole new exam and secondly I know Verada Kohler in computer science who presented as part of the CTLT Gen AI studio last month talked about using it to create randomized questions so she could have multiple randomized questions that she used we're also going to think about creating rubrics with it as well as creating scenarios with these tools so the first example I wanted to give you is creating assignments and again I think that it requires as Louisa mentioned kind of some expertise knowledge in understanding these assignments and in this case what I've done is I've used the work of Eric Mazur who if you've read Eric Mazur in the context of physics he takes an approach called peer instruction which is a highly structured approaches for assignments so in this case the prompt is create a comprehensive peer instruction activity for first-year university physics students around understanding displacement within a frame of reference and I've asked it to ground the activity in Eric Mazur's work so in this case I'm going to use Bing again for the chat just give me a moment here and let's see what we can get from this particular assignment so create a comprehensive peer instruction activity and so it's going to come up with the learning objectives in this case with the materials needed and what I like about this is it's starting to build on the notion of peer instruction so a concept introduction individuals think about something they get together in a group and then they answer a question again and it's going through and it's creating the conceptual questions for this activity I could refine this further if I wanted to and you know in order to get more specific answers I'm finding by going through a process of refinement I'm getting better results so act as an educational assessment expert and evaluate this activity and I'm going to ask it what criteria used luckily spelling doesn't count let's see if it's able to do this Bing allows you with enterprise Bing you can go back and forth 30 times so it's not giving me too many critiques I would expect the GPT-4 would be able to give me a little bit more critiques on that and then what I do typically is I ask it to rewrite it based on the particular critiques it used so assignments was one example the other example I wanted to mention was questions and I'm not going to demonstrate this one but again by using very specific prompts so in this case you are an expert teacher in cognitive psychology I'm using a role-based prompt here you're skilled in creating intriguing and thought-provoking questions for upper-level undergraduate students as well as in assessing their work effectively so you can really build out these roles and give the model some characteristics you've prepared two questions with solutions for an upcoming quiz however due to scheduling conflicts you need to develop a separate quiz could you please help develop comparative questions and then I've given it a model of a couple questions again using that single shot and what I found is get you get fairly good quality multiple choice questions by doing this the third example I'm going to demonstrate again using Bing is creating rubrics and these tools are quite good at crow these technologies are quite good at creating rubrics when you use Bing for creating rubrics what's also nice is the rubrics exportable into excel you can also ask it to generate a rubric similar to the canvas learning management system or similar to grade scope if you want a particular format of a rubric to develop so again in this prompt which I'm going to demonstrate now I asked it to act as a communicating science instructor with an expertise in science communication create a rubric I told it what it's going to assess and I explained specifically what the rubric needs to have including gradation levels so let's go in here I'm going to create a new topic so that it doesn't have some memory of the old topic and let's give it a try I think I prefer Bing to GPT4 for rubric creation just because of the table layout that it gives it so it's going to give me a table layout now it's going to add each of the different criterias as well as you know what unsatisfactory is what satisfactory is you'll see that it's talking about scientific language it's talking about audience engagement two aspects of science communication so again this typically the results aren't perfect but there's something you can work on and what I like about Bing now is that I can download it so I can export this rubric or if I scroll up to the top and I click on the edit in excel I can edit this rubric directly in excel so here's the same rubric or a slightly different model of the rubric the last one I wanted to mention to you was creating scenarios and case studies I recently did a workshop where there was a lawyer taking the workshop and she started running some hypotheticals hypotheticals are examples of case laws that are hypothetical and after she had finished using GPT she said I don't think I'll ever write a hypothetical again in a lot of our teaching contexts we're using cases we're using scenarios and these tools can be quite good at creating scenarios for us that we can reuse within our teaching so I am going to demonstrate this one in GPT4 and in this case I asked the tool to write a complex case based on the idea of teacher digital identity and I chose this one because it's an area that I work in I do I do a course called digital the digital tattoo where we instruct teacher candidates perspective pharmacists around their digital identities so I have an idea of what these case studies should look like so it's using British Columbia now it's talking about navigating digital identity you'll see that it understands I'm speaking about a teacher candidate it's talking about her social media presence it's thinking about an incident that happened and it talks about the complexity of the case so in my case with these workshops this would require some elaborating on it however it would be a good start to a reasonable case study that I could use when teaching these classes so again over to you I'm just gonna do a full screen share and in the spirit of kind of experimenting and playing with these tools I'm gonna give us a few minutes now to create a rubric for your course using the provided template or an imaginary course if you wanted create three questions that you could use with your students to practice a concept in your course and if you want to try creating a case study or a scenario so just like before I'll give you a few minutes to run this and then I'll get folks to share what they got back and I see a question in the chat about from Louise asking about students using our material to generate their own test questions I mean that's got a lot of layers to the question Louisa I guess one is I love the idea that they can do that the other challenge here is that by taking our material and putting it into these systems they're putting it out into the open and it brings up questions around intellectual property and you know what's being done with this material what happens if the data links leaks etc but as it's also a great learning activity so I'll give folks a few minutes again go in there give these prompts a try and Louise I wanted to share a quick way that I've been using it with my son he's 13 years old he's in grade eight right now or grade seven sorry in his in his class he forgot to study for an exam so what we did is we took a picture of his notes which were pretty messy and took his notes took the picture put the picture into chat gpt4 and asked it to create questions based on the notes and asked it to test him and help him understand the concepts one at a time so he spent about half an hour going back and forth with the tool having it ask him questions if he did it wrong it would correct him and learning the material I kind of supervised it because again you're never sure about the accuracy but it does really open up different learning approaches especially from a grade seven students who's probably using very inefficient study methods to read off their notes and try to memorize all right so we've talked about so far thinking about gen ai in course design thinking about gen ai in assignment design what I'd like to talk about now is how we might think about using it within our classes and maybe I'll get a thumbs up again from the group could you give me a thumbs up if you are using this in your teaching now as a court you know getting students to use this in a course assignment and I see a couple thumbs up there so sky looks like she's using it great so a couple folks are using an in-course assignment and when we get through this section I'd love to hear from folks about how they're using these tools in course assignment so first of all I think when we're starting to use these tools it's worth thinking about equity around using these tools in the classroom and now that Bing has a PIA at UBC or a privacy impact assessment this is less of a concern but I still think those students are going to be reluctant to using these tools what's nice about Bing is that students don't need to log into it and so this is going to assuage a lot of students but how are we going to ensure that if we do an assignment in the classroom there's equity built into the assignment meaning do we need to prepare students for prompting if one student can generate something better what about students using a paid tool what about students who are uncomfortable using the tool period is there something equitable we can get for them number two is how do we support them using these technologies including things like understanding the biases within these tools understanding the privacy that putting private data in these tools could result in that data leaking out I don't know if you saw it recently but a researcher was able to get chat gpt to link personally identifiable data by asking it to write the word poem forever and by doing that it started giving out personal information that had been put into it so how do we help students understand that these tools do have privacy limitations how do we help students understand that sometimes these tools are going to lie to them and tell them things that are incorrect number three because these tools are emerging so quickly how can we collaborate with our students and partner with our students to integrate these tools within our teaching so a couple ways that we've been seen faculty using these tools as well as I've linked a document in the worksheet from the University of Central Florida that has about 50 different ways you could use these tools and I've adapted that a little bit for this section so one way we can use it in the classroom is to get students to analyze the output so this could be creating their own work comparing the output for inaccuracies looking at the output for biases looking at the output for embedded perspectives that may be within the output this is an example of doing this would be getting students to create get chat gpt to provide a detail output and then try to look at the output and find out the extent that it's biased Patrick Pennyfather does this in theater and film at UBC with image generation so getting it to generate an image say of a computer science faculty member often a tool like mid journey will generate older male older white male when it does this so the idea of using these tools for evaluation and analysis here's an example from Yale where they've done this so each student posed a question relevant to their problem statement and then annotated it with gpt and they focused on ways that the ai write up may be inaccurate misleading incomplete or unethical it also in this assignment he helped them think about how chat gpt helped them refine their research question I think one challenge of this type assignment that we're going to see more and more and the researcher Ethan mollick who if he get a chance to follow and linked in write some great stuff about generative ai mentioned that in his classes as chat gpt for comes out and as these tools gets more powerful he's finding it harder and harder to get output that has as many inaccuracies in it therefore it's harder to evaluate the second way that I'm seen we're seeing faculty using these tools is have them use it as part of a generative process so acting as chat gpt as a partner or being ai as a partner someone to do the assignment with them to generate new things from the assignment and to kind of take it further and augment their work a little bit so this is an example from that university of florida paper so students brainstorm a new business idea for products or services related to their area of interest and ask students to select a topic combine it in chat gpt with a list of their own skills and then use it as a brainstorming companion to kind of take things farther again there's a little bit of an evaluation piece in here but they are using it as a tool to kind of augment or generate more or brainstorm with the tool this is an example from ubc from food science where students were asked to use chat gpt to generate formulations for specific food products to make them two-year sweeter etc so again going a little bit further and augmenting the output of their assignment the third example of types of assignment we can develop are self-study assignments we know that students are already using these tools as part of their studies so finding ways of helping students use these tools giving them prompts to use them and helping them do independent study with these tools so assignments that scaffold students independent learning and provide them with more practice assignments for example summarizing long articles generative ai is quite good at summarizing long articles in this case for the prompt students summarize the following article write 10 bullet points as if the reader is in middle school and 10 bullet points as if the person is a leader in industry so the ability to kind of take apart articles synthesize them what's interesting with Bing now is you can actually take a URL for an article put it into Bing and ask it to do a quick summary of the article another way and this is an example that Louisa mentioned earlier was getting these tools to create self-quizzing questions for students so having them use your course notes if you're comfortable with the IP or having them use a particular topic area and create a quiz or a game to test themselves and to learn within these areas