 I have the top of the hour, so let's begin. Greetings everybody, welcome to the Future Trends Forum. I'm delighted to see you all here today. We have one of the best guests anybody could hope for on a topic of a great deal of interest and I'm really looking forward to our questions, answers and overall conversation. We've been looking at artificial intelligence and automation for several years now and since chat EBT was released last November, we've had three sessions on the topic and we have to have more coming up, because this is a subject of a great deal of interest. It's one that seems to be having a huge, huge impact on the world. Now, there are a lot of people we could talk to about this, but the one I'd like to invite is Reuben Puttador. Reuben is an education researcher, speaker and consultant best known for his creation of the SAMRA model, which is a framework to help educators integrate technology and teaching and learning. Throughout his career, Dr. Puttador is focused on the effective use of technology and education. Now, I could say more, but I'm just reading off of what Chat EBT told me to say. I asked it to introduce Reuben and I think it did a pretty good job. So, with that note, already part of me is becoming obsolete. Let me just bring our wonderful guest up on stage. Greetings, sir. Hi, Brian. How are you doing? Good to see you. Well, I'm glad that my background or my foreground matches your background to your degree, some blue to blue. Well, my background is part of an experiment I've been running using AI, which I'll talk perhaps a little bit about later. So what you're seeing here, for those of you who are curious, you say, wait a second, that looks kind of like Gaudi, but it's not like Gaudi I've ever seen. You're right, this is Gaudi you've never seen because it only exists in imaginary land. But I'll talk more about why and how and so on later. Well, what a tease, what a tease. I can't wait to hear about that. But speaking of imaginary lands, you know how we ask people to introduce themselves in the forum by talking about what they're working on for the next year. So what's ahead for you besides impressing my students, all of whom are big Sammer fans? Well, thank you. I appreciate your students' enjoyment of use of the Sammer model. So a couple of things. One of the aspects that I'm going to continue working on is something that has been a focus of my work over the last few years, and that's Black Swans and antifragile thinking, particularly applied to education and educational institutions. So that's going to continue looking at what happens now as things change relative to the pandemic, but also giving your book coming up, challenges such as those associated with climate change, challenges associated with shifts in interest and focus of students. So there's a whole world of topics there that I'm going to be looking at. But the other key focus for me in coming months is going to be, in fact, AI. And particularly some aspects related to both generative AI, so what people look at, but you look at something like a chat GPT or GPT. And something like the image generation tools like stable diffusion, mid-journey, etc. But also other forms of AI that have more to do with how somebody can adapt the tools for their own purpose. So again, I can talk a little bit more about that in the context of medical education, as it says. But concretely for today's topic, there are three angles from which I look at tools like GPT and their relatives. And one of them is to look at them from the perspective of saying, so in academia in higher ed, what do we do with these as creative tools? How do we use them to create new things, new ideas, new options and so on? So that's the first one. The second one is to look at them as tools for research. And again, I can talk a bit more about this, but they can be used as tools for looking and saying, are there patterns that we can use the tools to dig into to bring out that we might not have otherwise seen that are important of interest that allow us again to go in new directions with research and so on. And the third direction to me is also an important one, which is what do these tools do in terms of societal transformations? The future of all sorts of things, not just of the technology, but the future of work and to me just as crucially the future of, well, how do people want to live? How would people like to live? Well, for lack of a better term, a good life be in a world where we see these tools in common use. What can they do that could help some of those good aspects bring into coming to being? What can they do that might block some of those aspects? And how do we think about this actively? Because if there's one thing that I, when people ask me, well, what can we in academia do? Well, we need to be thinking about these things actively, engaging with them, engaging with people in the world about them. Because if not, I fear that the things could go very much in the not so great direction. I hear that. That's quite a 12 months ahead. That's an awful long on deck. And friends, you can see why I had him bring Ribbon on stage to talk about this today. I'm going to ask Ribbon a couple of questions to get the ball rolling. But he's here for you, for your questions and comments. So as we start going back and forth, please hit that Q&A box and type in some Qs so we can prepare some A's. And also, if you want to join us on stage, you don't have to have a beard in order to be on stage, but apparently it helps. One thing that I'm trying to figure out is when I talk to educators, they seem to have one of two basic responses. One is this is evil and must be destroyed. And that comes from a few different places. Sometimes it's a fear of being rendered obsolete. It's a fear of cheating. Sometimes it's a fear that's political and we don't trust these companies and they can be doing terrible things. Maybe part of the generalized fear of Silicon Valley. I'm compressing a lot into one stage, but that's one mentality I see. The second argues that we need to rethink a lot of our assessment. Now, I haven't heard this in terms of visuals. I only heard this in terms of text. But then the argument comes either we should remove writing completely and move to say oral exams or air-gapped machines or we should just rethink the entire apparatus of assessment. If you're going to be signing papers, how do you come up with different topics? Do you have multiple scaffolds in order to build a system that reduces the chance of plagiarism? I think I'm seeing those two schools. The first is that it needs to be constricted and the second is it needs to be worked with. How do you see these playing out right now in practice all around you? That's a great question. I think in terms of the first school, I think there's a certain fear of, you know, well, we've been doing this all along and suddenly this comes along and pretty much makes a lot of the tools we've been using useless, meaningless, etc. But to me, it's interesting how it's focused, right? When people start out with the first thing as well, students are going to cheat. They're, wow, that's the first thing you think about with your students, not what they're going to learn, not what life they want to lead, not what you think about cheating. That's a kind of dark lens to look at things through. And it's not that I want to negate the fact that, yes, in some context, students will cheat or there can be encouragements for different reasons, career pressures, etc. But that to me is a weird way to look at it. To me, it's a question of saying, well, okay, what can we talk about engaging students with so that the motivation for cheating is not there or so that it's irrelevant in terms of what's going on. So if you've been doing something like assigning the standard five paragraph essay on topic X, and that's been your final form of assessment and that's been your final form of summative assessment, and you make it a very high pressure, okay, so you're going to show me everything you got in this essay and so on. Well, yeah, I'll tell you right now, GPT-4 just came out, I've been testing it. GPT-35 was still a little shaky on that. GPT-4 is very good, I have to say, at constructing a standard five paragraph essay given a topic and a little bit of coddling along the way. So in theory, you could say, sure, that's going to be what students are going to do, but you have to be asking the question, why were you asking the students to do this in the first place, right? And if you were doing this as summative assessment as a way of saying, hey, what have you learned? The truth is, this is a really bad tool for that, because all it generally is doing is asking students to rehash something that they got and guess what, yeah, a tool like a large language model like GPT-4 is excellent at rehashing things and will only get better with time. So you're asking your students to perform a task that, frankly, a machine as we're seeing can get better and better at doing. But if the reason you were asking your students to write essays was to say, well, look, there are some things I would like my students to have at the front of their mind, so this is a way to get them to put it at the front of their mind. I wanted to do it as a way of, you know, if I asked them to put things together, can I gauge their understanding of how that's coming together and so on? There's a form of formative assessment for which there will be a feedback group and so on. Then suddenly, two things happen. Number one, the incentive to keep for the worry that people have tends to receive into the background. Why? Because this is not any more a kind of this is the sole thing I will judge you on, but more a question of, listen, I want to see where you're at, how your thoughts are evolving. I will give you a tool for analyzing, examining your own thinking. Then there's not much motivation for, see, you start to take away the motivation. And number two, the other thing you can start thinking is about, well, okay, absent a tool like GPT I could assign this writing, but can I change how I do writing with GPT and similar tools in the middle, so I can add richer things. So I might ask my students, for instance, to still write in the standard five-paragraph essay, by the way, I've never liked the five-paragraph form, but we'll say that for another time. I can say, well, okay, let's engage with a topic and let's have the student engage with it using GPT to bring together some arguments, but to go beyond those arguments. So what I'm asking the students, yes, I'm still asking them to thread together ideas, but I'm giving them, if you will, a partner in construction. And this is something we're just, let me be very clear, we're learning how to do that. As the tool evolves, as the tool becomes more useful, et cetera, we can start doing richer, more interesting things. So just between GPT35 and GPT4, the world has changed in terms of what I can ask my students to engage with. So I can ask GPT to give me a brief summary of the thinking of philosopher X, right? So I can ask it to tell me, well, tell me what, for instance, Gadamer might have said about this, or tell me what the following school of continental thought is compared to this school of Anglo-French. That's easy. You can get that, but then you can have your students start to interplay with that in ways that do not go to what something like GPT can easily do. And do tell me, does the student understand how this thinking occurs? Does the student understand how to put it together, how to synthesize something from it? So that's, I think, where I would address that first school, if you will, the negative one. And as you can see, it's closer to the viewpoint of the second school, right? But I also have a comment to make in the context of the second school, and that is, as I said, we're just learning how to use these tools. And to me, it's important that we not lock ourselves into what I would call sub-optimal ways of using the tool. What do I mean by this? I'm seeing already a rush to saying, oh, we have these tutorial models that we developed using some type of scaffolded platforms, whether it's Khan Academy or another one. And now we can use a tool like GPT-4 to provide directed instruction by engaging with the student in a dialogue about what they understand, what they don't understand. Now that in first instance would be indeed an improvement, okay? In summer terms, an augmentation of what was being done, because it is responsive to the student to a certain degree. But my question is, we want to be careful, because I don't want to get locked in and say, well, okay, so what I did is I enhanced that mode of instruction and that's it. But what about constructing a whole new platform that could not be constructed without this type of tool that goes beyond what you could do with the type of standard presentation, video, plus quiz structure with some type of responsive structure, etc. Which, if I have to be perfectly frank, hasn't evolved that much other than in terms of software technology on top of it since the 50s. In other words, so for the positive aspects, what I would encourage us to try to do is to think deeply about saying, well, when you have a tool with this type of plasticity, with this type of power, with this type of possible uses, can we take a step back and say, well, we could enhance one of our existing tools, and maybe we start there because that's fine, but we don't just say and now we call it the day and we're all done. I would encourage us again, summer terms to push through the modification to redefinition levels, where you're doing something that you couldn't do without the doing. That's an excellent, excellent answer. You were by the late last night, I was a little tired, I asked GPT Ford to create a synthesis of the political philosophies of Enver Hoekstra, the Albanian Marxist and Henry George, an American 19th century economist, and then to do so in the style of the Varks Brothers. It did a pretty good job. It did a pretty good job. It really synthesized all of that together. But we need to have that creation, that sense of co-creation and redesign for something that we couldn't do before. We have more questions piling up, and I was going to ask you another one, but I think I want to open the floor here. We have one from Kate Herzog. Hello, Kate. Kate says, do you see institutions in those schools as negotiating group subscriptions so all teachers and students have equal access to those today? The librarian wonders. Kate, excellent question. Short answer? Yes. Long answer? Yes, yes, yes. By all means. In other words, we need to have equity of access to these tools, and that's very important. Equity means, of course, covering that cost of subscription through libraries, through whichever front end for education we're looking at. Those need to be there. It can't be the case that if you can afford it $20 per month you get the good GPT access, and you can't afford the $20 per month you get to this answer brought to you by, fill in the blank of your favorite ad buyer for the day who will proceed to customize the answer to your terms, right? So we need equity, but I want to emphasize that equity needs to stretch beyond just, okay, so we all have this type of, you know, it's not a burden based on economic terms to gain access to this. You also need to be talking about equity in terms of access to what you can do with the tool, because I want to emphasize something in terms of what you can do right now. Right now everybody has access to at least the chat GPT 3.5, right? And you have access to Bing's own interface, whatever Microsoft ultimately decides, well, let's go to Bingbot for now. But that's it. Then pay $20, and you get access to chat GPT 4, and we'll see how that evolves over time, and you have fewer limits on time, what you can ask the depth, how deep a conversation will have and so on. And then pay per token with a developer's license, and you have access to the engine itself. And now I can start doing things that I cannot do through chat GPT. For instance, some of the research I do in terms of exploring patterns that emerge from text cannot be done easily with chat GPT. The chat GPT interface gets in the way. I need to be able to ask and say, here's a huge volume of text classified this way, but if I do it in the interactive chat GPT way, it's going to take me five months to complete it. If I can just pay per token, I just say, boom, just do this whole thing, give it back to me in a file when you're done, and we're all set. And that's what I mean by equity of access. In other words, we need our students to be able to access on all of these terms, because otherwise chat GPT, don't get me wrong, it's a wonderful frontend, but it is a frontend with certain assumptions, certain limitations, certain conditions imposed upon it, some of which may make sense for an educational audience, some of which may not. And again, if we don't allow that fuller access through our institutions, whatever media we choose to provide access to, then we immediately create a situation where there is no equity of access, and you can do very, very different conditions of how you can use, how you can think, how you can create with the tool. I think that's a crucial point. Well, that was in the direction I thought you would go in, and it's a great one. And Kate Herzog, thank you for the superb question. Friends, you can see that Ruben will be very kind to you, and this is a great place for you to put forth your questions and comments. We have a few more coming up, and this is one coming from Ranjana Dutta. Totally butchered your name. This has to do with, I have got creative data and references. I would worry about them using ChatGPT as a partner to build on right now. How can we verify information without having the teachers do that? Ransana, could you clarify for me a little bit more? When you're talking about verifying information, are you talking about verifying whether a student has used ChatGPT or are you talking about whether a body of information is sourced that the student would then be using ChatGPT? I want to make sure I understand your question correctly. Ranjana, please feel free to type in a follow-up question to clarify that. And of course, if you want to just talk out loud, just hit the raise hand button and it will beam you up on stage. Thank you. And while Ranjana is wrestling with that, we have a question from John Hollenbeck. John, I hope you're staying warm. And John asks, why is it we hardly blinked over tools like Grammarly, which passively edit student essays? You know, John, that's a great question. And I have to be honest with you, I think it's a question I don't know the answer to the why explicitly other than to say that I think the tools merged enough with what we already had that we didn't really pay attention or if I could even pay attention to what the tool could do or would do. In other words, you have a, you say, well, a spell checker. A spell checker is like looking up in a dictionary. And the grammar checker sounds like the same thing, except it's not, of course. It starts to get into the structure of communication, the structure of how students talk or what the structures that students use to communicate and so on. But it was approximately enough to the spell checker to be accepted. But that being said, and Don, you're right. It went largely unexamined and it should have been. And again, one of the things you can do, of course, with GPT, particularly GPT-4 is superb. If you give it the sentence that's can you make this clear? Can you make this better grammatically and so on? It nails it, no question about it. But then we again get into a question of, okay, so rather than talking about grammar and if the students have good grammar, bad grammar or however you choose to phrase it can we talk about, well, why are we concerned about grammar and can we instead talk more, a little bit more deeply about communication, meaning, creation and how the tool can be used for that, because a lot of the old grammar tools, of course, my favorite was they either had an obsession with the passive voice. It was nothing may ever be in the passive voice. It's not that. I agree. Yeah, I agree overuse of the passive voice, but or they would be obsessed with Latinate forms or whoever wrote the tool had some obsession typically and some of these were primitive forms of expert systems in AI. It depends on which one you were looking at how it worked. Right? But again, with a GPT-4 you have something that's much more plastic. It isn't going to force you into just this or that. You can instead query and say, well, for this type of audience, in this type of context, how might I best convey this? And, for instance, Brad, one of the points that I think is crucial to the work you're doing with academia and climate change. A lot of the climate change reports, a lot of the climate change documents are deep, they are rich, they have important information. But let's face it, not everybody has the time to read several thousand pages worth of reports. Even the executive summary can be 40 pages or more of that stuff. And I can use GPT to say, hey, just take this, summarize it. And I'm not just going to say, and I'll just spit it out, right? But I can use it to say contextualize it for a given audience. Help me communicate. And of course, a student can use it for themselves, saying, okay, I don't get this. Can you help me by explaining, reframing, etc.? So if you will, we're now taking the old grammar checking tool, the old grammar checking task, and making it a richer task of communication. I had a very cynical reading myself on that, which was that grammar really didn't threaten journalists. But chat GPT scares them. Shin Lee in the chat had a much more thoughtful response, which is that in her understanding chat GPT is not bound up to a particular academic discipline. So everybody can feel challenged. Thank you, Ribbon, for that great, great answer. I wanted to circle back to Ron Johnna, who had her question before. And let's see if we can bring her up on stage. Hello, Aaron Johnna. Welcome. Hello, hello. Yes, this is very interesting. Thank you for answering the questions. And I'm one of those I like to take light for what it is and try to go from wherever we are. And I was just verifying this for my own sake, putting in some questions and getting references to see what comes through and many, if not, like I asked for 10 references on a topic of first generation college students that I was interested in. And it showed up with a number of references. I think about nine of the 10 given were wrong. They were literally amalgamations with similar sounding names, similar sounding titles, very creatively done, but none of them authentic. I couldn't find them anywhere. And what I would worry about as a teacher aid, this is not a teacher would be doing is verifying your references be students don't I mean, especially current current state of the science or state of the world is they don't want to start in time to do the things that they need to do for the paper. So it's usually being done last minute. So they're not going to verify this. And if you're going to build on things that you don't even know where they came from, it becomes very problematic. So I would at this time, I'm hoping future versions may get us to that point where we can do those kinds of things. So that was my concern at this time when you had said that we can use it as a partner, I would love to, but given what I saw, I was just like, ah, can't go that road. Right. Let me that's a very, okay, thank you and that may say very clear and you're absolutely right. So let's be clear one very important thing. We have to also be very aware of the limitations of the tool as it exists right now. And you mentioned one of the key ones, right? If you take just chat GPT even for and ask it to provide references for something, it will tend to do after a certain point what sometimes is called hallucinate. In other words, it creates a new reference and people say why is it making up a reference? You have to understand a little bit how the whole GPT ensemble, all large language model tools work. What they're doing is they're finding patterns, right? And initially you'd say, well it found the pattern and the pattern included as part of that pattern where the pattern was sourced. The trouble is the larger and the more complex the model becomes, the harder it is to keep those references in there. In fact, at some point you have to give up the exact references. So you can do something that would be the equivalent of this is an approximate but almost an exact an exercise and recall. I know this is generally true and I think I saw it there. The trouble is at some point it starts matching patterns to a there that never existed. It's a little bit as though you said, you know I know I heard that it must have been my friend Freddie. Actually, Freddie never said that but you say it must have been my friend Freddie because in your mind you matched it to this. And one thing I would start to emphasize is please, the large language models don't even remotely think like a human being does. These are large pattern discovery and pattern matching generation engines. So how do you solve this? Well, the way in which you solve it is to say, look, the large language model works like a reasonable human being might be in terms of its capacity to have a general body of knowledge to interpret things and that has been growing from model to model. And then for things like references it has to build upon another body of references just like you would go and look it up in your life. Instead of saying, I think Freddie told me this you'd say, let me go to my bookshelves or my virtual bookshelves now. Google scholar, etc. and make sure I'm remembering correctly where it is. That is where the GPT family is going right now and you see some of that in Bingbot. If you use the Bing engine it will give you references and those references are legit. I have not been able to make the Bingbot lose its place but I have not yet been able to generate a complete link to a website that doesn't exist. It links to a website. However, Bingbot is still very primitive because the quality of the reference websites it links to varies very wildly with respect to the question even when better websites exist. So for instance, I ran a test on something that would be of interest to any student or in fact any member of the community who was interested in something relative to environmental rules and protection of the environment in the community and so on. And I felt I could get Bing to give me a good answer and give me references but the references were too low quality. So who is it? And it's an interesting question why they got back. I did an analysis later for which ones showed up and why. It's fairly good. It's well beyond what most people would want to do. But again, the team is aware of this. I suspect what we're going to see at some point is an option to say well I would like to have it point towards this set of sources or to fine tune my sources. So that for instance in this particular case, even though government websites, EPA websites organizations devoted to protection of a particular species or environment all existed it was going to as I say very very low a third tier sources and so on. So that still needs to be fine tuned and needs work on. So that's a very long answer. I apologize for that to your point which is to say that at this point would I tell students to go out and get sources references from GPT? No. But what I might do is to say instead, okay so here's a topic you have I give you the references or you find the references and then you use GPT to help you analyze or understand these references. So that's where I would do it. I would not at this point even with GPT or chat GPT for give it that task because the hookup to sources is not yet refined enough. I hope it will be and I hope it will be fine tuned. So that would be exactly my concern was that not only were the and I don't I'm not an AI person at all I'm a psychologist and I was looking at what has what it produced were also even the ideas were some of them were amalgamations. I happen to know some of the people who are in the field and I'm like if somebody cited this about my work I would be up in arms because I never said that but it's like similar people might have said something similar and there was that sort of amalgamation that had me rather concerned like it's not just that it's creating these websites and they look very good. I use APA format and they were in APA format and everything with DOIs and it was all like literally fake that's not hallucination and as I say no no that's exactly it's important to realize that that is a limitation of the technology as it is implemented in chat GPT and as I say the crudeness of the approach so far in being GPT but since you brought this up there's one more aspect and I suspect Brian thought I was going to mention this earlier so I'll mention it now which is that all of these are commercial and all of them are associated in one way or another with advertising the sale of advertising the sale of a platform of voice to somebody that wants to sell a product get a particular message out and so on we can work with them as we have all along that's not anything new but I would also like to see in parallel particularly for those of us in academia but are open source slash you know it's a European term libre efforts develop in AI because I think it's going to be crucial for multiple reasons number one the more engines they're out there the better the more richly the more interestingly I think the field will develop number two having an engine I can look at and look at the source code and look at the body of references materially was trained upon and to say oh I see this working not working because of this and I can make sure that I can answer why did this happen and I can do more and say well that's in the general pattern of hallucination I can say well geez let me see because I have the tools to dig specifically into why this particular failure occurred or this particular thing occurred is I think also crucial so I don't view this as a optional thing I do view the development of open source libre AI as a necessary component as we move forward for again full use of this in academia and for all frankly addressing some of the challenges and potential risks of the existing engines so that's I think one more point that emerges from what you've brought up so thank you again for bringing up that point thank you and thank you professor Dutta and please stay warm if you're in Saginaw I hope it's warm and sunny yes thanks friends that's an example of video question and we now have another example of video question just to show you our geographical reach we are now going to bring on stage a wonderful professor from Armenia this is Brent Anders who's been on our stage before Brent is doing tremendous work talking about chat GPT his Twitter feed is admirable and his videos are just essential welcome okay so I have a couple of different comments I'll try to be quick the first thing is the whole idea with the hallucinations yeah that's a big deal the GPT-4 is supposed to be 40% better but the hallucinations are still there so now it's a little bit above 80% where it's going to be right and then there's still around 15 to 20% where there's a problem so given that one of the big things that I'm always pushing for all my instructors every speech when I talk about chat GPT is this idea of AI literacy of knowing that hey it could be wrong so critical thinking is very important that means that we have to analyze any response but that should go without saying I mean you're a subject matter expert but I'm not just taking what you're saying as dogma right I still have to critically analyze but going back to what you just talked about with the professor that was just talking there the psychology professor so what I've been recommending this is kind of unique I guess is to I have a lot of instructors that teach freshman seminars so that's all right having to know all the different techniques here what I've been recommending to them is this idea of well one of the big things if you happen to use chat GPT which of course you can use it it's their choice we're very flexible with that here at my university but what I've been recommending to them because I'm very much into role play right I think that's how students really learn so I tell them this you need to increase this idea of how severely wrong it would be to have some sort of reference in any essay in any article that's wrong or made up so now what I tell them is that this is exactly what I would do is a student submits a paper and then I would say this if there was an incorrect made up reference I would stop the class and say you know come up you're now going to have to go in front of a board because what you committed was something that's extremely bad it's not just like oh you made a mistake no you created something wrong so that means that you didn't do your due diligence and you just lost credibility in the field wow this is a major thing isn't some minor problem this is wow you're probably not going to be able to present at conferences anymore you're not going to be able to be published anymore this is a major deal and again to role play it out so that they have this emotional connection as opposed to well I'm just going to use chat gpt and turn it in because I'm trying to save time so again whatever we can do to really connect some emotion to the process that's where we're going to make a lot of gains because that learning process needs to have some emotional connection in order for it to really be retained and to really sink in so that would be my little recommendation for that last incident but let me get to my question so with gpt4 that came out OpenAI also released an article to help explain gpt4 and interestingly enough in their article they had a whole section there talking about the dangers of overreliance on any AI model it was very interesting because they talked about some of the big dangers of if we start to use AI for all these other tasks that will you know save us time we're going to actually start to lose skills which makes a lot of sense but it's this balance of what we have to do because on the same day that I read that and I just released the video talking about the dangers of that Microsoft just came out with their big presentation talking about Microsoft 365 co-pilot and what is it going to do it's going to totally change how we do work so that you have a reliance on them to do so much of your in-between work summarizes for me give me drafts really quickly which is great I mean I'm all about productivity so my question to you then is what do you see as the proper balance and how do we express that in the best way to our students to get them to ensure that they have fundamental skills but then also be able to properly use the AI because that's going to be required in whatever job they go into because that's the new reality they still if they lose that fundamental skill they're going to be able to be as effective with that AI and they'll start to lose what Wright actually looks like so I'm really interested in your thoughts on that and if you've given any thoughts as to how we might approach that because this AI literacy thing to me seems like this is what we need to be pushing for in academia as far as the way things were a couple of years ago when oh the term critical thinking yeah that has to be part of every single class that has to be part of I see AI literacy as the new critical thinking I mean critical thinking is part of that but AI literacy I see that as being a necessity an SLO part of every single class from now on because it's so important all right that's it thank you oh thank you that's a great question and I agree wholeheartedly that it is crucial I mean absolutely crucial to get that type of critical relationship to AI and this is something actually even before the large language models I've been working on for quite some time which is where I get people to try to think of these tools shouldn't be a replacement for your thinking they shouldn't be a replacement for what you make how you act they should be a tool in your tool set for getting at this and the example I used I'll go in a slight tangent here because this is a different model from GBT but when IBM was pushing Watson and Watson for medicine in particular they were talking about and then Watson is going to come in and it's going to interpret all of these things for you and the implication was always and you don't need that silly doctor in the middle and what I would always point out is to say that look I don't have the budget that IBM does for Watson but I could on a rather beat up little old laptop since upgrading my machine but just right before the pandemic I had a laptop so I pretty beat up and few years old and I could show how by using some a tools for image recognition say very primitive compared to what you're looking at with the large language models and so on but very specialized very focused on how for instance you read and actually say you work in this region this region has a problem for instance with MBR XDR so multi drug resistant extensive drug resistant tuberculosis and what you have them is a scenario where your patients don't look like a generic patient so you can work with the AI to train it to your needs and now you working with this can do triage you can say right away ok given the analysis this person clearly doesn't have TB or given the analysis this person clearly has TB and then working together you can focus on ok this is the tricky case and that's the type of thinking so I was doing that as I say prior to the pandemic working on different scenarios with the idea that no you don't have to buy the big model and in fact you can do better than buying the big model if instead you focus on this type of thinking alongside the AI with it as a tool focusing on those cases that they work together so to speak in a certain sense gives you a better result than you could get from each chunk on its own and that goes to the heart of what you're saying because it's exactly the same situation now with GPT or for that matter with some of the image generation models right like stable diffusion and to say so how can I work together with this and this is where we need to work with our students and again we're making this up as we go along so please understand I don't have a huge list of you know and here are 10,000 examples here's the book you can go out and buy it tomorrow etc no we're inventing right now the examples where we say look here's where you using chat GPT can do something that chat GPT can do on its own you can do on your own but look at what happens when you have the synergy of the two components and what do you need to know for that synergy to be effective which goes to your point right and make sure but we need to create those examples so you mentioned something earlier which I think is also an excellent arena and I know in fact all three of us now on stage I know are interested in it which is the whole arena of what you do with games role-playing and games and for instance if you look at the guest brand that you have on your forum in terms of the whole concept of teaching history via role-playing with reacting to the past RTD. Mark Harms exactly and what I think you can do there is you can say well can I use chat GPT to generate what would what might have been a speech by filling the blank of you know whoever in whatever location so let's pretend this is a speech by Denton right made to a crowd as you know seeking support etc and then ask a student okay now critique it why would you find this more persuasive or non persuasive what about you construct okay now let's have this play out in the role-play with your fellow students which one of these works which one doesn't why was this telling you about history was this telling you about motivation how would you go about finding more data to make your speech even more persuasive how would you go about as a historian right researching the period and what could you use from GPT but what is not there from GPT that you need to bring to the table how does this influence what you need to know what you need to learn again I think we need to create this type of experience but really very much needs to be something where the students have their hands in the actual machine for a better term so they can experiment and feel for themselves if you will what's working what's working and where they're going to need to I like that so it's more of this dynamic assessment more of a skills application so we're talking about less writing of let's say essays to explain what I know and more opportunity to actually engage and utilize my skills to prove that I can actually go through it I can actually do it so simulation would be a great example of that yeah absolutely so thank you that's a great point that's great commentary friends we're running low on time so I want to make sure that we get in some of the biggest questions and we're going if you'd like we can also publish the unasked unadressed questions to my blog as soon as we get to recording it we have one question from our good friend George Station who circles back to the question of business and to the question of privacy as well related to equity should we expect our students and ourselves to keep feeding chat GPT for corporate game I'll turn in and others is that a fair tradeoff great question George and again this is one of those places where I think we need to push for transparency as much transparency as possible from everybody involved here so to to your point as far as I can tell right now given the contracts given the licensing agreements yes I've done the boring job of reading through the licensing agreements in detail isn't that fun and I'm not a lawyer nor do I play one on TV if I did that play parry Mason that would be at least more fun but anyhow but the point is as far as I can tell right now the GPT family whether it's three five or four is not being trained on materials that we're submitting chat GPT and then GPT themselves so the interface may be a different story and that's a very good question in other words do we have full knowledge of how much input is being used how it's being used I'd like more clarity myself and I think that's something that I'm gonna be honest with you I don't think it's the sort of thing that I am expecting Microsoft to say and here you go everything for you I suspect we're going to have to go and say hey if you want us to you know use this we need to talk about what these conditions are and again here is we're having Libre tools becomes once again a key component because one of the things that is if the only choice you have is Bing Bingbot then yeah you ask all you like but it's either that or nothing and particularly since Bingbot appears to be poised to become the major interface to all of the search engine for Bing so but if I'm the other can you can say well no come on I'm just going to go here where I do know the conditions I do know the toolset and I'll be honest with you some of the Libre efforts are very far along too so it's not like you're saying well here's GPT-4 and the Libre efforts are years behind I'd say some of them are very close and at comparable level so this is the point of which you can say we all have you know at the greater diversity of tools the better but there needs to be transparency in terms of how this is being used and why sorry let's take this in a slightly different direction we've been talking about chat GPT producing texts producing papers producing answers co-producing for us but the tool is actually much stranger and much deeper than that here's a question that I'd like to chime in on myself actually this is from Robert Ventress who says have you tried GPT-4 as new Socratic tutor system it's quite impressive and he also shared a link to some examples which I will not put in the chat for everyone to get a chance to play with the part I wanted to chime in with was I followed a Wharton university professor to coach chat GPT into serving as a game master for a simulation and found that enormously successful and very positive that kind of role that strong interlocutor seems very different from chat GPT as text generator what do you think about this part it's an incredibly rich aspect and you're right it's a very different use of the pattern engine than what you have when you're just using it for text generation that's part of the beauty of large language models this starts to get at some of the directions for research that I was talking about and for expanding the range of what can happen because one of the interesting things is yes I have played a little bit with the Socratic tutor as well it is very intriguing you know I have to I haven't pushed it quite yet as hard as I'd like so I've just begun to push it but but it is very intriguing and it speaks to possibilities that are to go to one of the things I was talking about earlier not locking ourselves into old forms of instruction with just an enhancement from the tool to say well what can we do where still can use the tool to get better insight into their own patterns of thinking to get better insight into how they might think what they might construct and so on so those are possibilities and it's very interesting too to see when you start pushing GPT-4 in these directions just a little bit to see where it starts almost for lack of a better word and I know what's going on at a mechanical level but let's metaphorically say it's almost like it's reaching and saying well clearly I didn't reach quite far enough in the patterns let me pull this in as well so it starts to speak to very much a model that can start very focused on okay I have an immediate issue here that I need help thinking through and so on but it builds out a scaffolding and it's not a pre-established scaffolding that I pre-designed what I call sometimes the abuse of Bigotsky and frameworks etc it is rather a scaffolding that is helping the student explore new directions and that's new in other words that is really something that we haven't had tools for so this is an area both for research exploration personal development and again I emphasize we're at the beginnings of this we all need to just start working on it with an open mind and also know that we're going to come into points work and say oh wait no that didn't work so how do we fix, how do we reroute and so on but I agree it's a fascinating direction and very different from if connected to they write the paper or summarize this or extract this information etc Very good, very good we have time for one more question another angle again and this is from our good friend Charles Finlay from Northeastern who asks this simple but major question what happens to copyright and publishers in this new era? Oh boy huge question Charles I don't know the full answer but let me tell you one of the things I think we need to do we need to take a step back speaking off just like I said take a step back from how we've been teaching and say hey we're going to come up with a whole new approach here we need to do something likewise with copyright because one of the problems is we've gotten down a set of rabbit holes for like a very term with copyright which becomes say well it's the life of the author plus so many years oh is that too many years is that too few years and does this corporation own the rights and does this person retain the rights when do the rights refer we get into a whole business of copyright as a whole system of ownership but we've lost sight of the purpose of copyright and this is where I'm going to suggest that we need to go back to the ideas of copyright and yes pull out the quote from Jefferson but it's not just Jefferson plenty of other thinkers at the time before during and after have weighed in on the topic with nuances complexities you know who gets to have access to on what for what reason but the idea that ultimately the goal of copyright is not just to protect ownership period which is I fear what sometimes it gets constructed as but to provide a scaffold for building upon so I think we're going to need to start rethinking aspects of copyright in this age and it comes out you see behind me an image okay this is from experiments I've been doing on learning spaces and thinking about them using Gaudi's approach to architecture and many of you may not know this but Gaudi designed the school and it's one of the most beautiful spaces I've seen conceptually even though sadly I've never seen it in action because it got made into offices years and years ago but there's a whole series of ideas there's the idea of solar power that's how we use solar energy so there's a bit of solar punk in here as well idea of open joyful spaces for creation by learners so I've been playing with the engine this is with stable diffusion to generate not one not two but literally thousands of variants on this idea to help me think through the process okay but stable diffusion was trained on a ton of images many and many and many of which are in fact under copyright owned by an artist owned by a corporation owned by somebody and the question is well none of those images subsist in the set it's a lot more like somebody going to a museum and getting a lot of ideas from what they see around them then summarizing them but at the same time I don't want to say that you can just go in and say well just because of that then there's no giving back how do we weigh this how do we weigh you know matrices of how we think about what copyright is what copyright might be that allow us to get the richness that makes something like this possible but without starving frankly every artist that might be producing art out there and recognizing that they need to eat living and they have a right to their art also continue to be recognized as something of their own with the voice of their own so the answer to your question is it's a great question and I think we need to think about it but on a very deep level not just at the what judge do we send this to or how many more years do we add or subtract to copyright well thinking at a deep level is what you've had us doing nonstop for the past hour ribbon and I thank you very very much for it but the hour has passed and I'm afraid I have to wrap things up thank you for that answer to that great question Charles ribbon you've been fantastic what's the best way to keep up with your thoughts on AI right now the best way to find me is once upon a time I would have said Twitter but Twitter has been a bit unreliable of late it still is ribbon RP and I'll continue using it so long as it can more or less be used a mechanically but I'm also I'm on LinkedIn and you can find my blog at hipassus.com slash blog and I'm also posting on there last but far from least I also work with Arizona with the shipping you project at ASU and some of my work is also in that context so a bit I will admit it a little bit this you know disseminated in different places but any and all of those should give you access to the work I'm doing these days well everyone we should pay attention to that follow that and he did and we're going to bring you back once we get the chat GPT version nine so we can see what it goes thank you so much and thanks to everybody for your great questions and comments I'm going to share we have a raft of questions that we didn't get a chance to go to because this subject is so huge so we're going to try and save those and post them to the blog so people can access them in the meantime if you'd like to keep up with us please use the hashtag FTT on Twitter or on mastodon or wherever you like looking ahead we have some sessions coming up on a wide range of topics if you we also have all the archive available if you'd like to go back into our previous sessions including our ones on AI above all thanks everybody for wrestling with this great great issue together I'm so excited to see all of these great minds thinking together to try to work our way to the best possible way forward everyone take care I hope springtime for those of you that are in hemisphere is coming to you very sweetly I hope everyone is safe and we'll talk to you next time online bye bye