 We have two sessions, you can enjoy this up in this room. We can have any questions. All right, hello, everyone, my name is Virginia Clint-Claesale, and I'm from the University of North Dakota, which I'll show you where exactly that is in a moment. And I'm here today to share the research findings from an in-classic study I did on student motivation, and I didn't mean to make that point, so I changed the title on the student social orientation, what are student assumptions, and how does social orientation work? All right, first I want to explain part of the reason I'm so excited to be here. I work in that red rectangle in the map from the United States, and you can probably guess from the location. It gets quite cold. And right now I am missing the biggest lizard of our winter, which we already have, our snowfall. So you see that red arrow? That is the direction the storm is moving, and my house is right there. I have my stepson on call to take his young, lumber 20-year-old self out to help dig out my 70-year-old mother, who is watching my children go to work at school. And that's my friend's cat watching the snowfall. And like the rest of us, when we explain, we'll ever come. Anyway, also a little bit more about me. I am a primary researcher with the Open Education Group, and I manage a research fellowship. We're a group of early career researchers from a variety of disciplines and backgrounds are supported through collaboration and mentorship and financial help for things like attending the Open Education Conference in the United States to research and promote research in open educational resources, and now we've developed more into open pedagogy as the future has come. That's a picture of us on Zoom. So my talk is going to be about open pedagogy and, or often known as OBR-enabled pedagogy, but basically things you can do because we have open license that we can't do so easily or not at all with closed license. Before I talk about open pedagogy, I want to talk about our roots in emancipatory pedagogies. These pedagogies developed by scholars such as those who were there about how to transform and redefine how we look at education. Historically, education has been a tool to meet racial and class hierarchies, especially in the United States. So what can we do in order to transform those knowledge structures to upend things through collaboration, through student empowerment, through student knowledge and ownership? And one way that this can be done, that's quite simple and straightforward, in my opinion, is through social annotation. Pause to take a drink before I run through all my slides. So what I mean by social annotation is there's a shared document or resource I used in OBR-cupsbook, audio, video, what have you. And students take notes on that resource. And unlike typical note-taking, students see each other's notes. They're allowed to comment on them and they can upvote their peer's notes. And upvoting would be similar to liking or farting something out of social media. The tool I used for social annotation is perusal. And the reason why I use perusal is because that happens to be what my institution supports. Perusal works beautifully with open educational resources because the licensing is flexible. You are able to post and have students share and comment as much as you'd like. There are other tools for social annotation. If you went to Kate Malloy's talk, she talked about hypothesis earlier. Hypothesis is a long-term tool in open education, but it works very well. A benefit of hypothesis is you can actually have the comments be public. In fact, some journals such as South American Psychological Association and MEPI, you can actually comment online publicly for others to see on their articles posted. So a future study of mine is to have my educational psychology students comment publicly that way. So students, this is not my class. This is a screenshot that I found from somebody else's class where I showed students names because I didn't get their permission to do so. So what do we know about social annotation? It's an area where there has to be research. So we know social annotation is a way that students can interact through a shared document or shared resource. They're seeing each other's comments. In terms of asynchronous, engage anytime, anywhere. This works very smoothly. It's a synchronous chat option if you want them doing it simultaneously. And it does help with peer interaction. And it's mostly examined in online courses because there's a tool for online interaction. It's more work that is shown in peer-to-peer interactions in general and allows for what's known as shared meeting making, which means students can share their experiences and knowledge as it relates to the text, which then helps their peers better understand as they see these shared experiences. Students can also ask questions and then peers for the instructor can answer them. And it's also been shown as a helpful method for encouraging students to prepare for face-to-face classes. Yet we all know that students are notorious for not doing the assigned grading before a class or somebody to hold them accountable to that rating. Oftentimes that accountability for a quiz or a reflection paper, this seems to be a good method as well. Looking at the emissions, students reported that social annotation means that they had more happy, positive, pleasant emotions and fewer unpleasant, negative emotions compared to individual not taping. Also, we know that social annotation has the potential to empower students. This is a way where students can engage in talking back to the text. So rather than having the text be this perfect source of knowledge that dropped from above, that they are just meant to take it in like a vessel, they can speak back to it. The encouragement to challenge it is an opportunity to encourage a way to dismantle those power structures. And also this is more focused on my specific area of research, but in my help with overcoming what's known as screen inferiority. So I did a study which made me very famous in the traditional education community as far as like economics go. So I found a paper like a better comprehension than screens and meta-analysis, a lot of very traditional, face-to-face educators. We're very excited about that. And like, no, no, no, that wasn't my point. I wanted to show like, okay, we have this problem with screens, but screens are here's how we work with them. Other people took it as throughout your candle with fables. Not my point. I had a lot of people very surprised that I'm actually very pro-educational technology and the opposite of that kind of programming. Anyway, I did a follow-up study for, you know, all the electronic tools such as interactive questions, video links, feedback, and such an annotation as a tool that you can only use within an electronic text. And I found that that really helps with complicated attempts in general. So this idea of screen inferiority is that if students are reading from screen, which students very rarely will actually get a paper copy unless if they have to, this is a way that can really help them improve their comprehension. So that's what we know. What do we need to know? What do we need to do? The thing is theoretical lens. So there's very much beating into me when I went to spread with students is what is your foundation? How are you connecting this to the roots of prior literature? Now it gets tricky when you're trying to do something transformative and knowledge-reducing and, you know, change things because changing things, there's always a great theoretical framework to go off of. However, a lot of social annotation research, including a study I've done a few years ago, lacked a theoretical lens. It was more of a, okay, let's look at how this works and let's see where some correlations are but without a real grounding inferiority. Another issue is social, there's a great paper by Brown and Croft about how social annotation is a wonderful opportunity for that transformative knowledge-reducing by students. And we really need some data to back that up. Where's the research to align with that in order to have the theoretical and the empirical and form each other. To face modality, you've done a lot of wonderful work in the online world, but, you know, we still have some face-to-face students. So how do we engage them, especially if we are expecting them to use these educational technology tools? And then grace. Others, a study that was done by Adam and colleagues. And they found that students who were a scientific group that did Brazil before class or social annotation before class did better on the exam than students who were not. But it was, they weren't sure, was it the social annotation that helped them learn or was it that the social annotation got them to read the book and to think about the book and to process the information related to themselves. So what exactly is the, what we call in, you know, my nerdy world, what's the mechanism going on here? So I wanted to peer into that event. So my framework is self-determination theory. It's a theory of motivation where humans have these basic psychological needs, autonomy, competence, and relatedness. We have optimal motivation when our needs are met. If these needs are not met, it is harder for us to be motivated, especially intrinsically motivated. That is to do something worthwhile. My thinking in this study was that with social annotation, we definitely have high levels of relatedness, especially thinking of things like quizzes and individual note-taking. Probably a good amount of autonomy compared to quizzes because students would get to choose what they commented on and where a quiz is given to them and they have to know X, Y, and Z. For competence, I wasn't so sure about. Our students could be really nervous about this. Are they going to feel confident? How's this going to go? And then a social justice framework. Sir Lambert has a very nice article going over these three components of justice. And just by the virtue of open licensing, we automatically get redistributive license justice with OER because those materials are more accessible without the financial barrier of an access cost. There's still Wi-Fi costs, internet costs are still costs, but they don't have to pay to log on. We're cognitive because of editing, we're able to change things in an open-layed license textbook. That can allow for better access for people to see themselves. But where open pedagogy really has a lot of potential power is representational justice. And this is where students are allowed to have some power and don't need to speak back to the text. You get to flatten our structures. And this also gives an opportunity for communities to have been historically underserved in education and to speak from their voice and their experience. So representational justice is something I consider in this study. So here's my research question. I compared it to quizzes in individual note-taking because quizzes are commonly used as a method of accountability between reading before class. Individual note-taking is an obvious comparison to social annotation. And then I was also inquiring as to what's the seed for representational justice. And then as far as how this matters with grades, I've looked at associations. So basic correlations and then a little bit fancier mediation model I'll talk about later on. So in my study, I used my basic face job development course. This is mostly first and second year students, mostly education students, mostly female, mostly white. So it is not the most diverse sample and I want to begin with that caveat. However, this is material that is brand new to that. They're usually it's fall semester right at the beginning of their college group. And they were required to write annotations at each chapter before we covered that chapter in class. In groups of five or six students. So they only saw the comments from their peers in their groups. After they were done submitting all of those, I went through and you filter and only see the questions which I recommend because 45 students times six comments it's a lot. So I would only view the questions and go through and answer the questions and you can take them so they get a notification that you answered that. Then I asked you to do a survey and you were very nice and almost all of them with the motivation. I use analytics from the through the soul software itself. It gives you a wealth of information. So how much time did they spend reading each time with this actively reading how many annotations they made for each annotation. I looked at a few key features I'll talk about later and then their first grade minus the credit they got for each social annotation that same that's again one of the different social annotation with social annotation and just to clarify very low stakes assessment the directions were six comments including your peers and comments to your peers is encouraged and you had to have them completed by a certain date if they did it they got full credit they didn't do it they got less credit. So I didn't dig into the quality of their responses or anything. This was a very low stakes. You can make it much stricter and much more of a qualitative approach. So first question what were their motivation laws? So without autonomy having those feelings of independence you look at choice and pressure you can see that choice and social annotation was definitely more than quizzes but less than individual notation which makes sense social annotations require rarely or seems required turning notes on their reading but they definitely felt less pressure compared to a quiz and both the quiz and social annotations require competence they felt more competent than a quiz but both the same as individual note taking so that feeling of skill can I do this approximately to say relatedness was higher with quizzes than individual note thinking I honestly was really I'm like relatedness do really feel with individuals and then interesting enjoyment can see is about the same with the two note taking there's some comments they had about what was interesting or enjoyable let me look over but mostly it was things I liked getting to see what my peers had to say and I liked being able to share what I experienced with the book to be able to share it out with my friends Next question was on representational justice so this was asked in a instrument that I developed with my colleague Lindsay here's some examples of items in the representational instrument we developed and it's a scale of 1 to 5 I only looked at social annotation because I cannot think of how to phrase these for quizzes and individual note taking and not have it just be really weird like my voice not I would have never in a place so what I did instead was I compared that to the midpoint and found that it is significantly higher than midpoint so that's not awesome but it's better than average in other words we could do there are ways this could definitely be developed Next research question what about some correlations and associations so the measure of rates in the force correlated with how much active reading they did that meant they can just have it open that they were moving their mouths and flipping and gauge storeroom the number of annotations average words for annotation the feelings of competence matter and the feelings of relatedness so there weren't any reliable associations with those other measures social annotation I did what's called a mediation analysis and I found that the more annotations the students did the more they read actively and that explained why they're great so it seemed like this really helped motivate them to read and understand the chapter and to spend more time with the material and they're also reading their pierce talk so discussion and conclusions overall there's clearly more motivation for social annotation than quizzes like that is a clear take on but it's pretty mixed with individual the more related same competence but less pressure somewhat promising representation of justice findings I'm not going back let's over that finding by any means but it's a it's a tiny step in the right direction kind of inquiry we really need to be doing so this is one anybody data point that I'm hoping it's going to be part of a larger trickle and stream and reviewer study and social annotation and course materials it encourages students to interact with each other and the text and interact with each other about the most context and what it's meaningful to them and so my future directions I'm going to be using this like areas to edit the open textbook the OER we use is excellent but I did realize going through it with my students that a lot of their comments I was like oh that would be worth it but that could be stated better so their upcoming assignment this is my sabbatical project is to develop it they're going to be flagging areas that purposely need to be improved which then the next cohort of students will then use the information to look at the text so my grand adventure will lay out me back in a couple years to share all about how it was at least so much successful I'm going to do a controlled experiment comparing social annotation with individual note tagging the learning management system allows you to randomize groups and hide the directions for you can only see the information for the group there so one group is only going to see instructions to individually take notes and they're going to be required to upload their notes the other group is going to be social annotating on proposal it's an online course the odds that they're going to know pretty slim so we're pretty excited about that my colleague Alison and I are doing that this summer this I'm also going to be doing comparing social annotation to quizzes so I'm going to be able to see which one is better first I'm going to be doing the course I'm going to be doing a deeper examination into social justice particularly with more diverse groups my institution is a white institution I'm not proud of that but that is the reality of the matter you really need to be looking at this for students who aren't the ones who have been traditional in the world so that is a big area that's my correlation matrix because everybody has knowledge and then here is my contact and resources I'm on TikTok until it goes down so yes this is my guide renewable resources that covers social annotations as well as a variety of other ideas and then that's my materials and data thank you very much 3 minutes no I love what you're doing it's fantastic, very inspiring have you thought a further step in student empowerment would be to move from annotating into repurposing republishing because annotation is more or less like glossing in the middle ages but what about incorporating have them flag the areas edit the textbook and then have those flagged areas be used to actually what about then oh they'll do it they're doing the flagging and then they're going to help find the materials to improve it not then publishing down they'll be they'll be listed as offers so they will be publishing it like I said hopefully this will be moderately successful at least any other questions have you ever looked at annotations that are not text by either voice or video I have not personally looked at that so bias admitted here I am a reading comprehension researcher I like words but yes that is something I have thought about is there's things like voice thread and audio replies or video replies perusal setup is text only so that is why I use that thank you Virginia your work is wonderful and the number of annotations does that is that a predictor of active reading time or factor? yes it was a strong predictor of active reading time so I kind of rushed through that if you're teaching people and you're writing them to put in annotations is that helping them develop their active reading time or is it just a kind of an artifact of I'm an active reader when I'm reading a text which one comes first? well it could be that they're reading more so then they post more and I was surprised students were posting more than they had to yes yes and I found that with open pedagogy in general but with Wikipedia editing my students were giving me way more than I had aspired that is not a normally a problem I have yes thank you for the presentation so one question I have is I work with a lot of faculty who are interested in social annotation in general and it's what we normally use it's very similar one recurring problem that faculty reference with social annotations doing scale I'm just curious how does the user manage that aspect managing and creating oh yeah the grade book doesn't actually work up for you so thankfully I have 45 students and it's a very like you did it or you didn't do it so it only took me a minute or two to flip through and hear the words manually so but there is a through the grade book that if I wanted to set up requirements I could do that like if I wanted to say your grade depends on reading this many pages and opening it this many times but yeah I just haven't done that and then I'm sorry there's another person we've got literally a minute so is it a quick question yeah I just wanted to know tell me how are you going to navigate the efforts around the students taking part in the social annotation and maybe a word yes they get an informed consent document that they'll be in a research study and if they do not want their data used they let me know so yes excellent question you should always consult your institution but thanks for doing it thank you thank you so much so there's a bitly link at the bottom so sorry I didn't do the QR code I actually locked my laptop due to my son changing the password right before I left so I have to get back to Canada before I can get into my laptop looped so we'll give you a second so talk a bit about some research that we've been working on at the University of Calgary just to talk a bit about Virginia so Virginia pointed out that she was in the middle of that blizzard if you go 10 hours north that's where I live but we don't have a blizzard we have beautiful weather and I'm not sure you know we're always with so just checking as we're going on I'll come back to this at the end to meet it in Canada it's really important to acknowledge the traditional territories in which we come the traditional territories between seven regions of southern Alberta whereas the University of Calgary is located even though I'm from Edmonton which is which includes the Blackwood Confederacy confines us as Singapore, the county, the United First Nations as well as Zucsina, First Nations and Stormy Dakota including the Zucsina, First Nations and the City of Calgary is also home to the Métis Nation Alberta region number three and this is when we talk about the land in which we come from in Canada we think about why we appreciate the land as well and in this case this research all stemmed from my dissertation research and I was that naive I would say successful recipient of my dissertation or completion of my dissertation research and part of it is we get to go to the Dean of our University Dean of Education and have a little meeting with her and we talk about what our research was about which I think was lovely but I went to talk to her about open education and she said that's nice dear basically she might not say dear but that's how I felt but that's your thing like this doesn't exist in the rest of the world and I was shocked not only because I went into a knowing about open education around the world but she then prepared it to her study on classroom uniforms and I thought wow like you really it was that first moment really in research thinking I had changed the world and had really done nothing even though my dissertation was open it was very creatively common licensed no it didn't mean anything so I went outside and this is how it connects back to the land and I looked out to the mountains and how you could cry and said I'll fix her so what I did was there's a scholarship teaching and learning opportunity on our campus where you can apply for about a $2,000 bet and the people that I did change I did make a difference for my supervisor or my supervisory committee and together with my supervisor my supervisory committee and students some of my classes so some undergraduate students and some graduate students I suggested that the course that I was teaching I knew that it was in this which I'm going to talk about in a second this graduate course so to be a program called meeting and learning in a digital age I said I'm going to work for my students and we're going to design and create our own press book or textbook for our graduate course and they said I don't know what you're talking about but fantastic sounds good, we'll support you so that was what I did for my dissertation research which I had my own framework I knew where we were going but we used the dissertation research in order to support the solo research so Dr. Michelle Jacobson the senior that was my supervisor Dr. Barb Brown was my supervisor and Dr. Christy Hurl was one of the librarians we don't have an open librarian necessarily at our university the university is about 35,000 students we have an undergraduate we graduate students in Edmonton I work at Concordia University also important because I like to work for them and there's a reason I'm here this week as well and they're really fit I was in students me and Charles Hayward undergraduate students and a cool newsletter was in one of the classes we'd be talking about so leading and learning in a digital age is part of our master's of education in the interest of standard learning progress you start off with a graduate certificate by taking four courses and there are specialized courses and the goal is that they all build upon what it is that you could just get a graduate certificate and they call this part of the micro-prudential program so in regard to some other programs which is kind of what you would want to or if you take eight courses you get the graduate diploma often my students will take design focus emphasis for their second specialization topic or they might take neuroscience or like the ed site kind of focus or you know all over don't necessarily connect between the two but there is I would say some discrepancy and room for improvement and some interdisciplinary learning between those two then to get to their master's in education program they take form where requiring research courses and this is really important for this study that we're going to talk about because this is the first time that they really have their research skills so that's where the emphasis would be in the third year I guess you would say and then if they wanted to apply for a doctoral degree they could merge this we use it in different conferences obviously but it's quite a big context for everyone here we use it as a definition of open educational resources and we used the wonderful captain Cronin's definition for open educational practices although I would suggest that as this research continues we've changed which definitions we've used for open educational practices so this research question I kind of have to explain this wasn't the original research question the original research question was just it had to be very focused on OER in order to get to $2,000 so you had to create an idea on OER and it was to what extent do students benefit from co-designing OER something like that but when we did the research and this is what we're going to get to we discovered that there was a lot of data that really started to inform and emerge about research skills so we said that our ethics after we did our original research and we came up with this research question how do open educational practices support the conditions for student learning of research based skills so this was done over two years two different iterations of the course and it was open-ended one-on-one interviews this was all done after the course was over in terms of ethics or any questions about it was all voluntary so it's not anonymous because we obviously I didn't know who said what but everyone else did because I taught the course there was a survey with 18 on-the-line questions and the artifacts were all the students could decide to share the artifacts or not there were only 13 participants in the survey of 24 so it was already 23 possible and the interview participants were 8 and the OER chapter contributions were 15 so 15 total chapters of 23 possible from the students you always had a choice whether you wanted to share or not so my EBR I was so fascinated yesterday they go to the end day because when I started the go to the end I was one of the first students to consider using design based research is that true what I would say is that fair to say? I can't think of another one there we go so we have it here yesterday there were four in a row that presented so I was really excited now design based research is really it can be really challenging because of the length of time to do the iterations required in the phases you don't have to do them all for dissertation research that's one great thing about museum but the good thing about DVR in this case just as innovations and it sustains their development so you're you're choosing to use this research approach in order to create or to focus on intervention or change you know there's something wrong and you want to make a difference from an educational context it's not confined by the methodology so to change the research findings then back into the cycles so I kept changing the design of the course I changed the methods that were created in the chapter we changed all sorts of things as a result of the findings as we went along so the big difference is that you don't find findings at the end and you didn't make changes at all which is the next one continual and preventive practice the uniqueness of this is that the researchers and practitioners work together so the students in the course knew that I was doing this design based research as we were doing it but they knew that I wasn't they always could they knew that they would have a choice if they wanted to participate or not if they never wanted to stay paid at all, that was fine none of their data was taken but they would still be a part of the process which made them think about and model what research could be it's problem-based and it's theoretically informed and hopefully it contributes to theoretical insights but also design principles and changes and changes of practice so getting back to the graduate certificate they take a summer interdisciplinary learning and technology course and the focus is on completing a critical article with you and I will definitely suggest the social orientation as offered to me there their fall technological literacy course their focus is on a literature review and then they got me and they take the winter ethics and technology course and at that point in time there was no textbook so it was a great opportunity to create our own textbook anyway and they created their draft OER value script for their chapter and then in spring they take a leading citizen tree in digital age and they actually take their idea and figure out how they put it into practice from a leadership perspective when we were designing this program we took those initial ideas and we compared them to those in an audience facets of development research skills so we knew from the original initial design our expectations like what we expected we didn't see so when they did this this is what we hoped we would see what we didn't anticipate was the increase in skill development as a result of number three the course number three when I used both educational practices was interesting so this is a very brief overview of what we do in the course to give you a design of it the overarching is the course is full of open tasks which encourage feedback loops internally within the course and externally that would mean using social media social notation connecting with experts asking your significant other to read your paper things like that just taking that first step on sharing and we did a lot of reflective activities including blogging a lot of blogging and difference so they started these are the things that they were assessed on that built upon each other throughout the course they have a digital outline they did a one-minute pitch they did a draft chapter they received feedback whether or not they wanted to include it in the press book during this time the program director also went into their chapter they gave them feedback we noticed the number one thing that helped us realize there was something about the research skill development was their initial feedback that said I've never received feedback this is really hard for me to handle right now just give me a minute because they were really specific we were getting them to that level that writing level that you'd want for an article or something else but we didn't really judge the way they thought but we got a lot of feedback because they weren't used to it because they got feedback almost immediately like as they were going along they just kept giving them more and more feedback and they had to learn to deal with constructive feedback they were also giving feedback to their peers and we created social pods which are groups of three to four people of different interests and different perspectives and they gave each other feedback and received feedback and this was really important because it showed that you could have a totally different topic but you are still able to academically think about a topic and think about how something is structured as an article or a chapter but also what theory from the course from outside the course is important I provided the students with a choice and they're used to tools and approaches in fact we tried to pull it all in and ran it all in and it never worked and we provided some important frameworks of a class of tools for example tools suggest I would share with them how to use Twitter this is a template for how you can write it to your top here and I modeled for providing constructive feedback so we have two versions of the class but we have the first year version and the second year version and on the PowerPoint you can get the links to this so you can just have one and these are some of the participant responses in that initial that initial research like 92% of the survey participants have read that connection to outside experts in class and perhaps they're learning participant engagement and feedback is beyond the duration of the course specifically what that meant is when you had a hashtag they kept going after the course was like the course that never ended reflecting how they had heightened commitment to ensure that their original average would top of their interest with the designs because if you chose to complete an actual publication as we all know it doesn't happen overnight it should be another year for it to actually be published each one every year their commitment outside the course continued in that they kept talking about it they kept engaging with each other the integration of Twitter and publicly accessible blogs made the learning open to the world over there before authentic the hashtag and the ability to determine the subject of the chapter created an internal motivation I don't think I've ever seen internal motivation written by my students talking about how they were excited about being internally motivated and enthusiastic about learning this motivation in their opinion is not but not exist would not be strong if the subject or chapter was a signed by instructor so everyone obviously got to choose their own topics as long as they could connect to topics and technology and then the other points were cohorts and peer feedback strongly supported my learning they feel alone in my learning I felt like I was part of a team I felt like I was collaborating I was with others and that each participant through the survey agreed the authenticity of the assignments connected either to professional or personal interest and increased their learning standard so then what we did was we coded the data and we used three different frameworks to support us so at this point we took the facets of research based skills that we used to design the course and then we also used the attributes of open pedagogy from Tiger T we used dimensions of connected learning curriculum framework which some of you might remember from UCL from their connected learning project we compared and contrasted them and these are our major findings such things that emerged the layered assignments the formative feedback and the peer learning proved to be the major conditions specific things that we do in particular practices that led to the increase and development of their research skills so offering renewable assignments to students could develop ideas as progressed through each course so the idea that the work that they were putting in they'd be able to use later would be thrown out they provided accountability through continuing to be needing to find information their learning never stopped they could connect with peers in their own course or outside the course they learned to critically evaluate others and organize their information synthesize new data for example and in peer learning specific as we use the social pods they were able to mark an increase with peers with diverse perspectives and experiences and they helped each other find and analyze and synthesize the needed information we got them to blog as they went through as I said and they had to answer each other's blogs to reply to each other's blogs obviously they were learning from and with each other as they were going through steps of creating and writing the chapter so they were able to see that everyone goes through and see and the issues that you go through writing a large paper like this but the other most important one is that form and see back which they received from their peer instructors outside experts lots of outside experts and this helped them find information there were always some negatives some found the feedback overwhelming so that meant that there was too much feedback and again I don't have that there's too much feedback in this part and the primary reason was because they were getting feedback from fellow students as well as outside experts as well as myself so they just needed time to think about that feedback and figure out what they were going to do and the other one nope that was really the the negative sort of the thing to work on the talent so when we think about it the condition one most important is design of layered assignments for authentic learning and engagement specifically we encourage them to think about authentic tasks or sorry be integrated authentic tasks with learning experiences that's what we do you know professional practices like having a Twitter chat which we won't probably do anymore but we would think about it doing it differently with some other social beings while teaching them how to do it so I think it's the important part we intentionally went into this thing to be about the programming course design and made sure that there was scaffolding between the courses as well as the program itself we looked at ongoing and constructive formative feedback and connection to experts internal and external feedback here in learning we ensured that there was support and timelines so we didn't let the students just go off into it all by themselves and make sure that there were specific timelines and check-ins at the peer or social group peer groups or social boards well these are some things to think about if you want to do something like this you need to have a coherent program and course learning design the students need to know what you're doing and why you're doing it that includes learning outcomes infrastructure you need to have access to textbooks or whatever the open source tool that you're using be it your institution or region sometimes it's a great idea that you might have but you don't have access to which is not the one that my institution uses but you don't have what software this was a team commitment I think I learned that more than ever as a dissertation student I thought I could do everything by myself and I thought I did everything by myself but the reality is research is a community is a huge community so our team has diverse expertise in ODP pedagogy, full design digital often tools, copyright library and peer review, copyright editing it really helps that you have students focus on learning and collaborating in few ways students who didn't really want to buy into the process felt that they didn't get anything out of this and that we have others that would never do anything differently we don't know how that works and it's really helpful even to be motivated and we don't stop in the second iteration of the course when we didn't have any funding so thank you for the people I think we really like you to read it under I want to speak to another plenty of type of questions only one easy question but this is wonderful because it has a great vision but also you have very much value driven it also has the granularity we can tell what you're doing very useful and very practical I've seen a lot of stages in those processes in which one could use for the students chat GPT have you considered how integrate purposefully chat GPT in any of these stages of the collaboration yeah so I'm an advocate for AI mostly because it don't know how I'm going to live through the world without it's not something I want to fight I want to fight some amount of time in those original stages where you get them to write their original drafts I would encourage them to use TAPI tier and AI tool to start those ideas I think that you can really be inspired by some of the that's why you do the best for AI to get them going I know even with my son who is in high school we both texted into chat GPT the same thing and his essay was different than mine so we came together to look at why a 16 year old essay would be different than my age essay and the big difference was the way I asked the questions how I changed it to get what I wanted and in that moment I said to myself I need to focus on it I need to help you shape your questions I need to make those comparisons and we had a great conversation and we talked and talked and as a result you had a better essay but we both started with an essay so I think it's an easy easy way to use AI for chat GPT or whatever I'm invited to chat GPT as a a learner for students it's kind of like a toddler so how to use it please do write something about this rather than that kind of demand we get to use it point to neutralise it and talk about the kind of errors yes when it is about the errors because what we're trying to teach is critical thinking they know given them the template they're still struggling with that in a master's level that's something they can help with because of peer the other thing that I didn't mention sorry I was going to say that the number one person I asked Martin Weller to jump in on all the topic with edtech ethics and I use Google many of you and Martin would go in and it would say Martin Weller says this you know how it does so the real point is that's why the expert kept coming up all of you are making contributions in different ways all of you make them cry in a good way and get excited when they get that feedback the feedback was the number one thing of course Martin Weller's but all of us make a difference and that's what they achieved I wonder what the rationale was behind having two different versions number one one version for the first it was just the course outline or the press books it was just those are the chapters the students wrote so you didn't want them to be one together no they actually there's a real difference between them the first year version they didn't have an example or really anything to follow but we all learned in it and there was some the second version second version is strong and there's some really great topics because it was also the first one was when COVID started and the second one was during COVID so the second one has why you turn your camera on or off full of really good chapters really good chapters in the second one thank you and there's ones on AI and that's kind of me too that as an instructor I would be able to see the progression of that tech through these press books that were being created but then I wasn't hard to teach in third course that last year so I taught one of the different ones before so be soft questions I was wondering what standing between this type of approach might work on the graduate level so I see I think it's just the size of my writing so I changed it I used to be at it my undergrad class was a lot more it's the same idea it's just what you asked me to do the assignment I totally agree with the social entity it's that collaboration feedback feedback feedback they think they're contributing they're building knowledge they think they're making a difference it's like I wanted to when I got out of that dean's office and tears thinking I made a difference but I did thank you very much some stop with the time though the next sessions we'll start by