 yourself. All right good I think officially afternoon now everybody I'm the only thing standing between you and much. Thank you so much. I'm Will I work at the North Carolina State University Libraries with. I'm David I also work with Will at the NC State University Libraries and we're representing a team of really really incredible people some of the names you see up there as well. We're excited to talk to you today about something we've been thinking and talking a lot about. Thank you David. Next slide about open pedagogy and about our sort of quest to take it from a really really cool one-on-one or small group activity to something that scales in the way that open educational resources can scale really really well. So we're going to talk a little bit about what we mean when we use that term open pedagogy. We're going to talk about this project that we've been working on called the incubator that we've done at an individual institutional level at a state sort of consortia level and then a few other levels as well. So very very quickly here we a lot of us came in the door to talk about cost and we recognize that there are really important reasons around equity around strategy and so forth to think about and talk about cost as part of what we talk about when we talk about open education. Next slide please. Thank you. But we also recognize that it's not enough to stop there that open education can't simply be about cost. There are I think pretty important reasons around equity and transformative pedagogies that make us think about more than just cost. There are also really important strategic reasons that I could talk to you at great length about about why I think open pedagogy and open practices are a really important part of what we want to talk about when we talk about open education. The open license and the sort of open enabled stuff is a small part of that but I'll let my friend Robin on the next slide say it much better than I could which is that this as a movement can't just be a movement about replacing crappy expensive textbooks with crappy free textbooks. That open education has to be about a vision that is about making it better. If you talk to an open science person they'll say open science is just another term for science done right or science done well and I think open education similarly should be about education done right or education done better that's the open piece of it. Again Robin said it better and faster too. Let's go to the next slide. This is a really really big area and in the next 15 or minutes or so I don't think I could do it justice so I'll commend to you Virginia's article among many many others if you want to get into the nitty-gritty of what is all this what's the difference between an OEP and open pedagogy and all that other stuff that's a super fun conversation to have but one that we're not going to talk about much more beyond this right now except to say that when we talk about it at NC State in North Carolina and the places that we have these conversations this framing has been pretty useful for us. The idea that open pedagogy or open educational practices are an access oriented commitment to learner driven education. So when we break that down we think about an access oriented commitment on the next slide as being this idea of equitable pedagogy so one size doesn't fit all that when we talk about this stuff education can't just be about the presumed normal average student with all the sort of problems and biases that can be built into ideas like normal or average that it should be about more than that and then the learner driven stuff is about more than intellectual stair masters right now we want to get beyond what they call the disposable assignment and have students generating new knowledge that resonates beyond the walls but that's a that's a pretty imperfect and hand fisted definition of something we spend a lot of time thinking about it's probably good enough for our purposes today so access oriented learner driven the other thing to say about open educational practices and open pedagogy is it can be a pretty heavy lift if I talk to a faculty member and I say I want you to replace that textbook with this open textbook okay I need to maybe revise my syllabus a little or look at the TOC and see if it fits but asking us to do access oriented stuff in a way that's ethical that centers students agency asking them to do that that's sort of learner driven stuff there's a there's a big ask that's built into that so on our campus we've had a tremendous amount of success putting in the time and sitting down one on one and saying okay for your course that's changing up completely for your course we're going to do this other thing but when we start to talk about scaling that stuff on the next slide and we start talking about not just looking at the really cool models that we have out there for changing a classroom we're talking to a faculty member about the values and the practices that they bring but when we talk about actually scaling that up to work that's done across the next slide please thank you the state level or a system level or a consortium level or a national level that can start to get more complicated and that's the problem statement that I'm going to set up and then do the good lawyerly thing of setting up a problem and then running away so David has to solve the problem here today how do we scale this open pedagogy stuff that we're excited about David I'm going to shut up thank you yeah well I think one way in which we attempted to address this at NC State is through our open pedagogy incubator that some of you might be familiar we have already seen a past presentation about this this is a program that was NC State exclusive in its sort of first iteration and it sort of it was planned before the pandemic and it was going to be an in-person program whereby a small cohort of faculty come together with librarians and they develop competencies in open pedagogy through this series of curated readings cohort discussions and workshops that take in different tools of open pedagogy such as wiki edu or hypothesis among other things and then the theories and the values pedagogy so the way it works would be that if you were a faculty member and it sounded like it might be exciting to you you would apply to the incubator and then your application would be reviewed and then it ultimately might be accepted for the first cohort we may ask you to wait and join the second one so our first cohort we had six members we had 19 applications and actually provided a very small in the grand scheme of things grant for those interested in joining that program our structure is based around these three sort of key meetings the first meeting is kind of the kickoff meeting of the program this is where you come into the room you meet the other folks who are going to be part of your cohort and you meet the librarians and so on and this is the point we're really where we find out about well why are you here what do you find exciting about this why do you want to do something different than what you've been doing up to this particular point our second meeting is at the point really whereby you've you've been through the sort of core readings of the program you've taken part in some of the workshops you have an idea by this point of well what's exciting to you what do you think might not work in the context of your your pedagogy and so on and things that you're unsure about you can sort of bounce off of your different cohort members and everything else so you don't yet have a concrete plan for exiting the program with an open intervention in mind but you're sort of at the halfway point of getting there and then the final meeting is you've completed the program taken on board everything that we've thrown at you and you leave the program with this concrete idea for an open open intervention and implementation whether it might be a wiki edu assignment building a textbook of students writing quiz questions rather than have students taking a quiz or just making your syllabi open so all sorts of different levels of open intervention now we supplement this we have different workshops I mentioned a few of them already we do ask our participants to attend at least four but on the menu is sort of far more than that and what we've tended find is folks generally seem to prefer taking as many as possible so rather than the minimum amount they want to take on board as much as we can as much as we can offer and then as mentioned you know come the end of the program you should leave hopefully with a concrete idea of an open intervention so the way that we've sort of done it is sort of a flexible blended structure I mentioned this was going to be an in-person experience the pandemic sort of sort of that so it quickly became an online only and in some ways I think that was kind of like a happy accident because as I'll come on to talk about I think this might be the ideal format for something like this so again so foundation meeting workshops inspiration and then the second stage is where we sort of turn over part of the program to the participants and we provide a much larger menu we ask for them to decide amongst themselves different topics that they want to explore first half of it is mainly like the core stuff that is essential the second half is much more elective in its nature and we've been quite intentional in terms of how we've built these cohorts this is interdisciplinary as fun as it might have been we didn't want to have a bunch of you know mathematicians talking to another bunch of mathematicians that might make sense in other scenarios as I'll come on to talk about but for this one we are keen to have instructors at different stages of career from different departments in different disciplines and we find that I think across my first and our second cohort on the next slide as well so this is very much a model that we think is built to scale we've done it successfully at a local level but we think through virtue of this being online only this is something that is scalable for certain and our next tackle to challenge was the system level one so next slide please will so NC State as some of you will be aware we are one of 17 schools in the University of North Carolina system and you know we are keen like we're very open minded the community aspect is important to us so the taking the incubator and transplanting it and broadening it out to a system level is something that made a lot of sense for us and for our friends elsewhere too now I'm going to come on to talk about a sort of particular pilot program regarding math pathways which is a system level sort of framework for our sort of introductory or lower level math courses but I just wanted to provide like a bit of context before I move on to that so math as many of you know and I know for my math GCSE it's an extremely difficult subject for a lot of students at the system level like sort of one-third of students didn't complete their math requirements they had to repeat or impeded their progress or so on and it's particularly in true and some of those particularly challenging gateway courses like algebra which is basically my kryptonite right I did my undergraduate in England so I didn't have to do this portion very luckily will unfortunately did so with this in mind the system decided to approach this from a different different angle and they created these alternate pathways the math pathway program which was implemented in around sort of 2020 so this is from a student's perspective the math course that you take is more closely aligned with your major so for instance if you're a liberal arts student in theater or humanities so on you won't have to take the college algebra you'll be taking something that's more closely aligned with your or future career goals so curriculum is developed that's just more responsive to students needs and also just more equitable and applicable to what they're interested in in studying and an advantage that the math pathways program has perhaps over some of the other disciplines is it has this common framework that runs throughout the system all undergrad and lower level math courses and also within our community college system as well so it enables this almost like seamless transfer across the system so I hope you're still with me on that I never thought I'd be in the front of a group of people talking about math education believe me so this presented an opportunity for us so math OER has been done many times before it's often at like an individual level when you try to scale it to a system or a consortium level it's been sort of much less successful but bearing in mind the advantages of the framework of our gateway courses and system and the opportunity as we see it open education to do things at scale this kind of was like a many ways like a happy marriage of convenience so what we decided to do was pilot things bring together a bunch of math instructors bunch of librarians and a bunch of instructional designers or learning technologists give them another another name into a like a large group so we tried to represent as many of the schools in the system as possible and I think we near enough hit most of them we would I should say this ably like enabled generously funded by the lyricist foundation I think of them as foundation yeah the lyricist provided us some money in order to sort of perform this work and therefore was born the summer institute for open course development try saying that after a few weeks this is a program that essentially looked to take the incubator model and apply it to more of a discipline specific or solve a problem which was creating math OER or open educational practices at scales and it all sort of sort of was brought around this particular math course which is the one to my fire right here is quantitative reasoning math course which most liberal arts majors in the system level are required to take so this course was redesigned through virtue of this program and used by hundreds of educators across the UNC system and maybe even more importantly has like significant buy-in from those educators so imagine like the impact for students across 17 schools you know imagine the the scale in which this work can affect and also again go back to community going back to the community aspect through bringing all these folks together and developing localized inclusive OER and OEP we were able to develop a community of practice because one thing that's really important to make sure that things succeed and open in general is there has to be sustainable and to be sustainable often you do need to have an ongoing community of practice around it you can work together on these challenges so in terms of the activities we broke it into sort of four chunks so the first one was the course mapping in design where the faculty and the instructional designers sort of work together to map the course identify what the resource needs are for the lessons activities and the assessments the librarians participated in those meetings and we use one of our skills you know to good effect which is we're quite good at finding stuff right so we could locate the appropriate OER resources then we moved on to building the course so faculty the IDs and the librarians collaborated to build the course continent in an open shareable environment and then critically training and guides were provided to support faculty who then come on to teach the course so our timeline sort of looks like this um he applied to and was successful with lyricist back in the early spring of last year probably about almost a year ago today maybe and sort of work began in earnest in sort of June 2022 September 2022 stage three was the real sort of critical moment where folks participated in this sort of incubator model I spoke to you earlier on we're now in the course delivery and the assessment phase and then we'll move on to sort of you know any revision that needs to take place and a white paper now of course will be the toolkit being published for the wider community so in terms of outreach sort of how we got the sort of the word out there one of the things that we were keen to do because it's one of those things where if you build it people necessarily won't come unless they don't replace it we we connected with the chief academic officers and disseminated this information at the university and community college level to make them aware of it so in terms of communications that also looks like things like a lot of webinars blog posts both during and then what we're moving on to now is is more sort of post and then that white paper and that toolkit that will be developed and made available this is my sustainability piece our UNC system OER implementation collection so it's available for anybody to find anywhere in the world and use the toolkit that's been developed and then pivoting very slightly back to our original model because this seemed to work very well we've also as of like right now also have system level incubator which is moving outside of the math discipline and moving back to our interdisciplinary model and again we're midway through but things seem to be progressing very well already talking about what we can do to further scale that is forward or awesome and this is our current cohort so again representative of the different schools different years different levels of experience career progression and of course different disciplines reflection so what have we what have we learned so far we've learned a lot right um there's many different perspectives the participants but also instructors and that's one of the most exciting things about an incubator is that it's not a bunch of people coming together and agreeing with each other there's some challenging conversations to be had within it all of that is constructive and I think open interventions at all levels is important too so I mean it's it's easy to overlook it but just making your syllabi open is still a significant step you know elsewhere so if you're the way that you teach a course for instance if it doesn't lend itself well to some of these like open pedagogy tools it might but if it doesn't then it's not a bad thing still that your syllabus is open and available for others as well I touched upon the the participant guided stuff as well we think this is really important because again going back down to one of the other ones other points that we make here folks come into the incubator with different levels of experience and knowledge as well and it is often the beginning rather than the journey so allowing folks like freedom to actually dictate different stages of the incubator I think is really really important and acts as a as a guide because ultimately you do want to you do want to provide information and everything else and skills for folks that are real to them and applicable as well and then the support is what is important too I mentioned at the beginning our first cohort we provided small grants well actually afterwards when we're in the stage of collecting feedback we asked them how important was that grant you know was that like a deal breaker for joining the cohort and not one of them said that they would have not taken part you know but for the grant so we haven't provided any funding since folks are happy to take time out of everything else they do teaching research and so on to join this this incubator model thank you for the next part yeah and so to David's point I think one of the most significant outcomes of this work have been the community of practice it says right the people matter more than the resources that there are some people who are now doing open pedagogy as as broadly as you can imagine and they're telling all their peers about it and they've been incredible evangelists and there are other people who said I have a class of 400 people and that all sounds really cool for you all with smaller courses I've opened my syllabus I'm bringing the values into it but I love to have feedback that I can get from other people in the same field or or maybe from different fields in different perspectives as well you've also tried to sort of keep that loop that David showed going where every time we do a new cohort we'll have speakers from the cohort before talk about this was really cool and I'm glad I learned about this and they said to do this but that's a bad idea just to know the moment whatever it's right to get that that authentic feedback it's been really valuable it's been tremendously exciting for me to go from our sort of local this works at one institution to the regional and national model and our great hope is that we will throw out a line and somebody will go like we should really have those conversations across national borders as well getting different perspectives can be different things so we're going to share on the next slide our materials if you want to see what we've been up to those are on osf and we're going to open up the floor for questions now but we we hope this is has something useful for everybody and we can start to continue to build up our community of practice around how does this work in different contexts and in different ways that's a lot of us talking yeah please go back to the slide with the osf link yeah absolutely we will share out all the slides which have all the links as well absolutely thanks for asking yeah pretty that's ultimately owner of the material that you would use is that um um how do you say that is there a community manager for example yeah that's a really good question so the training materials are I think owned by the institution and openly licensed so everybody can use them the teaching materials that are developed by individual faculty generally belong to that faculty member themselves my buddy Josh and I back there have been having a spirited debate about who owns educational materials in different contexts and with my lawyer head on I would love to spend an hour or six hours talking to you about that but the short answer is generally speaking faculty owned the stuff they make and we own the training materials so we can openly license them so everybody can use them great great question thank you other questions can I ask a question yes very interesting program put together it seems to be popular for what you said because you had more applications than most could accept so what's your magic formula because we know that academics don't have that much time I say all right I'm not going for that problem so what was the attractive thing I'm also interested not changed after they completed good question I've been talking a lot do you want to jump in or do you want me to talk a little more so just put you on the spot yeah I mean I think there's a couple of things yeah the answer is David David is the magic I mean I think for those of you attended our session yesterday we you know we've got a well-established sort of OER program old textbook it's been successful over the years you know and there's folks all the time you know who are interested in open in general but for whatever reason they haven't yet made the jump to sort of OER maybe there's not the existing content right there therefore they ask is heavier and so on but they maintain an interest in like the principles of open education so when you talk to them about open pedagogy which in many ways is a further sort of step along suddenly it might be more like applicable when you talk about maybe redesigning redesigning assignment so there's different sort of routes in I think we also I mean it's fair to say we've got a very effective communications network so back to libraries we have our own communications department and don't think it's common across every academic library who do a great job of promoting us helping get the word out really reaching faculty where they are again yeah and another in going back to our textbook we do have that community on campus well-established for many years where we can say to folks hey we're doing open pedagogy incubator you know do you know anyone you know in your department that might be interested would you share a post or a blog post or something else so we have a few things like in our in our favor to that regard as well sorry miss no a question about what they did after um so I think David talked about the open intervention that's sort of meeting people where they are um everybody has done cool stuff and we could do a presentation on any of the people who have participated um each sort of in the context of their discipline but the main thing that they have done is continue to be excited by and inspired this stuff and so when they talk about their work at their conferences they say I did this wacky incubator program and look at this new assignment my students love it or you know I use this new resource and it saved a bunch of money so it's getting people invested in something they're already invested in which is like doing the stuff they dedicated their life to doing well and that's been a message that's really resonating with folks a really good question you have time for one more or should we give the other one more on the note okay if there's one more very quick if not we're happy to hush up I'm excited to see the next presentation okay I think this is perfect I'm gonna get out of here oh you're fine and and that's still defying minutes before you finish yourself okay thank you thank you well thank you everybody for being here and wonderful presentation to you thank you I just want to point out that David and I match uh give me guys my same color shoes so yeah I'm missing the color shirt but so welcome my name is Christina Colton in America they say at Calhoun because no one knows what to do with the silent q you um but I am the instructional design and online learning librarian at Oklahoma State University Libraries um and these are my colleagues if you'd like to introduce yourselves I'm Kathy as the coordinator of open okay state at Oklahoma State University and a geo gym member but I didn't get my penguin but I have penguins and I'm Holly Ryder I'm the director for teaching and learning in our library at OSU so thank you all for being here today um in this session oh and also we have um the slides are linked up there if you want to use a little QR code or go to it um the little URL the link has also been dropped in the discord and twitter everyone um so today we are going to talk about and I hope you forgive me using notes my brain's still recovering from uh I went through cancer treatment the past year so but uh but my uh thank you thank you I appreciate that but my short term memory is a little bit to be desired so um if you so we hope that today you know we want to bring you into this conversation about learning analytics and use in open education resources um we've done that in our teaching and learning practice at the Oklahoma State University Libraries um and specifically with OER and so to do this we're going to kind of couch the conversation in that scholarly and ethical conversation around student data privacy and ethics um we're going to also talk about the path that we took to do this amongst ourselves um and um we'll provide some guidance if you want to do this in your own space and I'm going to interject here in a scripted moment and we uh our program is guided by Catherine Cronin's 2018 uh exploration of open practices yes if you're wondering what informs this intentionally ethical approach yes yes yes we love Catherine okay so first we're going to start framing our conversation uh with some about the learning analytics and student data privacy um so we have a shared understanding of what we're all talking about here and why okay so learning analytics um what is it um so there's all kinds of data as we know that are collected on our students these days um they can't go two steps without having data collected on them um and a lot of it is for good right you know they use it in advising to track where they are on campus it was used a lot during COVID for protection um but um with the evolving so rapidly evolving technology landscape especially since COVID um it's a little bit difficult to keep up with things um and especially when it comes to student safety and their privacy and being able to respect their autonomy um and so uh but the type of data we're going to talk about today is specific to teaching and learning and that's learning analytics um so there's not one accepted definition that people use but we're going to use the one from the first international conference on learning analytics and knowledge happened first so why not um and learning analytics is data about students and their in their learning environment and basically how they're learning what they're using to learn what they're doing there how they're interacting with it and we use it to understand and optimize their learning basically it's for their benefit um so with instructional design using learning analytics in instructional design in OER it can be incredibly beneficial um it's provides information about the pedagogy and about the design choices it can also help us improve over time um the teaching and learning materials that we use most of us have stakeholders also that we report to and we have to show hey here's why this is working and so it can help with program and material efficacy as well so with that in mind um and I want to encourage anyone to just either shout out or if you feel more comfortable raising your hand um if you use any type of learning analytics within your OER right now it can mean a lot of different things um but if you'd like to share with us what you do I'll tell you what we do but only if you tell me first and if you're not that's totally fine I'm glad you're here today I mean I know you guys talked about feedback that you got from people and that's key that's one of the big ways that we get data is literally feedback from the students themselves yeah I'm using the learning analytics to uh test if it is good if the questions the tasks everything in uh the OER is suitable and actual good so if a complete group is making something wrong or uh you know okay I have to adjust it or so it's not always to uh test the student it's also to test the material if it's okay absolutely yeah is it meeting those objectives that you've set for it yeah for sure that's one of the big ways yeah we can say as I did to you um we can check the resources or the practices are good and are the will it uh effective for other people so how we can reuse whether it's yeah I love that she brought up uh being able to is it effective can you reuse it sustainable thing that you can go ahead with right I love that like showing is a sustainable that's a great point thank you anyone else wish okay well thank you guys well for sharing appreciate it um so you know we talked about the good ways that data and learning analytics can be used but there is a lot of also very well documented cases of it not being used for very uh good purposes kind of nefarious if you will um uh there's a variety of laws and policies that are meant to protect students um but a lot of them and they very obviously vary from country to country and I can speak specifically to the US um and not necessarily for everybody that's for sure not for everybody but I will tell you in the US there they leave a lot to be desired in the way of protecting our students in practice um so they also um usually lack required safety measures like a documented do this do this do this um and that's you know there isn't one place we can look and say yes this is exactly what we do um and so I mean for example FERPA in the United States it protects students to a certain degree but there's a ton of leeway and there's actually a lot of loopholes in it that people can get around to harm harm students you know uh when you look at it um you know companies can use that data in a lot of ways that we haven't even thought of yet and that's where a lot of the danger lies is what can it be used for later um additionally since there isn't that one prescribed way of handling data that everyone follows legally or otherwise um it usually comes down to the ethics of it and the ethics conversation runs a spectrum there's not two people who are going to tell you the exact same thing usually I'm not sure there are but but with this all combined we're putting students at risk for a great harm unless we take great care with it um so and especially if it's students from marginalized groups who are definitely at much more of a risk and that's also very well documented so um sadly data practices no matter what they are are never neutral um they aren't without the potential harm and there's numerous documented cases of students being harmed by that collection in use so data breaches um using data against marginalized students or marginalized populations bias in algorithms bias in decision-making based off of data um companies profiting off of student data um and even use of student data without their knowledge or consent um all of this can happen while institutions are fully in compliance with laws um and so if you could click one more yeah Audrey Waters um she says that we have confused surveillance for care when you work for an institution that collects or trades data you're making it easy to surveil people and the stakes are high they're always high for the most vulnerable um and I remember reading this the first time I'm thinking oh my gosh that's true and so I really took her uh words there it's a heart um we all did I speak for us yes um so what do we do with that oh sorry nope you're fine she's got it there we go um what the heck happened that's fine that's fine it's the next so I'll let oh there you say that any conversation any policy any work that you're going to do with learning analytics needs to um also have considerations for student data privacy and student data safety those can't be separated um and special care has to be taken so let's talk about what ethical student data privacy looks like practice so what did we do well well before we get to that we're going to talk about uh you may have heard people say well students don't care anyway they'll they'll just sign up and give their email or their data to anybody that is actually not true there has been a lot of um research especially recently um there's a couple groups who have been doing a ton of research on this um and it is very well documented that students really really care about their data privacy uh what it comes down to is that they just don't know who to ask where to go what to do about it it's very overwhelming if you've ever tried to read a privacy policy it sucks it's not fun um so we took Amy Coller in 2017 she had this challenge and we actually took it to heart um and it informed our practice she said to recognize and deconstruct our perspectives on the relationship of data to our understanding of student learning so taking what we thought we knew what we thought was good and proper or maybe what we learned along the way and say uh wait a minute how's this affecting students what choices do they have how are we helping them with this so that is literally what we did we took apart what we thought we knew and we put it back together um and the ways that we did this as a team our teaching and learning team at the library um was with lots and lots of reading if you've ever tried to wade into the scholarly conversation it is deep and wide and it knows no bans um it does but you know i'm being i'm exaggerating right right right right um no but no you can easily wade into it as well and help some resources at the end but um so we did a lot of listening we did a lot of talking to people from different groups um and just really learning um before we tried to do anything else uh then we went through um each step in the following slides you'll see what we did um but we started figuring out what our values were as a team um and as educators and librarians and instructional designers and uh really going from there then we jumped into that scholarly conversation ourselves around learning analytics student day privacy and ethics um we looked at some different frameworks that are out there at the time of us constructing this there was a handful of really really really excellent ones now there are a lot more so okay there's a lot to look at um but those really informed our practice the codes of practice frameworks for today's um and then we eventually created our own data privacy framework um important to that was asking students to give their input into shape that for us because after all this was about them and for them and i'm gonna so in the united states policy is law and so that's why we're using the word framework we couldn't call it a data policy because we are not in charge of the university nor apparently we're in charge of the library but we are in charge of our own little team but so that's why framework can be substituted I think for understanding policy as you have that work here yeah that was one of the lessons we learned very quickly was do not call this the policy you are instead of representing nobody and nobody can sue you for this because if they do you're my girl so um so step one of that what I mentioned it was to articulate the values that inform our practice and so a lot of times these values are you know included in your mission and vision if you have that your teams your libraries your universities whatever that might be um and for us there were several things that we all valued as a team and individually in our own practice um that we already were incorporating and wanted to incorporate into this as well so you see the ethics of care that's what Kathy mentioned at the beginning uh Katherine Cronin for ethics of care really did inform that we put students in their well-being first um and that is a guiding beacon for us um through our teaching and learning we aim to identify question and combat oppressive practices um so that laboratory and critical practice um for us we wanted to restore and protect the equity of students um and assist students in doing the same for themselves and others um we also seek to incorporate student voice and experience and shape our practice with it um and as Collier put it we want to provide digital sanctuaries for students or digital spaces that are safe for them to be in and that they feel safe to be in where we're actively protecting them and they know that they can go there safely so that ethical scholarly stance once we waited into that little conversation there um uh it could take up years of sessions really to talk about all of it i'm not going to say years because that sounds overwhelming but days of sessions to talk about what the spectrum is out there um of the ethics of student data privacy but i'll tell you what we um included in ours um and at the end there are a bunch of references that you can go to if you'd like to um to get you started on that um but in a nutshell we believe student data privacy should be prioritized to protect students from harm to respect their autonomy um empower them and establish equity provide protections for their intellectual freedom build a trust relationship between students in the library and as part of specific duties as librarians to enable students literacy with data privacy um and so this was really key to us to guiding excuse me how we would be able to use or collect their data so when it comes to learning analytics and OER there is an additional layer there of ethical complexity if you will um we are striving to provide a free or reduced cost to materials and that shouldn't introduce those financial barriers to education um and so we shouldn't be exchanging that for uh their data being used in some way um if we're saying free or reduced cost we're going to say free or reduced cost to mean it um so we say we do say free right free yes our uh yeah materials that we offer are free yeah um thank you Kathy um yeah because usually when you say free that equates to commodification of data like you are the product right if you give your email for something you know free uh burger king a free burger right burger king they've just sold your data so um so in offering OER we committed to doing so without that exchange of goods um and so we believe it's a public good and it shouldn't be commodified so uh deciphering paradigms I mentioned earlier that um that lack of standards um for student data use so in response to that a number of institutions and committees and organizations sought to create some ethical frameworks for using this for using student data and like I mentioned at the time that we started doing it um there was a number of these exemplars um it was much smaller than it is now but now it's really a lot of institutions have them posted um on their pages about privacy um and so we're going to share these are the ones that initially formed our practice um very appreciative to the open university uh jist uh NISO um the Stanford Carroll and Ithaca SR project um UC Berkeley and UC or sorry UC University of Guaya at Manila um and one specific concept was incredibly influential to us um and it helped us set some ethical parameters that really um kind of tied everything together um and this is the concept of info information fiduciaries um and it was originally introduced by Balkan and then um Jones Rebell and Leclerc um uh added it for a higher ed excuse me specified it for higher education institutions and this is a concept that explores how um that when we collect and use data that we're these information fiduciaries we're responsible for it and we should act um ethically because we have the onus to protect students um and their data collection so we use this concept basically as a checklist for data use parameters for ourselves um and to only collect data that directly benefits students and their educational experiences supports our university and library mission vision and goals honors and protects students intellectual freedom and creates and upholds a trust bond between us and the students one of the things that you'll find as you wait into this conversation is those ethical things that are at stake and some of the big ones are students autonomy and their intellectual freedom is it being protected is it being sold in some way are you allowing them to create and be who they are through these materials or is there some mechanism that's watching them um so so once these values and ethics were set for us we um and we had those previous paradigms that we looked at um we started creating our own framework and as we mentioned it is a framework and not your policy so we came up with some core principles that was the first thing we decided to do was what are our core principles that will guide their use and you can see here responsibility transparency privacy and consent confidentiality and security and access um and we thought that these best captured our values and ethics that we decided on um as well as the paradigms that we valued um each of these principles lays out our specific parameters and practices for data collection use analysis storage um and as well student choice with their data um we have a longer document that is um still in the editing phase but if any student were to ask for it they would absolutely have access to it that literally lays out all of these things for every piece of learning analytics that we collect um so they can say oh that's where it's going that's how it's being used so okay so we set a precedent to only collect for learning design improvement um showing the efficacy of our program or if we had to share it with instructors of record um and we always first defined whenever we encountered a new learning analytics something um we were going to define the objectives of that collection first um and this just goes over how we did uh what we did to add it when we put the policy in place but we could skip that so it's very much the opposite of collect everything you can just in case you need it later do not do that yes thank you um student voice was incorporated holly taught a um three credit credit class for a couple semesters and every semester oh no she taught it for a lot of semesters but over yeah so over a few semesters of those six years um she would ask students to review it when she was teaching about data privacy and they would give feedback and we shaped it according to what they said and we will likely do that again um very soon probably in a couple of weeks yeah um some pitfalls I already mentioned that being able to call it you know a policy versus a framework you know what's outside of your control the time it takes to do this process is a pitfall um buy in beyond our team we actually got a lot of pushback from some people who have a foot in the student data that we're collecting but didn't really want to be involved with what we were doing um and then google and amazon they're just everywhere you can't get around them so our way around that was telling students how to avoid them um but yeah and acknowledging where they are part of that collection process yeah exactly um and so now we have our guidelines posted it's you can link to it it's also posted our open okay state um platform um we use xapi and grass blade lrs to collect those learning analytics um the only thing it does collect that I cannot get rid of at the moment is their ip address but as soon as it's collected I delete it forever um and we have standards in place for how to protect that data um we also have reusable analytics in our press books platform um but we do notify students about how they can get rid of that uh or go around it um I uh there's an okay state data literacy project that just came up we were so excited about it this past semester where they've involved us in being able to tell students on campus here's how data everywhere on campus is used in the library was able to share what we do um and I'll also be holding in a workshop uh this summer semester about student data privacy that came from the data file classroom research project I encourage you to check it out they will be publishing those materials soon I'm piloting it for them so it's not quite up yet um the rest of these are about how to do this on your own which is basically what I just said and then there's steps for it so holly if you just want to put them on the screen and then uh we could take some questions well what's up there I don't want to keep you lunch is important so that's the end are there time for questions that's yeah yeah questions yeah ask you some questions this is our first conference yes first of all I would like thank you very much uh I would like to ask what is the difference between learning analytics in the general student data that university collecting and oh yeah what is the special things here about the OER because you put it I know that we are the way our conference is there a difference in if the framework should be different and second is if students like to do like up out do you do like opt in or opt out those are the way the two are enabling or descending the collection of the data I'm gonna remember I'm going to start with the opt out question because that's what I remember so with opt out what we do is in that document I talked about where it talks about all the each individual type of learning analytics whether it's a tutorial open oak estate platform there's a section for each one that says here's how you can opt out and so for instance if it's google analytics there's a browser which is so backwards to me there's a browser plugin you have to put in a google plugin you have to put in in order to avoid google anyway um but they can also uh you know email us that they have an issue with it um and so there are different options depending on what it is um that they're talking or that they're using um we have holly's email up on there so if they ever have any questions they can go talk to her email her we haven't had anybody no and you would know because if they emailed me it would go directly to you it would be forwarded to Christina saying please help me answer that right right um and you said how okay so the first one was how is this different google analytics is aggregated right so you don't get it in the individual student level right yeah i'm not worried about this is a good way analytics just to communicate with the faculty and say you've had this many hits on your book someone's using it you know on this continent and things like that but you're not collecting individual level data no no because they don't sign into our books we there are books are right now are completely open uh so it's a faculty member has h5p in there or something and they they can link it through their canvas and so it's particularly their canvas and then yeah but then the canvas will collect the data right then because it's the idea is that you i have so many problems with learning on analytics because i like to yeah but they don't allow me to use this for research purposes so i think the framework that you you present is very very smart very good but i think that there are so many questions that you will have to deal with along the way and one of it is the idea of the individual level data that the systems are collecting and i think the students don't care about it by the way because they have those machines that collect data all the time and we have now microphones collecting data now and we'll ask us and we don't care about it right so i think the students don't care about it but we are giving more too much attention but if you think that it's they care about them it's important uh there's so much research now that tells us they do care so much when you ask someone do like that so we like that but it's you raise your hands but but it's another discussion but i would like to ask so because the systems are collecting all the time data and the question now if we want to analyze that because most of the data is available if we are enabling some of our constant agreement so that's how we deal with that yeah it's very go ahead Kathy we can't use student data for research purposes without going through the institutional review board process so there'd already be we would already have to be pushing buttons there's like things to bring that situation about right yeah so we're not really losing anything by not collecting it up front because we can't use it anyway without permission so that's a good question that going back to our ethics of care too like we always put you know that's the first thing we start with always is how our students be impacted by you know and so that kind of drives everything we do and if we were to do research there would be different hoops we would jump through to make sure i'm gonna jump in and just say that our time is up so if you are really need to go to lunch we'll not be offended we have to leave to go get food if you want to stay we'll hang out for a few minutes and talk and answer questions yes because again i think it's okay