 So thank you so much for joining us in this session. It's great to see all of you here We know that you are actually in greater number than it looks like looking out at you But at some point we're actually hoping for some conversation and discussion So if you decide you'd like to migrate towards the center towards the mics, that would be great I'm Lisa Hinchliffe. I'm the coordinator for information literacy services and instruction as well as the coordinator for strategic planning at the University of Illinois Library at Urbana, Champaign And I am very excited to be working with Andrew Escher. I'm the assessment librarian at for the Indiana University Bloomington libraries And we're here together Engaging on this topic of analytics and privacy Coming to it from sort of two different perspectives But that have led us to be in some really engaged conversations over the past couple years Around negotiating the boundaries between the value that we're trying to create as libraries and the value that we have on Privacy and some of you were perhaps in the last session on privacy the first session after the keynote also in this room Which raised some of these same very important issues and ideas I myself come to this topic thinking deeply about it because of my work when I was a serial president on Creating the value of academic libraries initiative where we came to understand that with the accountability that libraries are under We need to be able to tell our story of the impact that we have on the users that we have as Well as the importance of using data in order to improve our services so we can have greater impact Be more efficient in our operations and the like and one of the things that we really put forward in that report and in That initiative was the importance of data collection and honestly beginning data collection at the individual user level and so Quite contrary to some of our historic practices of aggressively Discarding data about our users and what they're doing what I've learned over the past four years as a result is that's going to take Some conversation about whether we want to do this and if we want to do it How are we going to do this and I found Andrew to be a great partner in this conversation because he well We're probably both in the middle of the spectrum on this. He might be on the other side So what we're present today really builds on a lot of questions that began last August at the ARL assessment conference, and I'm sure many of you were there about the tension between Using this data and linking this data up with linking library up with institutional data and protections on user privacy and also things like thinking through the risks to our individual users and Thinking through issues of consent of the people that were gathering this data from My background is an anthropology. I come into libraries from the social sciences side And so I've I've been thinking through this data, especially in relation to Human subjects research and the ethics of human subject research And so I'm coming at it from from a little different perspective Then Lisa and really building on that background I have in social science research and thinking about if I were designing data collection For a anthropological or sociological protocol. What would be the things I would think through for these types of data items So thinking through this Andrew often reminds me about the importance of privacy and I often remind him about the importance of developing excellent services So hopefully over time we'll be able to come up with a framework that lets us do both As was well demonstrated not too long ago in this very room Users leave a trail of data Throughout our library systems and with the work that they do elsewhere on campus and on the free open internet Many also of our campuses are aggressively moving towards collecting data about students and their learning analytics or their Their activities as students and they are capturing that data and collecting that data Intending hopefully to make a better undergraduate learning experience for our students So one of the questions that we should that we need to think about with this is how will we connect our library data with that campus user data? Do we want to does the campus want us to are they asking us to all of those sorts of questions? but it's sort of a given that people are leaving a trail of data and even without the You know extensive things that we saw earlier today with wire shark and the cookie sniffer and some of those other Technologies users are still leaving trails of data May or may not be super personally identifiable But as much as we might expunge for example circulation records while they have the book checked out There is a record that they have it checked out and many of us Have any number of other places in our libraries that we're collecting data Even not on the network services though clearly the network services are collecting data And I really appreciated the I think it was a library director who came up in the last session and said okay So what should I do about this? Like should I just give up on privacy? Should I keep going with what I've been doing which is? Restricting the development of services because we're trying to protect privacy really that's the question I think we're grappling with today Which is how do we balance and I'm going to pull out two components of the ALA code of ethics It is absolutely the case that the ALA code of ethics has a code statement around privacy Which says that we protect each library users right to privacy and confidentiality With respect to the information sought and received resources consulted borrowed acquired or transmitted But in this very same statement code of ethics It also says that we provide the highest level of service to all library users through appropriate and usefully organized resources Equitable service policies equitable access and accurate unbiased and courteous response to all requests So really while we've grappled I think and come to see some of the challenges to the privacy statement in the code of ethics One might begin to ask ourselves the question of whether all of this data Whether we actually have an ethical obligation to put that data to use in the development of this highest level of service to our library users in addition I would point out that While we're obligated to protect library users right to privacy at least this document Does not provide any guidance on how a library should respond to a user who themselves? seeks to disclose information about themselves and so while we protect There's there's an open question on how much libraries should enable sharing if a user wants to Or whether we sort of continue in a sort of paternal role saying oh no We won't let you so there's a lot of questions here even with if we're protecting data And we have data about how it plays in with user agency as well as our service development So as Andrew and I talked through this We came up with five assertions and the next slide is why you might want the handout because it's kind of a wall of text So if you don't have one there in the back on yellow sheets of paper It's the same text that's on the slide, but we just thought it might be easier to look at it in front of you So we have five assertions that we'd like to put forward That given the increased emphasis on analytics in higher education. It is not a matter of If libraries will participate it is a matter of how our campuses themselves are collecting data about library use Inadvertently or advert or on purpose and so other people have data about what Users are doing in our libraries because those logs are going through campus servers They're going through the network traffic and the like so one of the questions We might want to ask ourselves is if this data is being collected How do we want to engage with it or do we want to leave other people to engage with it? We also will assert that library user data can be useful in the development of high quality services one of the areas that is very clear that people like the customized experience of a lot of the web tools that they use Where Google begins to learn you and say well This is the kinds of things you're looking for it the ability to connect your own ever note To Google so that it's searching across your personal collection as well as the Google corpus These sorts of things are only possible with data that connects different systems together We also point out that libraries already collect identifiable user data and enable collection of that data by third parties All of our vendors are actually collecting data on all of the searches that our libraries or users are doing in their systems They use that data to improve their tools and products. I hope I presume We might want to ask ourselves Whether we're benefiting from that data in the same way We also point out that to meaningfully relate library use data to other institutional data Requires user identifiers So sometimes librarians talk about like well, we'll collect it in the abstract and then we'll sort of say well We'll take this aggregate data about students and we'll connect it to institutional aggregate data You really can't get Analysis done on that level that's going to help us develop our information systems And then the final thing that we do have to admit is that it's really difficult to predict future uses of created data sets So it's possible today to say well, this doesn't seem like so bad or Alternatively, I don't know should we collect this or not in the future This cuts both ways data could be collected that could turn out to be harmful to people if it's released On the other hand if we don't collect that data It may be harmful for us to develop the kinds of services that we want to but I would kind of go further here and say that if we are collecting the data and Presumably we're securing it and encrypting it and doing all those good things We probably have an ethical obligation to use the data and Andrew will talk a little bit more about that in this next part So but before we go on we wanted to give people a chance to say These assertions seem wrong to me. They seem right to me. They seem incomplete questions comments Okay, we're gonna go forward with silent equals consent, which really wouldn't work for IRB, but it'll work for us right here so Thank you So I'm going to talk through Some library use data practices in the hopes of starting a broader discussion And I'm framing these as related to especially the ongoing data collection that we do in our libraries as a matter of our day-to-day operations Kind of bracketing out things that we might do for specific projects So here we're really thinking about a framework that would apply to the everyday data collection that we're doing and the assumption that This library data could be connected to other institutional data things like student achievement data financial aid data GPA Engagement and those sorts of things they're becoming more and more important as we think about Student and educational analytics I come at this from the side that we have an ethical responsibility To our users to think through the risks and benefits of any data that we're collecting about them So I'm a bit more conservative from where I'm coming from Than Lisa is and here I would like to talk through especially Thinking about principles about data collection and usage rather than mechanisms for the sake of this conversation So we've developed seven Recommendations and again, I refer you to your handout because this will be quite difficult to see On on the single screen But we wanted to get it all in one place for reference and I'll just talk through each of these In turn and and develop them a little more before we move into our discussion The first is that libraries should regularly undertake a privacy and data collection audit Of their systems as well as their procedures within the library and play particular attention to the levels of risks Presented by these kind of data. This is something. I think we really need to think about doing regularly and making a regular part of our library operations Institutions very pretty widely and how they approach this, but I think it's safe to say that many Institutions including my own have not done this as recently as we should have and don't do this as regular as we should Second the data collected should be aggregated a level that balances analytical specific Specificity with user privacy. So we should think about what level of aggregation we actually need Before we collect data because we can use aggregation as a strategy To help de-identify data and help protect user privacy. So for example, we could collect things at the level of Students in particular programs rather than the level of individual students So we should think very carefully about the level of specific specificity we actually need I'm third and I'll talk about this one a bit in a bit more detail is this issue of transaction level data And I think I really think that we should be very very careful about collecting data that identifies both a Specific item or a specific resource and specific users, and I don't think we should collect this Systematically, but we should only do this if we have a good reason. We have a specific purpose And I'll talk a little bit more about why And if we do collect this data, we should develop measures such as local encryption of files and separating identifying keys from demographic data sets in order to further Protect the individuals contained in these data sets and the reason I worry about this particular data Specifically is because I think it's inherently risky If we think about what a data set of that looks like We could potentially collect every search of every user and every resource used or viewed of every researcher in our libraries So at my library that would be 40 to 50,000 people per year and we can do it in perpetuity So it's a really Really large and detailed data set And we really don't know what that data could be used for This kind of data once it exists Is subject to things like subpoenas for any of you who've been following the Boston College Research on the troubles in Ireland a data set was learned to exist and this was a a Interview data set, but it was learned to exist by authorities and then it was subpoenaed So once we create these kinds of data sets, it really opens us up to the potential for for a lot of for exploration by law enforcement And various other things that we may or may not have the ability to defend Once we once we get the subpoena or the national security level letter in some cases and In many cases our privacy policies already Expositely forbid this kind of data collection and that Indiana our Privacy policy does forbid this data collection Even though we have the ability and as we've seen earlier today many of our resources may be logging this data Behind the scenes in a way that that could potentially be analyzed or data mined Fourth data sets containing user demographic data should be destroyed after reasonable time period and this is especially to To prevent things like re-identification analysis, which has become easier and easier And to protect the long-term interests of our users so we don't have this data sitting around So can be used in you know, whatever whatever ways it wasn't intended to at the beginning Fourth we need to develop new consent procedures And we need to review the ones we have and provide more explicit opt-out or opt-in for This type of data collection. I think many of our privacy policies aren't sufficient Given the the new risk of this kind of data And also they're not sufficient because there is no effective alternative for students And thinking through from Institution review board or a human subject perspective. This really wouldn't be Adequate an adequate form of consent because there is no alternative. It's a form of coercion If we're thinking through it from a Sociological research point of view It's also problematic because no one reads our privacy statements. And so the consent that we're gathering by The tacit usage is an informed consent. So it's really problematic Sixth library should hold vendors to the same data analysis and retention standards that we hold ourselves to and we shouldn't purchase Or otherwise use data from these vendors that doesn't meet our ethical standards So if we decide that our this type of data shouldn't be collected and used We shouldn't then go out and purchase it from someone else So this is the issue of the the data will be either collected by us or others will collect it about us And then we'll wind up using it anyway. I mean, we should really make sure that we review those contracts and policies very very carefully And then finally libraries should advocate for their institutions to develop And adopt a code of practice for data related to learning analytics And this is not just in libraries, but across the institution In the absence of institutional codes, we should think about developing our own and this is really what's motivating this Conversation today and in this framework is we want to start this conversation with you And we'll be carrying this forward. We're planning a follow-up presentation at ala on this after we have about six months of conversation Thinking through what might this code of practice or code of ethics look like and so we'll open it up to discussion Good Peter. I knew I could count on you Oops Not yet, but Todd's got one in the back now. Oh now it's on Peter McDonald California State University Fresno Lisa and I have served on the ACRL values committee and talked a lot about this topic Couple of comments and then maybe a question or two. I I Really well first this CSU the largest university system in the United States has had a white paper approved by the chancellor's office that Essentially codifies this in different languages and provides cover for any university in the system to Expand swipe technology from Circulation to any site in the library And What the basis of that document says is that? We are interested that a student used a service or an instruction module or a Whatever not what they were after so when they come to the reference desk That they came we don't believe it's a privacy issue What they talked about we consider a privacy issue and that's both a statement and a question I'd like your opinion on that. That's how we vetted it and Two universities have done have opened up swipe To every aspect of library service and that's Fresno State and Cal Poly both in the CSU system and Of the thousands of swipes We have it is statistically negligible that a single student has raised a red flag So one in a thousand So so why am I swiping the rest swipe and swipe is our method of getting that information beyond the circulation desk So I guess a couple of questions. There are products now like Tableau which are dashboard data products that universities are buying to do and They regularly display student data with grades and GPA and so forth. So there's a real move amongst the vendors to Make this data pretty Leaky I'm not aware of the CSU of a single instance of breach of student Privacy data anywhere in the system and for as long as I've known this system and California has a state law Where it is actually codified the law that library records are private So I guess two questions one. Do you really feel that that comment? It's That they came that we're trying to get at which is your number three in I think And not what I mean the fact they may have asked this question of that question We're trying to aggregate that these students came and the swipe lets you know information about their ethnicity maybe or whatever if you correlate it to people's off, but it's Their privacy is protected because we're not capturing the what and I'd like Given the students swipe day and night and reveal their lives every minute of the day on social media You know what brawl does vendors have been protecting that data because I see on Tableau which we have at Fresno State Wow pop right in and anybody can seem to get to Student data. I can't get to people soft my circulation staff can but Anyway Were those two questions? So one of the things I guess I would resist even with my advocacy that we collect and use the data for decision-making is any argument that says Students or users don't care I don't think that's a very compelling argument for collecting data Not the least of which is it is it is it takes resources to collect and analyze data so And I think that So that that to me is sort of like a non-starter whether people care or not As far as whether we should do something or not or and whether we should give up on our value on privacy I think one of the things we want to distinguish between is This whole question of data that is then aggregated and shared out to show patterns Which is what Tableau is doing versus the individual students record Which I believe typically would not be displayed as an individual record even without an identifier on it But I think one of the other things that you said in there is an interesting question that we need to grapple with so I personally Would say it's sure a lot better to know that they came to the reference desk Then to not know it at all But I could pause at any number of questions that I might like to be able to interrogate Particularly if I can connect that with learning analytics that might be interesting that that the fact that they came won't Sufficiently answer so it's absolutely I mean sure you can now answer more questions about the learner by knowing that they came But there's a pretty big difference between they came and asked me about something for a class And that's even still an egg. That's still a that's not it You know transaction or they came and asked me about something that's a personal interest Still not recording what was asked, but at least into a category of type of thing that was asked But even more so what if it's which class right so now so there's layers here that I think you have to Untangle and say which of these things is the institution Trying to figure out in most cases I think our institutions are trying to figure out how to have students be successful in a class and then in completion So retention and completion so you've got two dimensions once you add in the in a class dimension knowing that they came to the desk is probably It's gonna help but it's not gonna ask all the answer all those kinds of questions that we wanted but now Andrew All the things you'd like to not collect Yeah, well you bring it you bring up a really a number of really interesting issues the first is Thinking about things like Tableau, I think Tableau has a really specific problem with its structure In that you can't turn off the background data when you present it and that that's a big issue and I'm a Little surprised that people haven't thought about the FERPA implications of having things like great point average in those I mean, maybe maybe it's if it's locked down with in the institution. That's kind of the solution We've had IU as long as it's internal to the institution We consider everyone to to have access to a certain number of records But the potential for leakage of one of those data sets is really really high And like we were discussing in this morning. I think once we start creating those kinds of data sets The issue isn't whether or not there will be information getting out But how bad and when? And I think that that Causes me to want to think very very hard about the the data sets I keep in the long term and where they're stored And what information is in them And that has to do both with the tools and with the data set itself I Think I broadly agree with what Lisa was saying in terms of the the aggregation. It's certainly wonderful to know If the students it's better to know That the students are there and broadly what they're using that to not know at all But then we have to think how much further do we want to go with that because the follow-up questions that I always get when we start talking about those is Which resources are they using because that's a collection and development issue which departments that are in because then we can staff the reference dust better and so we can start getting pretty fine-grained about The demographic things that go into that and then we can it would be interesting to know Every resource that a student would use for particular courses, especially some big courses that we have repeated every year because that has interesting Interesting implications in a lot of areas of our library and so as we start doing this There's there's more and more push to keep this kind of data and to keep it in the long term And to get into this transaction level data And it it starts getting pushed against more and more and more and so that's part of part of what we're trying to do is to is to Hopefully get out in front of some of these questions or we're talking about at our at our institutions as we start to combine up These types of data sets is to think about okay maybe The risk of having this information outweighs the benefit Because I'm a real firm believer in that in that just because we can collect this kind of data Doesn't make it ethical to do so and there's a lot of literature out talking about big data broadly outside of just the educational context About this kind of ethical decision just because people create this data about themselves Doesn't necessarily mean as researchers as ethical for us to use it even if we can just go grab it Even if it's public even who's on Facebook And I think we really have an obligation to think about that especially from an institutional institutional standpoint trying to challenge that by Putting in place an international dynamic, and I'm thinking particularly about your point six and This notion of vendors being held to the same standards and we shouldn't deal with vendors who don't do what we want the ALA is I think the fifth I think it's the fifth professional body in our field that I've been a member of in four different countries And I've just been checking and every one of those bodies has a different set of ethical standards and codes of conduct Sometimes in this particular area almost contradictory. I'm conscious also that in the different countries in which I've worked the data protection type legislation theories and Again things that are possible in one country may not be possible in another and Finally, I'm conscious that many of the vendors to which you refer are operating in a global marketplace Is it reasonable to expect them to? suit every need or are we going to try and impose a world view that Plays to the nabless possible view of what constitutes ethical practice just to be provocative So I think part of you know that the unsettled Thing in number six is we say hold them to the same standards But we haven't really said what those standards should be except putting out here that we should begin to develop this and Sort of the same tension that we have around well what level of data should we collect? We could also have that tension in what how detailed should a set of standards in this this or best practices in this area be So we could have something that says thou shalt never collect data about an individual person ever without their absolute consent Or we could have something that says There should be policies developed around the collection of individual data and how it is shared And so it might be quite reasonable I think to to assert to every vendor that you should have policies around this that you will share with us That we can then make an assessment about whether or not it fits within our understanding of what's acceptable It may not be a reasonable thing to say that every vendor should have to comply with every single library's interpretation of how these things would play out because I think it's also quite likely that Any given set of libraries that we look at will find the point in this in a different place as far as balance between service Development and tension and the tension around privacy But I think that is a it sort of asks the question in some ways of what level do we need to determine this kind of standard or best practice It's a great point My comment just about students don't care. It's really that this is really important because it's so easy To say well, you know the state is being gathered everywhere and so forth. So I really took your comment So if you getting a little beyond whether We should be capturing the state or not and how we do it and and the ethical standards If you use You know, you've been on the values committee and you know, we're sort of hitting some brick walls And certainly if you did the poster sessions at the assessment in action at ALA You realize there are hundreds of libraries now doing assessment and they're all coming up against this problem that You're you're not like a medical trial Taking a hundred people who have the placebo and a hundred who get the cancer pill and you can see which ones and Are so I'm gonna pose an ethical question. So it's an embedded librarian in Gen Ed 101 English 200 students What's the ethics of giving full library service to a hundred students and Not doing squat for the other hundred to see whether that hundred That got the service actually does better because I think there are a lot of people grappling with that because that actually is a sort of Methodologically more sound type of study speaking to what Lisa said that it just aggregates don't necessarily get you much So I'm just curious on your thought So first we'll resist anything that says that it's methodologically more sound to use scientific controlled research Experimental trials when it comes to other kinds of questions. We might be asking about the world Anthropological ethnographic qualitative can all be very methodologically sound whether or not they meet our sense of what is scientific And sort of an abstracted sort of way is a different question But I'm quite confident that Andrew will be able to just pull out a bunch of stuff here about the ethics of denial of service Yeah, so you're hitting up you're hitting up against Another issue that we didn't really talk about too much yet It's whether or not this type of research falls under the auspices of our ethical review boards so I'm fairly certain that the the study you just described is kind of a Big example Would be interpreted as a human subject research at my institution And would then need to be evaluated by the institution review board, which would probably say that it was not ethical to deny students Services in order to to have an experiment But they would certainly review it and there would be a negotiation process I don't I think That kind of research isn't The reviewed research like that is is not necessarily problematic. We just need to think through the As we would for any other type of human subject research. What are the risk and benefits and what are the the potential problems with it? But then there's all this other Data and research that we're doing that sort of falls under Is sort of exempted from Institutional review because it's it's data that's collected its data is collected like student demographic data So it's automatically collected. It's used for improving the institutions the things that fall under the automatic exemptions The IRB and now we're really hitting up against issues that cross over into these two different areas and it's really hard to Find the right institutional place to have some of these discussions because I couldn't go to my RB IRB and say okay I'm collecting this site. I'm I've decided that we should collect this data Here's what I think the risk and benefits are and I'm going to do it in perpetuity because the IRB isn't set up to address Very well those kind of things these kind of institutional data collection things And so that's one kind of weak point in this So I kind of yeah, so I think the other thing that I would add to this is This whole so you mentioned assessment in action. I appreciate you mentioning that Peter because as one of the co-leaders of that I'm struggling I'm I'm there for all 75 institutions last year and all 75 institutions this year that are struggling through this question Which is what is the right sort of research method or assessment method to learning what? We want to learn in and so part of what becomes crucial here is this sort of tension as well between Proving our value, you know, we went in we taught this session look everyone learned and therefore now they're better human beings that kind of Claim that we'd like to make And sort of the the other version of assessment, which is this is diagnostic. It helps us plan future interventions It's not necessarily about making your budget request, right? So untangling these two purposes is really important as well But there are I think also just for you know with respect to your particular question One of the things that we could just do so much better at that we that doesn't even have to push us towards the Let's see what happens if we deny humans the services that we've promised them It would be things like we are not even good at knowing when we design our research studies Typically whether the student actually received the instruction that we're claiming had an impact So if you look at most studies for example of library instruction They never actually take attendance to determine whether the students who you intended to impact were actually present Foreset instruction so at best what we're actually able to claim is a student who was enrolled in a class in which the faculty members scheduled a session Did it have an impact? Well, I don't think any of us actually think that there's an impact of just being enrolled in a class that the Faculty member happens to have instruction in right so we've got this sort of need to really get a lot more Specificity in what we're really studying which I think is where IRB processes can sometimes help us with this The other thing I just think is worth mentioning about IRB is a lot of times people like to say oh IRB requires us to X Y or Z They don't require anonymity. They they don't even always require confidentiality So it's not IRB that's going to regulate sort of our decisions ultimately on privacy So in some ways, I think that's also one way another thing We don't want to go to is sort of saying well, we'll let the IRB decide so Todd This is a Todd carpenter with nice. So this is a fascinating conversation. I really appreciate you Kind of advancing it. Thank you the Thinking about this from the perspective of vendors and some of the services that they provide Thinking here about some of the index discovery services some of the altmetrics analysis that is provided a Number of these systems actually require data and data analytics in order to function, right? And so, I mean, how could you how could a system like say X led versus BX system? work if you Adhere to some of these policies And are you then explicitly saying that? Shouldn't be using recommender services based on user behavior So which one of these do you think says you couldn't do the things that are necessary for BX? I'm just just to help focus the discussion number six Because it says they should adhere to our same standards. Well, certainly six, okay By which you're saying that there's already an assumption that collecting this data isn't okay Right, but also four Because who's to say what a reasonable time is based on, you know Well, that was actually our wiggle room for getting out of this You know, a reasonable time is the next time the user wants to search for that information I'm not sure Again transactional level data that identifies the user right and specific item is also You're gonna run into problems there. Yeah, well unless required for a specific or limited No, I mean I take your point I take your point I mean, this is exactly the question Do we want to be able to have services like BX from recommender and I just that's one Yeah, I think there are other systems that use that data in less obvious ways correct that Enhance those those systems and it's part of the special sauce that you'll never hear about right So I'm wondering how a vendor would be able to comply with some of these these policies your recommendation I think these are good questions and Partially putting out these seven things as a matter of saying okay if these aren't the seven things What are the seven things? Where do we want to land with this and I guess I'll make it a little bit more? Sort of complex for us, which is from what I see at least in the science community They're perfectly happy to be going ahead and developing all kinds of tools and resources that historically We would have thought as library tools and resources just having attended shaking it up 14 via virtual streaming All kinds of services that are based on use data that the libraries are not engaging with and so this also becomes part Of the question and so maybe these seven are too restrictive There's a rebuttal or an agreement. No, actually just it raised in my mind some of the other related services where patrons are participating and willingly giving their information for like Mendeley and howdy's use of systems like that where They are intentionally providing information to a third party To get access to things right And intentionally sharing that data with their friends colleagues, etc. Yeah So and that's I think that question too of the degree to which the Mendeley exists whether the library like wants it to or not Right. It's out there people can use it So do we say but we will have nothing to do with this tool because of those things or do we say this is an important Tool clearly our users have said and not only should we help them use it help them use it better We should actually maybe start thinking about how we can use data from Mendeley to drive other services that we offer in the library So we're actually going to be a consumer of That shared data in some way in order to drive forward our other services So I just say a couple things And I'll maybe I'll get a little more radical for a second So I think we should think about how we can opt in people to those kind of services rather than just gathering Their data and or forcing them to figure out how to opt out Second I just as an example I killed my own Mendeley account because I no longer thought once it got acquired by Elsevier I could trust its privacy protections and I think number six was was probably driven a little more by a Feeling especially that I have that library should be willing to use the power of the purse to get our vendors to do what we want them to do and so When we should say we're willing not to buy something because we don't think it it follows the kind of ethical data collection standards that we want and Here I'm thinking of things like ebooks And other things we should be willing to just say no, we're not going to buy it and Then finally, I think we have to think about whether or not we're willing to kill services because we don't think that they're ethically viable and and That's a that's something we can choose to do or choose not to do and I think People will have different opinions on that I may be a little further further down the road than Lisa is but Just not to cite any specific examples or specific things But I think those are those are hard discussions that we need to think about having In a much more critical way, and I think our our conversation Has not caught up with the level of data collection and the services and the tools that Where those things are right now our conversation isn't where it needs to be because the tools are outrunning us And you know and this is a funny thing because I might also say that we also shouldn't buy things when the vendors won't Share data with us that we want so we might have a similar like Neither of us will buy this tool But Andrew won't buy it because the data is collected and I won't buy it because they won't give me the data I think this has been a I hope it's been thought-provoking for you. I know it has been for me I already want to pull up this file and revise some of the text Andrew and I would love to hear from any of you That have suggestions additions challenges criticisms and the like but for now I think we should take move the conversation to a more convivial setting of the conference reception. So thank you