 21st Century Scholar and Researcher Services, remarks by Kathleen Fitzpatrick, Myron Gutman, and John Unsworth at the 161st ARL membership meeting, convened by Harriet Hamasi. Please join me in welcoming Kathleen Fitzpatrick, Director of Scholarly Communications of the Modern Language Association, and Professor of Media Studies, on leave from Pomona College. Myron Gutman, Assistant Director of the NSF and Head of the Directorate for the Social, Behavioral, and Economic Sciences, and Former Head of ICPSR, Myron is also a faculty member at the University of Michigan. John Unsworth, Vice Provost for Library and Technology Services, and Chief Information Officer at Brandeis, formerly Dean of the Graduate School of Library and Information Science at the University of Illinois, Urbana-Champaign. Welcome to each of you. I want to begin with a question to Myron. One of the central themes of your professional life has been explaining and developing the fundamental role of research infrastructure. Please help us understand how you see that fundamental role, and perhaps even talk a bit about how the government sees our views, information. Well, thank you for that small question. I'll do my best. I hope the rest of the group will chime in. You know, I think that what infrastructure, scientific or research infrastructure does for the whole community is allows those who are doing research or contemplating research to get the job done as efficiently as possible and to open up a transformative view of what they're doing, whether it be in science or literature or behavior. So we task those who provide infrastructure with making available resources in general that can be shared and to do so in ways that are both readily available and flexible and easy to use. So that's a big challenge to do, to have that kind of infrastructure. NSF's traditional infrastructure that we support at the foundation, of course, a lot of people see in the really big things, the big enormous research site at the South Pole that's shared, the telescopes in Chile and on the mountaintop in Hawaii, the light source at Cornell, but it's really more than that and it's increasingly more than that. The ships that we operate together with scientists and the aircraft that we operate in terms of understanding the atmosphere. Increasingly, the infrastructure world is one that is both those physical things that people use and the digital information that flows from them and we see that the big project of the next decade in the astronomy world is the large synoptic survey telescope which is gonna produce many petabytes of data a day from a mountaintop in Chile that somehow astronomers are gonna be able to use in near real time. There the product is not as much the site as it is the bit flow and we think very much about how to manage that. What's interesting to me as we look forward I think is the question of how are we going to integrate those bits of infrastructure? We've had a fair amount of that discussion of that and many of you have people on your campuses who have contributed to what we call the EarthQ project which is a project launched by the Office of Cyber Infrastructure and our Geosciences Directorate to try to come to a synthetic way of understanding data about our environment. A similar project that sort of lies aside that in some ways is the NEON project. From my perspective as a social scientist and someone who's worked in the social sciences and humanities all of my career the question I ask always is how does that then fit in with the integrated view that also includes information about behavior, about ideas, about the choices that people make because in the end it's those choices that are really interesting. So as we think about infrastructure we think about how we're gonna provide information and then how we're gonna integrate that information and then finally how we are gonna provide the tools that allow that analytic world to go on. So that's my view of where infrastructure is. The challenge of course is to try to figure out who's gonna do all of that and that's an unresolved question I think in the scientific world is where the expertise to manage all that infrastructure goes especially when it crosses, when providing that infrastructure crosses the domain of not only of different kinds of sciences but of the questions of protecting privacy and confidentiality of making sure that advanced algorithms are available and so on. So I think I'll stop there and see if others wanna add and go from there. Great. One thing that I would add and this builds off of your last point there is that in addition to the sort of physical infrastructural issues that we face and the informational infrastructure issues there are also questions around the human infrastructure that's gonna support the projects that we're developing and that the human infrastructure is increasingly important I think as more and more projects are being developed in more and more fields that require not just immediate and developmental support but ongoing support and preservation over time. That human infrastructure is gonna be something I think that's really crucial for us to consider who it is that's doing the research, how research projects are staffed and evaluated and how the folks who are doing those projects are themselves evaluated. What the implications of these kinds of new digital realms that we're moving into are going to be for traditional processes of training staff, of hiring staff and of tenuring faculty and staff who are working on these kinds of projects. Great. I just had the thinking about that question about infrastructure today and then thinking back 20 years or so it's kind of remarkable how much what I would consider important infrastructure in my area has been privatized. I've been privatizing it the internet itself but I don't think, for example, it's necessarily a bad thing. I don't think something like Google Books was probably ever going to get done without private investment. But I do think it raises an interesting question when you start to think about poor research infrastructure being somebody's business. John, you chaired the commission that produced our cultural commonwealth, the report on cyber infrastructure for humanities and social sciences. What do you think has changed since that report? Have we laid more groundwork? Well, in the humanities, I think one of the things that changed after that report and because of that report was the development of the NEH's Office of Digital Humanities and the targeting of some grant funding on that particular area. I think they did a really interesting thing with the startup grants program which is a lower overhead, smaller grant like it starts with its own. And that's actually had significant impact in broadening participation. At least that's an outcome. And as college societies like the MLA before that report came out, we're already doing a number of the things that we talked about in that report like their guidelines for tenure and promotion and the scholarly edition. Good. Can you hear? No. Okay. Can you hear me? No. Okay. Just John's mic. Yeah. All right. So let's take a little fact check here. Any response? Any questions that you wanna pose? We're not sure how this is gonna work but if you're, yeah. So kind of think about that as we're moving along. We won't hesitate a long time but. So Cynthia from York University and I've collaborated with a researcher in social science and I'm actually supporting some of the infrastructure like out of my pocket for her database infrastructure. I'm doing that. But it's closed. And normally what libraries do is they look after information that they can share. That's who we are. And we have very interesting conversations about her intellectual property and I think she says, well the students at Ontario can use that but I'm really interested in going broader. So when we worked with this and we collaborate with each other at some point we're looking actually to be able to share that more broadly. At the same time I wanna respect what she's doing and her rights and she wants to keep it for a little while. So do you ever talk with anyone and discuss this balance? Because I don't really wanna use most of my money to put things behind closed doors. So I've been talking about this problem a lot for a dozen years. And I can't speak specifically to the situation under your rules. I wish we had my counterpart from Shirk here to answer questions. Right, it is, it is, and I don't know what these data are. It's NSS policy that subject to protection of privacy and confidentiality, all data that are produced with our funding in principle should be made available to other researchers. That's a pretty broad in principle. There are lots of hedges on it. But in general the major US research agencies have policies that are pretty similar on the subject of data. And it would be, so if she were my grantee we would have a little talk about this as I do from very rarely but from time to time about that. My general sense is that this isn't a question of intellectual property but it's a question of intellectual priority. And I understand and we institutionally understand that researchers who have invested a great deal in collecting information or managing information in a unique way need to have some time to have priority over that information. And then that priority should dissolve gradually. And that's the policy that I advocated at ICPSR Director. It's a policy that's very much in place at the major US funding agencies. So I think that my position on that information on the data would be what, but then I don't know what the data are. And so I don't know how to do that. So let me stop there and see if in practice there are other things that we might wanna add. I think we're becoming more familiar with sort of moving walls and other kinds of arrangements where data is kept private for some period of time. And I think libraries can live with that as long as you know what the end of that period is. We do that all the time in archives. That we say no one will be able to use this for this period of time and then it would become available. But I agree, ultimately, it's a very tough sell to spend the money to hold on to something for a long time that no one's going to be able to use, right? So let me just add in the more general principle that it does get complicated the larger and more complex the data collection becomes. And I think that if we talk about and other kinds of infrastructure, we get very quickly to the question of what kind of infrastructure do we need to use to make sure that we provide access to data while protecting confidentiality or protecting intellectual property rights as they need to be protected. And I think that's again, if we talk about how to build out infrastructure very much it's a question of how would we do that and what are the roles in the world of research including libraries where that can be done well and obviously it becomes an important discussion to have about whether how you're going to on the one hand make things generally available and on the other hand protect what needs to be protected independent of this initial investment question of are we investing in things that don't have any that won't have a broader vision, broader view. Great. Kathleen in your book, Planned Obsolescence and also your work with Media Commons you address some of the issues that surround a kind of infrastructure tenure and promotion. I think that could be considered infrastructure. Could you summarize the meaning and importance of what you've described as a peer-to-peer review? I will do my best. I wanna start with a sort of caveat that I'm not sure that the book makes sufficiently clear because I do spend a lot of time in the book sort of pushing back against conventional structures of peer review and I wanna start by saying that those conventional structures have been there for a reason, right? Peer review as we know it is absolutely crucial to the scholarly enterprise in its ability to help sort of assess work that's being done, allowing scholars to say that their work has been evaluated by the authorities in their field and that therefore the work itself becomes authoritative in a way and so it's really absolutely vital that we have moving forward some continuing mode of evaluation and assessment for the new kinds of work that are being done. We also know about the traditional peer review process that many of us participate in very often that there are a vast range of problems with it. It is not an unflawed system. I mean studies have indicated the ways that it's susceptible to reviewer bias to utter failures of reviewer reliability but also inter-reviewer reliability and even when the system is perfectly functioning when it's as scrupulous and forthright as it can be it contributes enormously to the development of scholarly work but it also requires an enormous amount of labor from scholars and from the other folks who participate in this process and it's a kind of labor that remains invisible and that we know to be radically unevenly distributed throughout the academy and so all of that aside, right? So there are problems with the current system of peer review but there's also a really fundamental misalignment I think between the ways that conventional peer review has been done and the new kinds of digital communication processes that a number of faculty are really getting invested in and are increasingly bringing work out through. The conventional peer review process we know to be sort of placed before the publishing the moment of publication as a sort of moment of gatekeeping, right? The economics of conventional publishing have been that we can only put out so many pages in so many books and journals each year and so you wanna make sure that the stuff that's coming out is the best. In internet-based communication of a lot of different varieties that particular kind of scarcity is over, right? We're no longer facing that material scarcity of the means of production. Instead what we're facing is a scarcity of time and attention, right? And so rather than thinking about that pre-publication mode of peer review online I'm increasingly pushing for this peer-to-peer mode of post-publication review which doesn't serve so much as a gatekeeping process as of a filtering process, right? It becomes a means of allowing a community of practice that's working with and sort of responding to one another's work to be able to guide each other to the best stuff that's going on in a field at any given time. Some of this has, you know, some of the functioning of a peer-to-peer review process would be derived from the kinds of metrics that one can track on stuff that is published online, right? What seems to be gathering the most attention and why? But some of it, you know, it can't be tracked through that kind of quantitative measure of hits or downloads or what have you. Instead, it's a means of looking at the kinds of discussion that are being generated by work that has been published openly. And so we're thinking about that qualitative mode of assessment in the same way that traditional peer review has functioned but simply after the fact as a means of assessing not just the work that has been published but also the stuff that has gone on around it, the impact that it's had on the field on the ways that it has been taken up and discussed and shaped by the community that it's engaging with. Great. So in some ways, John, this touches on one of your very early points as far as the scholarly primitives. I'm thinking of annotation, the ability to really track what a person has said about something else. And would you care to jump in on this conversation? Yeah, but maybe not on the annotation topic. I have been dealing with questions about peer review and new modes of publication since 1990 when we started doing postmodern culture and we would occasionally have to write letters to department chairs saying, even though it's published online, it's peer reviewed and here's our process. And I went along for years doing that before it occurred to me to look into the, what was this other peer review process that everybody assumed things were going through in print. And I delivered a paper at a meeting of journal editors of some high value humanities journals in Virginia some years ago where it became clear to me as I was listening to their discussion that there really wasn't a lot of peer review going on in print journals in the humanities that by and large what was going on was coterie publishing where a group of people that knew each other invited each other to publish in this journal. And yet these were the journals and that was the process that we revered and that we were trying to measure up to. So I think an open peer review process where everybody can actually see what's going on would be a great example to set. And I think it would be wonderful if, oh let's say some group of libraries decided to start a registry of peer review processes where different outlets could describe their peer review process and document it in some way so you would know what it meant to say that this had been reviewed and published in this venue or that. Until we start doing that I think we'll be dealing and I'm sure Kathleen deals with this a lot with assumptions about the alternatives that are just wrong. Right, right so many of those assumptions around open peer review are things like well if anybody can comment on it who's opinion are we to trust? And one of the ways that a particular mode of open review can distinguish itself is by as you say making that kind of conversation making the review process visible and making the reviewers and their contributions to the work that's being produced visible as well. So it does open review at least presents a certain kind of potential for surfacing some of the things that have been going on behind the scenes in scholarly work all along that's been sort of sub-rosa conversations that really change and have dramatic effects on work in progress but that we never actually get to see. I know Myron's getting ready to say something but I just wanna ask John and Kathleen one question is this sort of online exchange something libraries should be collecting and preserving? The peer review itself around the world? I would say yes but I also think that it's I mean it's gonna differ by publication venue by what the principles that a particular community of practice sets up in terms of the persistence of a review process as it's visible online. You know there are a number of authors who are gonna be very concerned about publishing in a venue where there will be both visible the draft that has been commented on and then the final version that you know there is a certain desire among a number of authors to make sure that only that perfected work is the thing that's out there and readable. Personally I think that the persistence of the process is crucial to allow future scholars to see the genealogy of ideas and how they have developed and changed to let students see that we don't actually produce stuff that's perfect the first time out but that we really do sit down and revise and rethink based on comments that we get. I think there's an enormous value in that and so you know I would encourage publication venues to make that work persistent and in that case yes I think the library is absolutely ought to be collecting and preserving that stuff. Just as we have collected early versions of artists' work and manuscripts and of music and so on. So John, you're my head, no my head. So I think it could probably be boiled down to a policy for libraries you know are you going to collect the final product or are you gonna collect the social text that produced it and in some cases I think the social text will be interesting and in some cases it might be more interesting than the final product. I think I look at the work that people do in blogs now and I'll mention some examples later but you know you see the way that the comments actually fold back into the article and cause revisions and you know have an effect on the finished product in a way that would be hard to understand actually if you just lop the comments off the bottom of the blog entry when you preserved it. I think the process of production is in many cases more interesting than the product. The last one of the last things I did at the Institute for Advanced Technology in the Humanities before leaving for Illinois was to print out on acid-free paper and box in archival boxes the 15,000 emails from the project list for the Blake archive and all their grant proposals and printouts of the SGML and screenshots and everything else to send to the Babbage Institute at the University of Minnesota because I thought you know 100 years from now that record of all those fights that went on in the course of producing what now looks like a charming antique on the web will be some of the most important part of the record of this period and it would be a shame not to have collected that. So I think the broad thing that I would take from this discussion and also include then areas issues related to data as well as to scholarly publication is the question of the importance of the community in both defining standards and needs and in establishing a record of what they do and from the data perspective it takes the form of the data and research infrastructure perspective that I represent it takes the point of view that says that the community operates to help sponsors, researchers, research institutions like universities to understand where the future lies, what the needs are and how to shape that in the specific case of the things that I try to support the National Science Foundation. I find myself constantly talking with researchers who come with some big idea to collect some kind of data and essentially say to me if I build it they will come and I respond by saying if you build it what if they don't come and I've already spent millions of dollars on it. Blake is a special case of course but that's. And so we've been in a variety ways and I talked about things like EarthCube, we have another initiative about data communities that we've been supporting together with the Education Human Resources Directorate in which we've said very explicitly tell us how your community is going to evolve and shape and then in the process of doing that tell us the data resources that you're going to produce and that you need and give us some hint about the science that's going to come from that using science in the broadest word because we're supporting things that I think are going to be incredibly useful for what many people traditionally think of as the humanities. And then let's build a community and I see this as being our angle on this broader set of issues that we've just heard about about how important the community is now that we can see the community. Now that the community is no longer, I've been a journal editor several times in my career myself and the community is reflected in those files that I have in my office of the reviews of submissions to my journals that have been done. And I agree with this notion of making as much of it visible as possible. It is going to change, it's the only way to change some of the monopoly practices that we deal with including the monopolies of communities and the monopolies of other parts of it. The one last thing I wanted to talk about again from the perspective of the data world is it's interesting to hear the difference here between peer review and curation in this context. And I wanted to put the curation word back on the table because there are things that I did when I was in the data repository business that we quite happily called curation and justified as curation in terms of preparing material so that it was available to the community in the best form possible. I wanna say that that's not the same as peer review and it's not the same as saying because you could then take something that was beautifully curated, put it out for review and still discover that somebody said you know your measure of this indicator in the community is wrong, just like you can put out a beautifully formatted print-like electronic publication as it were, article, and have the community say, you know, Myron, that's a stupid idea. And so there is that, I wanna, and then curation is still, I mean preservation is another topic and I wanted to make sure that we talked about those things and made sure that they were treated. Each one of them not separately but understandably. Great, this is a good point to stop in case there are questions or comments from the colleagues. David Carlson, Texas A&M. It strikes me as I've been thinking, hearing you talk that these are fundamentally different values-based kind of approaches. I think the fear that scholars have of a more open process is that instead of American Idol, it's gonna be American scholar. And the other part of this, and they don't want that of course, and it strikes me too that a fundamental part of our validity, if you will, is the authority process. And that is, that's what peer review done, however badly, et cetera, so forth, really is all about. So I guess my question is as I think about this and have thought about it for several years now, is whether or not you all see a different system emerging? Do you see a hybrid system emerging that we haven't just thought of yet? Or do you think we're looking at fundamentally two different systems of scholarly communication sort of remaining in place? I would suggest that I see many systems emerging. I don't think we're going to come down to a system of scholarly communication again, frankly, because I think that communities are fragmenting and finding different ways that they find valuable to communicate with one another. And I think they're similarly gonna find different means of reviewing the communications that they share with one another that are gonna serve their purposes, but may not necessarily serve the purpose of another seemingly similar community. I think you're right about the anxieties about an open review system. I mean, the anxieties are legion, right? There are concerns about the devolution of review into a popularity contest. There are concerns about whether review can be conducted in a sufficiently critical fashion if people have to sign their names to their reviews. There are concerns about whether, if it is conducted in a sufficiently critical fashion, are things going to devolve into the kinds of discourse that you see in the comments sections of most American newspapers, right? Are we gonna end up in this sort of bloody internet mass that we see happening in a whole lot of other places? And I absolutely hear and understand all of those anxieties about these kinds of open processes, but the beauty of an open process is that the process itself is visible. And if it does become purely a popularity contest, that is visible right there on the surface of the review because the comments are visible and we can really assess how evaluation is taking place. Right now, there are any number of journals that, as John said, are really code republications, right? I mean, I remember this moment, I was so dismayed in my assistant professordom. I had just gotten a rejection letter back from a journal that I had sent an article to and a senior colleague of mine was standing there as I opened it and I said, oh, another rejection. I can't believe that this journal said they didn't want this article. And he said, oh my God, why did you even bother sending it there? You didn't go to grad school at Chicago, right? And it never would have occurred to me that there was a publication. Right, exactly. So there it was, right? And that's not visible on the surface of that journal's process, right? That journal is highly respected and yet is very, very much a closed shop. These kinds of open processes really have the potential to aerate these kinds of situations. And to say, right, this is an acceptable process because we understand what the kinds of discourse that are resulting in this review are. And this one, frankly, is just log rolling and we're not gonna call this an authoritative process. Yeah, I'd agree that there's gonna be multiple mechanisms. I think that the many of the traditional high value publications, whether they be books or journals or other mechanisms, are going to preserve their priority status. But I think that in other ways, the production and distribution mechanism, try not to use the word publishing too much, is going to change and it's going to evolve to be different things. But I think also the discovery system has to change in parallel with it. And I think if we think about the roles that we play, the role of help with the discovery system is a really important one. And how I'm trying to think about this is I think about what I tell my students about publishing, do we tell them to publish in places that are operating the way Kathleen describes? And the answer is yes, if I can be sure that there's a mechanism that in the fields of research where I care about, there's a way for their work to be found, to be discovered that isn't because it's in a table of contents of one of 15 pretty good journals or in a book that's published or a monograph published by one of 25 presses. So we're going to have to change, adapt to this variety but also adapt our discovery mechanisms. Mackenzie, John, go ahead. Just briefly on the question of authority which I think is really interesting. Authority is always conferred by somebody on someone and it would be interesting to shift the conversation to value because value's recognized, not conferred. And I think what we may be worrying about authority as you say because we're worried about who will be conferring my authority on me? Will it be the rabble or will it be the right people? But if you shifted it to a conversation about value, I think it would feel different and you'd have different kind of conversations because you bring the value, people can recognize it, people who are not academics can recognize it in many cases and exactly how it gets tested and what not depends on where on the spectrum of empirical to non-empirical that research is. Not all research is falsifiable. But it still might have value. Great, Mackenzie, did you have a? Yeah, this question is a change of direction but it gets back to something you said at the beginning that was slightly provocative and has come up again and other remarks and that is about commercialization of the cyber infrastructure as it evolves and you sort of framed it as a bad thing, potentially bad thing but if you think back to the pre-Internet era, almost all of our infrastructure was commercialized from the literature to instruments to everything, hardware. But now we have this notion of the common good, the public good and I wonder if you could speculate a little bit on the balance that we're looking for between the investment needed and the commercialization that will come from that which was okay for a long time and this new kind of obsession that we have with the public good which I happen to share but what changed and why is it so important now that we have this sense of public ownership of the infrastructure that we're all depending on compared to the past? So I'll just briefly start off the response to that by saying for me at least it's pretty directly related to the answer to this question is pretty directly related to copyright and the questionable valence of commercial partners in infrastructure has to do with that as well. I mean I think in many ways it's great to have a big company with deep pockets pushing back on Mickey Mouse. We need that. And I don't think it's necessarily a bad thing for infrastructure to be commercially produced but I think we then need to think about the fact that if we are allies and a cause for example, we're allies and a cause for as long as it makes business sense to that other partner Roy Tennant and other people like to say Google's more like Microsoft than they are like us and that's true. They operate under the same rules. So I think as long as we understand that and as long as we can identify common cause and see when it's going away, it's probably the only way to get some big things done. So I'd add to that that I think it depends very well on what our strategic needs are and strategic goals are. So if what you are talking about is availability of a lot of information for the next end days and not being too many, I think having it in private hands if there's a way for it to be monetized is not a bad thing. What John didn't say explicitly is that that's not the same as fulfilling the preservation function that our libraries have fulfilled over the last century or century and a half and arguably longer. Oh, I have some comments about that. But then the question is, if so that's on the preservation side. If you go on the discovery side, then you have the whole question of bias on the discovery side, which is another set of issues that I think John didn't raise in his question. So how in the privatized, if we have a privatized discovery system, how do we make sure that those results are not biased in some way? And from the point of view of if a discovery system is critical for research, how do we make sure that it is unbiased or that there is at least one unbiased discovery system out there? So I'd extend that is comments a little bit further and talk about asks strategically what are the things that interest us? Here I talked about a couple of them, preservation discovery, but there are others. I would just add very quickly that I think from a certain perspective at least the problem is less privatization or commercialization than it is consolidation, right? That in the period during which we've seen the rise of the internet, there has been this very intensive consolidation of media corporations, such that we have many, many fewer players in the marketplace than we have ever had before. And that is not a good thing. It's perfectly fine and good to have commercial enterprises who are involved in helping us build and support this infrastructure, but having one commercial enterprise that's doing this is a really serious problem. Good, other questions? I'm Ruth Jackson from the University of California Riverside. I watched the NSF webcast earlier this year on big science and funding, the curation and support of big science. So are you looking at big humanities and social sciences? Are you looking at that in a different way? That's my first question. So from NSF's point of view, we don't have a big responsibility in the humanities, but we certainly are making investments in big science in the social sciences. Big data, it actually was a big data discussion. And there certainly are investments in big data. There's a new important award that we made to Penn State on an IGRID award for interdisciplinary training in big data for the social sciences. And there's a lot of other things that are in the works in that area. We don't have someone here from the National Endowment for the Humanities. They're certainly making investments and we do some things together with them. No one's talked about the Digging into Data Initiative, which is an interesting thing that's produced a lot of terribly interesting research at what we would call a medium scale, what my colleagues in the physical sciences would call tiny scale, but that's just a question of scale. And a lot of important findings very quickly. And that's interesting. It raises a topic that we haven't talked about here, which is that's a multi-country partnership. It's three US entities plus Canada, the Netherlands and the UK. And one of the interesting things that we haven't talked about here is that in a lot of data and publications, boundaries, and we had the question from Canada before, national boundaries are hard to define. A Twitter feed is a Twitter feed and it's out there. So I think there definitely are investments being made and there will be more made in the future, not only by us, but by the National Institutes of Health as well. My second question actually had to do with the curation component of it. When ARL and the DLF had the capstone event on eScience this year, there were scientists in the audience, faculty. And so one of the questions raised was when we talk about data curation and preserving everything, versions of articles and comments, et cetera. Then who decides what is finally kept as the official record of anything or is it all going to be out there? Some of the scientists seem to think that the standards should be set by discipline. And also the number of years, so should it be 100 years? Should it be a thousand years or 5,000 years? I just throw that out there for you to comment on. Yeah, at the moment, bits are cheap. But I think it is an interesting set of questions. I don't have a clear answer. The consensus in the scientific community now is that we don't have consensus. And therefore there's a lot of difference by community. I don't think that's so bad. And it goes back to Kathleen's point about the venues and the mechanisms varying by community and by community standards. That's the point I guess I would make. I think at some level this is a problem which may actually take care of itself. It's one of the rare ones. Because I think digital data isn't gonna stay around for all that long unless somebody's using it. I think it's by picking it up and using it that we'll figure out that it's broken and it needs to be fixed. And if nobody uses it for long enough, it's not gonna be functional 100 years from now when somebody picks it up. So we'll just throw it away because it doesn't work anymore. And I think that preservation through use strategies in practice what's gonna happen with all kinds of digital data. Great, I'm gonna move the conversation just slightly. As we were talking about community and authority, earning of value and so on. Kathleen, I wonder what you see as the role of scholarly societies and associations in this changing environment. My sense is that the role of scholarly societies remains largely the same, right? Though it's sort of shifting in its orientation. I mean the primary function of scholarly societies has been since their founding, facilitating communication of a scholarly nature amongst their members. And I think that remains exactly as it has been. But the difference now and it's a profound difference is that instead of a system that functions around the journal, right? In which one becomes a member of the scholarly society in order to receive the journal that it publishes. Instead I think we're gonna be moving over the course of the next few years into a system in which one joins the scholarly society in order to have one's own work disseminated to the world, right? That it becomes less about the reception of a product being produced by the society than the society is a facilitator for conversation and discussion amongst the members. That has happened for a very long time around society meetings, the convention, the annual conference, right? That has been that space in which people come together face to face. I think the role of the scholarly society is increasingly gonna be to provide the platform on which those conversations can happen year round. And on which members can get their work in progress into communication with the people that they most wanted to reach. That's great. I hadn't intended quite to go to this topic so soon but I think I will because you gave a good lead in. An example of an emerging digital scholarly network, a little bit along the lines of what you were describing could be seen as an internet-based knowledge aggregation system such as Wikis, Boggs, or other open source software that encode a new model for editorial practices and a new model for publication or dissemination. What do you think is at stake for the members of the network and for those who might be excluded? This is a really interesting set of questions. So the MLA, you may have seen the announcements go by, is launching a new scholarly network for its membership called MLA Commons, which we're gonna be launching at the Boston Convention. And this is a platform that's meant to facilitate member-to-member communication at a range of different levels. We are grateful to have the collaboration of the CUNY Academic Commons team in the development of this platform. They're working on generalizing their platform into what they're calling the Commons in a box. And we are in the process of building on top of that for our membership's purposes. The platform, our work that I should note has been very generously supported by the Andrew W. Mellon Foundation, which we're extremely grateful for. The platform, as we're envisioning it, is gonna facilitate different levels of conversation among different groups within the MLA membership. There's a long history of existing groups, subgroups within the MLA that do certain kinds of work together. Our organizational structure includes divisions and discussion groups that operate based on subfield or interdisciplinary sub-subfield interests. And those groups will be able to have space in which they can carry on whatever sorts of conversations, create whatever kinds of publications, create whatever kinds of filtering systems for those publications that they want to be able to create based on their scholarly interests. We also have space in which those groups can do work, those groups and the committees that actually do the governance work of the MLA can work both in public and in private, can have discussions that they need to have of an organizational or structural nature, but also can put material out to the world. Now, the network is going to be, except for those private spaces in which governance-type conversations take place, it's going to be open in terms of readability, right? It will be findable on the internet. Work will be disseminatable as far as members want their work to be disseminated. But the line for participation in this community, the thing that makes it a member benefit, is the ability actually to publish through this system. So non-members will be able to participate in a lot of discussions around the kinds of material that are being published in this network, but actually doing the publishing requires membership. So I think that's where the, if I get the question, that's where the same slide. And I don't know if it was you or someone else who said this, both reading and writing to this space. Yes, absolutely, absolutely. Myron, do you see parallels in the sciences for this kind of collaborative network? Yes, I mean, I haven't, I mean, so I don't have a lot of examples, but I think that, I agree strongly with Kathleen, as I've been talking with people from professional associations, that they have to revisit their model. And it's a challenge, and again, this is nothing new about this. This is not new in 2012. This was maybe a little new 10 years ago, when online publication of professional materials by associations was new words, still wasn't new then. But I think the idea that professional associations have to find a place to enable communication among their members that supplements the traditional mechanisms is clear. And I think we're gonna see it in every mechanism that they have. I hear about it, I'm closer to the social science, social and behavioral science associations than I am to all the science associations. But I certainly hear it from those associations, and you hear it certainly to the extent that I'm tuned into it. These days, I think most visibly in the physical sciences, I'm beginning to see quite a lot of move towards new mechanisms for communication. And looked at it from today, it's obvious everyone should have done it five years ago. The technology was already probably there. But as something to be able to represent to one's members that one enables discussion, discovery, publication in new ways is really important. I think it's gonna happen. I think it's gonna be an explosion, to be honest with you. I think the MLA may just be at the head of the, it's nice to be at the front. It's nice to be at the front, yeah. So who's responsible for collecting, organizing, and preserving this scholarly network? That is an excellent question. A librarian. Right, my hope is that, how to put this, we are responsible on some level for collecting it. Because it is, the network is what the network is. And obviously we're going to be backing the thing up as robustly as we possibly can. But in terms of long-term preservation, I think we need a much, much more communal solution. I mean, I think these kinds of online scholarly networks particularly as they proliferate, which I think they are bound to do, are gonna demand their own sort of locked system. In which we have many institutions participating in ensuring that that material stays robust and active. So I would argue, in fact, that there's a strong case for a small number of platforms and not having a proliferation of platforms. Right. I mean, obviously groups are gonna do, communities are gonna do what they need to do to support their mode of communication. But if you're thinking about a preservation task, even one that thinks about a locks-type solution, the smaller the number of effective platforms you have, the more likely it is to be successful. And if you, and so, I think, but that kind of consolidation will definitely happen. So it's not, I don't even worry about that being the case. There will be providers and... We just put in another plug for the CUNY folks. One of the things that I think is really super smart about the way they're developing the commons in a box that they have an eye toward thinking about federating any commons that gets produced out of that installation. So if you run the commons in a box, at some point any commons will be able to federate with the others, such that if you have a commons at your institution that a faculty member belongs to and their disciplinary commons that they participate in, they will not have to update material in both places. Anything that they update will appear automatically in both because they're federated. So the dataverse network in Harvard is an equivalent that Harvard has started up, has many of these same attributes. John, do you think that there will need to be an increased use of artificial intelligence to index, query, organize this and other massive amounts of data that we have? An increased use of computational methods, yes. I'm not ready to make claims for artificial intelligence. But I think clearly just working with literary texts, there is now enough of that material around that you can do interesting computational work with it. And when you start to try to do that, one of the first things that you discover is that it doesn't have the metadata that you wish it had and you're not going to go back and recatalog those things. So there are reasonable techniques for inferring certain kinds of important and useful metadata and I think that's what people will do. And in fact, there are good examples out there now of people doing exactly that. And this is another reason for promoting the idea of data communities is that different people will be interested in doing that in different areas of large collections and you would like to keep the results of that around for other people's benefit. You'd like to have a big enough pile of data at the center of the community that it attracts tools and it attracts research results and things kind of stick to it. It's got some gravity. I think one of the interesting challenges for libraries around data communities is that libraries by and large have a local constituency. They provide a local good and they think of themselves in terms of their value to their local institution. Several years back in Illinois, we had at the instigation of the then provost there, Linda Kotehi, had started a conversation with Michigan and Penn State. Mike was in this conversation about data communities and the humanities and social sciences and we had deans of high schools, we had directors of libraries, we had provosts and we had directors of presses and really it was the presses. The presses were the only party at the table whose business it was to tap into disciplinary networks and the other party that wasn't at that table that has that mission is scholarly societies. So figuring out how the question of preservation comes up, it raises this issue in an interesting way. Well, who's responsible for that? So if the library serves a campus which has faculty members who belong to the MLA, does that make the library responsible for collecting the output of the MLA's scholarly network? It's a hard question. And if we don't, if libraries don't engage in those cross-institutional disciplinary communities, as more of this stuff starts to happen, what exactly will we be doing as far as the world be on the borders of our campus? And all of these communities are organized in ways that kind of cross these institutional walls by definition. So these, I mean, a consortial arrangements are what you're talking about. They're exciting and challenging and have to be paid for. And the challenge, of course, is just as you talked about local communities, the making sure that if you're doing that was old, that's the challenge that we had at ICPSR that they continue to have is, and they're the poster child for success in terms of after now, you know, 50 years of success at convincing institutions that the consortial good has enough value to pay for it out of institutional funds. But those are things that you have to justify, just like you have to justify membership to individual memberships, to professional associations, you have to have a reason for that that makes it work to get somebody to pay a bill, even if it's not a terribly big one every year. And that's what the association, that's what I faced when I was director of ICPSR, and that's what the associations are continuing to face, especially if people thought the journal was valuable and now can get the journal from their campus library without even doing anything, you know, without having to go down and walk down there and wait till the issue is available and take a look at it. All right, questions. Sorry. So I, I suppose you know me, Caroline Dow from NYU. And I'm really interested in this conversation about data communities and trying to, you know, go beyond one institution. And we're talking about it from the after the fact preservation angle. But of course I spent literally most of my time trying to figure out which of these kinds of initiatives I can support. And I love the idea of, because scholars, especially in humanities, each wants their own, you know, customized special law. And you know, and Kathleen knows, because we do media comments, but now they're not gonna, so and we say, well how about that platform? No, well I need a different for media comments. So I really like the concept of association, scholarly societies, starting to lead the way. And other kinds of consortia, getting folks into that mindset, that there may be these larger ways of doing it. And I would love to figure out how the library support structure can help a community instead of just scholar acts customize it to the way she wants it. Would love to talk with you about how libraries might get involved in helping to support this process. I completely agree that there's a lot of the kind of work that's been handled at the institutional level within libraries that has required a lot of customization based on the fact that you've got faculty coming to you from every discipline that could be better handled at the scholarly society level. I will say that I think one of those things may be the institutional repository. And you know, if there were ever a case for if you build it, they may not necessarily come. That may be it. Scholars in depositing their work tend not to identify with their institutions in the same way that they identify with their field groups. And so if we were to focus instead on creating field-based repositories that then could be mirrored or linked to or in some sense connected to from the institution, I think we might have a whole lot more buy-in. But we'll of course need libraries to help us out in figuring out how to build this thing and how to create those kinds of connections between the societies and the institutions. Brian Schotlander from UC San Diego, a partner in crime with Carol Mandel and a number of initiatives, including having to decide where to put one's money. I wanna respond to Myron's last point when he said to John, so what you're talking about are consortial arrangements. And I wanna suggest that I'm not sure consortial arrangements is actually what we should be talking about. So I think about something like aerospace for instance, which is not a consortial arrangement, at least not in the way we used to understand consortial arrangements that a number of us are approached by to join or sponsor in one way or another that are working with various of us in the community to develop various service capabilities. So the Cronopolis long-term digital preservation capability is a back end to duraspace. You can actually benefit from that service provided by that whatever you wanna call duraspace, it's not a collaborative, it's not a consortium, it's a thing, a gravitational thing. And I think there are gonna be more and more of these kinds of gravitational things that we're gonna wanna find ourselves thinking carefully about not only from the vantage point of which one should we put our money to, but how do we see them relating to each other in the ecosystem that is scholarly communication. So the panel this afternoon has done a very good job of teasing out the various elements that comprehend that ecosystem. And so we find ourselves for instance, for once, talking about that part of the ecosystem that is actually scholarly discourse, that part of scholarly communication, which frankly, with the exception of some manuscript collections held in Arkage Summer, we've never curated, we've never stewarded. And yet we've known all along faculty member X walks down the hall and says, dude, what the hell were you thinking when you drew this conclusion? Never makes it into the published literature. So good on you to develop an infrastructure that allows for that kind of discourse to come out of the hallway and onto the internet. And yet now that it's on the internet, all of a sudden everyone feels like, whoa, somebody ought to be saving that. And indeed, we should. And the big question is who is the we? And I, like Carol, welcome MLA and those of the rest of you in the community who are beginning to think in rather more expansive terms about what constitutes the scholarly communications system. We have tended to think about it as only the output end of the system and occasionally we think about the input end of the system and those of us who are into data curation are beginning to think about the input end of the system a lot more aggressively. Almost no one thinks about that stuff in the middle, which systems engineers call operators but we might call scholarly discourse or scientific discourse or whatever. Yeah. And Kenny Cornell. I do think we do have some exemplars of that. I think the archive.org is raw scholarship. There's a little bit of vetting. I think the issues of how baked things become is a very interesting thing and at what costs over those various levels we ought to be looking at. But I have a different question that I'll just throw out there and you can decide whether you want to answer it or comment on it or not. So this is all about 21st century scholarship and researcher services and transforming scholarship. And one of the biggest things on the horizon we're seeing now in terms of transforming education is the rise of MOOCs. But I would suggest that the impact of MOOCs on transforming scholarship will also be pretty profound. I had, was it some conference recently in which someone posited that MOOCs would be the new form of the monograph as we not only look at education but as we create the sort of content that will go along with it and support it. So I'd be interested in your comments on that and the role that research libraries can play in that domain. I've received better, yeah. So there's an emerging field of analytics of the data, the clickstream data that is being produced by the MOOC that the education research community is very excited by. So here we have another angle on this 21st century technology and how we think about what that means and how we protect the privacy and confidentiality of the people who participate, how we effectively analyze them is an additional twist on that. So what you are talking about, I think, is this is the multiplication of uses of the new technology. So this is something created to train people in with, to produce education. You're describing it as a mechanism for communicating knowledge. I'm describing it as a resource for research about how learning takes place and also as a resource when merged with appropriate other information for understanding both societal influences on learning and societal impacts of these things. It's what I mean, I'll take it extended and say this is very much a 21st century question of how do these new technologies operate? I would just add to that that I think the question actually predates MOOCs by a pretty good bet. The thing that first got me started thinking about the fate of the monograph and what other forms of scholarly communication might rise up alongside it was that I had a blog. I've been blogging for 10 years now and I was struggling to get my first monograph in front of a publisher that would actually be able to publish it. At the same time, I had hundreds of people every day coming to my blog and reading it and engaging in conversation with me. All of my first, those forms of official academic recognition that we sort of consider the markers of impact, lecture invitations, citations, all of those other kinds of things, they all had to do with stuff that I was publishing on my blog, not my traditionally vetted and published material. And so I started thinking, well, if I have this platform that's my platform on which I can put my material out there and I can get pushback from people in the field who say, you're completely wrong, how could you possibly have drawn that conclusion? But then can come back and revise and continue to engage in this kind of conversation isn't that on some level a better way of getting material out than holding it all back for the three to five years that it takes to produce this book and suddenly releasing it out into the world. There's been a lot of questions that have been raised about what the future of the monograph should be. I have argued in places that we really need to de-center it as the gold standard of scholarship in the humanities and start thinking about other ways in which humanities careers can be better assessed and evaluated. But I don't want to get rid of the monograph entirely. I think that it still serves a really key role at really pertinent moments in furthering a certain kind of scholarly communication. I think the problem that we faced with the monograph in the humanities is that it has been the only way that a career can be measured for much too long. And the more that we can allow a diversity of ways of scholars reaching the audiences that they're trying to reach and even reaching beyond their peers to other audiences who actually really like to engage with the material that we're producing, the better off we'll all be. And I think that lots of new technologies present these opportunities from blogs all the way through the kinds of video course-based work that's going on in MOOCs now. I think there are gonna be more of these sorts of opportunities, but I think in order for them to flourish, we've got to, as an academy more broadly and led by, I think, the scholarly societies, have got to find ways of saying this material must be evaluated on its own merits rather than being somehow required to be condensed and published in the static format. Great. Tom. On the subject of the scholarly societies. Earlier, I'm sorry, Tom Leonard from Berkeley. On the subject of scholarly societies, earlier you were explaining how forming judgments, even perhaps peer review, could become active in a new network of scholarly society spawned networks. And I wonder how that squares with an old maxim about these scholarly societies that they're all one lawsuit away from oblivion. And we've seen some evidence of that in the retreat from adjudicating plagiarism. Recently, many of us have been reading about the problems art historians face should they venture an opinion about the authenticity of a work held by some owner for sale. I just wanted to make the comment that libraries are actually much better in standing strong when we face lawsuits than these scholarly societies. We've had a lot of practice. And we've actually had some extraordinary victories. The tobacco papers at the University of California Medical School, San Francisco as an illustration. So I would hope the scholarly societies would think about the dangers they're opening themselves up to, even as we applaud them for going forward in this way. And it might be a very good idea to think about how libraries can help you to take the heat. I absolutely welcome that. And I think that that is a really important comment. I think that the tenuousness that you point to in scholarly societies has to do with a more generalizable tenuousness in the funding for higher education broadly right now. I mean, I think there are a lot of our institutions that are one lawsuit away from oblivion, not just the scholarly societies. So I think that we're at a moment of really dramatic change in business models, in the modes through which societies are able to keep themselves going, continuing to do the work that's really important to the membership but is ultimately non-revenue producing, right? While doing other stuff that members are willing to pay for. It is an extraordinary challenge. I would acknowledge that societies are opening themselves up to new kinds of challenges as you suggest, and that we're gonna need support from all kinds of institutions in facing those challenges. So I thank you for being willing to take some of that heat. Sorry. I would add a second challenge to the professional associations that comes from the tension between the disciplinary nature of many professional associations and the interdisciplinary thrust of most of the public organizations that are out there. And then that leaves aside the everyday challenge I have from the political parts of the political spectrum that want to write and the public spectrum that want to write parts of science off of the table. But I think we have an issue here in this scholarly communication and scholarly discourse issue that we're gonna have to resolve about what happens in the gaps. And just saying that we have new media to communicate with doesn't answer the question of what do we do if we are generally talking about a problem that is in literature, but the issues I want to write about are about spatial aspects of it. And the community says, you know, you don't even belong, you know, the reactions that come back say, you don't even belong here. We're gonna criticize you and not anything else just you don't belong here. And so that's another challenge. And again, to give credit to the libraries, you know, this is another area where the libraries are not disciplinary. You know, they're about knowledge and information and that's a point of safety. Another reason that you find a point of safety there. One last question. I'm Sarah Mahalek from the University of North Carolina at Chapel Hill. I wanted to mention about the Blake Archive, made its way from Charlottesville down to Chapel Hill and is residing on our institutional repository and our digital preservation repository. Joe Viscomi who you must know created the Blake Archive and continues to this day to be a poster person for this vast library of digital information about one author to be a legitimate career long scholarly project by which he is pleased to be evaluated. And he is a frequent speaker on behalf of the digital humanities and some different kinds of digital products as a new parallel to the scholarly monograph. And circling way back in the conversation, I wanted to tell about the young editor of the Daily Tar Heel, he's a junior and the general editor. And he announced a new policy last week for the comments.com aspect of the, the online aspect of the Daily Tar Heel. He said that from now on, no anonymous comments would be accepted because he felt that a university is a place where people should be accountable to their thoughts and actions and statements and that clearly anonymous commenting is just not really doing the job. So it's a great example right at home of a person who's caught onto it to this idea of being responsible for your thoughts online. That's really great to hear. One of the things that I failed to mention in that early description of peer-to-peer review and some of the things that I imagine as being important to a system like that is that in a peer-to-peer review system, one of the key components is perhaps less the review of any individual text or object that gets published through the system than a sort of ongoing process of review of the reviewers. Making sure that the reviewers are really actually doing the work that they should be doing there. And that absolutely requires a sense of membership in and accountability to a community of practice. And so I think this question of anonymity while fraught for many academics, particularly when it comes to peer review, again, there's the question about whether non-anonymous reviews will ever be as critical as anonymous reviews are. Being willing to be accountable for the opinions that you express about a piece of scholarship I think is the bedrock of the scholarly enterprise. So I applaud the Daily Tar Heel for that decision. So I think Joe wouldn't object if I slightly amend your statement. Joe is one of three principals in the Blake Archive, Morris Eves from Rochester and Bob Essek from UC Riverside. And they're an interesting group. They're little and feisty. And they fought a lot actually during the course of the creation of this thing. They're great friends and they've been working together on various Blake projects starting with the Tate, Princeton University Press series of facsimile volumes that they contributed to. But they're an interesting example for this question of blind peer review too because that's not the most esoteric field of study that we could come up with in the humanities. But it is a sufficiently small community that really anonymity isn't possible. I mean, you know from what people say who they are in that community. And yet they managed to get their work done. And in fact, the more sort of face to face they are, the less anonymous, you know, in my experience, the more the volume goes up. So it's not even a case where, you know, being online causes you to say things that you wouldn't say in person, not in their case at any rate. I want to thank you as audience for participating and allowing this different kind of forum in which we could have a real conversation. I especially want to thank our panelists who bring to us so much experience and expertise and such an open heart and willingness to share with us today. Thank you very much. Thank you for listening. Music was provided by Josh Woodward. For more talks from this meeting, please visit www.arl.org.