 So, it's really very wonderful to be able to welcome you all here. It feels very strange, doesn't it, to actually be back together in person. We've almost forgotten how to do this in some ways. But I hope we'll all be able to relearn it as we go. I did a session a little earlier to welcome first-time attendees and I asked if there were any questions and somebody unforgettably said, oh, don't those go in the chat? Which just underscored for me how profoundly our behaviors have been rearranged over the past almost two years now. I'm Cliff Lynch. I'm the director of CNI and I'm going to spend the next hour or so talking about some developments looking back over the last year, looking at some of the things that may lie ahead. Before I do that, though, I just want to take care of a few other things. First, I want to just extend a very warm welcome to our members, member representatives, to our guests, and we do have some guests with us. I think you will find that this meeting is heavier on member representatives in terms of who's here than our meetings in the pre-pandemic times. So I hope you'll have an opportunity to talk to many of your colleagues. We have a very, very small number of international people with us, and I want to extend a particular welcome to them. Attending our virtual meeting last week, if you were an international participant, might have been one of the easiest trips you've ever taken. I'm thinking international travel right now is quite challenging, and I'm very grateful to those who were able to navigate that. I also want to extend a special welcome to a number of upcoming leaders and fellows. This includes the leading fellows. You'll hear about leading and their work just before the reception tonight. I believe we may have a couple of clear fellows with us. We did a session as part of our virtual meeting because many of them were not in a position to join us in person, but if we've got any with us, please be very welcomed. We also welcome the ARL-LCDP fellows and any of the upcoming leadership fellows, Cadre. I understand that program technically doesn't start till January, but I know that we have a couple of those folks with us. We've done a number of things differently at this meeting. You may have noticed the masks, and I would suggest that if you're not actively eating or if you're not speaking, that you wear your masks. We have positioned all the podiums and things far enough away from the audience to give a bit of separation for people who are speaking without masks. We have had a couple of schedule changes, and we put out an email about that on Thursday or Friday, I believe, including a late edition of one session as well as the cancellation of a session. You can find any further schedule changes on the board over by registration, and as we all know, these days things can change, so do keep an eye on that in case there are schedule changes. You'll find that this meeting is a lot more leisurely than the pre-pandemic meetings in terms of the scheduling, the length of the breaks, lots of time for networking and talking. I will note that this hotel is also pretty well laid out for people to be able to find nooks and have conversations. At all of the meals, we will have some provision for takeaway as well in case you want to take your meal and take it off somewhere rather than dine in whatever room we're dining in. We've only got a maximum of three concurrent sessions, so it should be a lot easier for you to decide where you want to go. Furthermore, unlike the pre-pandemic times, we have actually put video capture in each of the three tracks, so you can count on having high-quality video of whatever sessions you didn't attend that were concurrently scheduled and that you want to see. We will, of course, make those available after the meeting, so this meeting will be comprehensively captured on video, and I hope that you find that useful. I think those are all of the sort of logistic announcements that I wanted to mention. Let me turn to the main things I want to talk about. One of the things I've been reflecting on a lot over the last six months or a year is innovation. Innovation, as I'm thinking about it, is something that starts with a technology base, which may be kind of pedestrian, and then enables social or cultural change, perhaps in response to various pressures or demands. So it's not just about technology, it's about that juncture of technology and social or cultural or behavioral change. And one of the things that's really striking is if you look at what's happened over the last just shy of two years, there's been a lot of what I might characterize as innovation of necessity, things we've had to do. And it's striking to me how many of those changes of necessity really did not involve new technology. Just look at the list, or make your own list. Control digital lending, e-reserves, digital first acquisition strategies, remote instruction, residential connectivity. Now that turned out to be a big problem and one where we had to innovate quite a bit and a lot of that innovation was innovation that we could do as educational institutions, for example, that helped but wasn't enough because the sort of core issues of residential connectivity really go substantially beyond the control of higher education proper. If you look in the broader world, we saw the emergence of telemedicine as a much bigger thing, remote work as a much bigger thing. We saw the adoption of digital signatures. None of those things that I listed were fundamentally new technologies. They were all there before and they were things that were foregrounded because we didn't have a choice, we had to. And one of the questions that I think is important to ask and I'll come back to this later, is when you do innovations of necessity, do they stay after the necessity is eliminated? That's a really interesting question. And I think the signs are very ambiguous right now, but I just invite you to be considering that as we continue on to this conversation. I will also note that under the pressures of necessity or perceived necessity, we developed some really nasty innovations too. One of my favorites in the nasty innovation category is remote proctoring, which was an effort to do something that we used to do in person virtually rather than going to the trouble of thinking about do we really need to be doing this, which we probably didn't in most cases. But that was one of the interesting and to me very unattractive technologies that are practices really, because it's not just a technology, as I said, that emerged of necessity. Now I think when you look at things that aren't being driven by necessity and you look at the nature of most of the innovation that's been going on lately, it feels like it's pretty incremental. You can identify trends, for example, towards centralization. That's being driven by economies of scale. It's being driven by certain labor trends. It's being driven by cybersecurity considerations and increased threat levels. You can see some innovation that's being driven, frankly, by market consolidation around the margins. It's really sort of a pretty incremental picture. When you look at transformative innovations, they are much rarer. I've been having some conversations last week. We held a couple of executive roundtables looking at some of the transformative innovations that might be wading in the wings and how to think about them. Sometimes, by the way, I would note that transformative innovations kind of sneak up on us. The story of Hottie Trust, for example, is really interesting. One wouldn't have regarded that perhaps as transformative until we had the pandemic and we moved to the emergency access program. Now we're fighting, and remember I raised the question about what persists and what doesn't. Institutions are struggling to figure out what do we do with this now that we had it and now that we can get back in our libraries again and circulate those materials. That's an example of something that we'd like to keep out of our innovations of necessity. When you look, though, at really transformative things, I see a relatively short list over the last few years and we've tried to highlight a number of them here. These are things that are maturing or in the planning stages. Some of them are things that we hope, rather than know, will be transformative innovations. One that I am very enamored of, for example, that looks like it's slowly but surely getting there is snack. And the work around that, providing names, sliding off into factual biographies that can provide the backbone for a tremendous amount of innovation. We're going to show you, at the closing plenary here, a very interesting institutional innovation that I think could be quite transformative. And it's really interesting to think of the differences between institutional innovation where you see a single institution making a relatively unilateral decision that we're going to go build acts or invest in Y or whatever. And perhaps others will adopt the innovation, perhaps they won't, but it doesn't matter moving ahead. And contrast that to what I'd call community innovation. One of the really interesting potential community innovations that I'm hearing a lot of rumbling about right now that maybe is coming out of or driven by the pandemic, but it's going to take time and it's going to be community-based rather than institutions acting unilaterally, I think, is going to be a shift of special collections to a more digital usage pattern to allow much more remote access, online access to think about how people can use special collections without being physically there. Now, that, as I said, needs to be, I think, a community innovation if it's going to take hold. Just having one institution say, oh, we're going to digitize all our special collections isn't going to be enough. Really, most special collections are used in a community fashion in the research and higher education community. Another place where I'm seeing attempts at community innovation are various new platforms for scholarly communication. For example, some of you who were at the virtual meeting last week may have seen the talk on Octopus, which is a platform that the UK is investing in to try and deliver scholarly outputs in a very different way and for very different purposes than the existing journal system. I'm very interested in people's thoughts about what other community innovations we might be seeing or how institutional innovations can evolve into community innovations. I mentioned the closing plenary on Cloud Lab and I'd invite you to juxtapose that, which is, as it stands today, very much of an institutional innovation with some of the discussion in the very recent paper out of Ithaca SNR on research core, research core instrumentation and how those are increasingly moving beyond the institutions that host them into things that are approximating regional or national shared facilities. This is a move that's still early, but something like that begins to represent a path that might converge with the institutional innovation at Carnegie Mellon over time. It's also worth noting another quiet set of developments that are very hard to really document and that Ithaca didn't explore in that paper that they put out. And by the way, the late breaking session that I mentioned earlier is on that paper and its findings. What they didn't explore, but I'm aware of happening in kind of a localized way, is regional networks, regional research and education networks are starting to connect up some of these core instrumentation facilities. Again, it's sort of a low profile thing, but there's definitely work going on in that area. Again, that starts moving these to be facilities that can be used more broadly and starts to increase the overall resilience potentially of the research enterprise. One of the other things that I'm wondering a lot about is how much innovation, particularly around research, is going to devolve to work in individual disciplines as opposed to fairly broad brush, multi-purpose innovation. It may be multi-purpose within a certain complex of disciplines. Core instrumentation is pretty much for the sciences. There's another interesting conversation that's just getting off the ground right now. Where the American Council of Learned Societies with funding from the Andrew Mellon Foundation and the National Endowment for the Humanities has just, as of a couple of days ago, announced a commission that could be viewed as a sort of a revisiting of the commission from around 2005-2006 that John Unsworth chaired looking at needs for cyber infrastructure in the humanities. And if you look at the charge for this commission, it's quite broad, but it very much is focused on the humanities. And I think you may see out of the work of that committee calls for new investment to support the humanistic disciplines. But I do think a lot about this question of how much innovation is happening in the disciplines. If you look at biomedicine, for example, which is a very well-funded area, it feels to me like we're seeing a lot of innovation. And it's really worth watching what's happening there. For example, they have started in biomedicine moving away from conceptions of what are now called generalist repositories, which basically you can think of as giant FTP archives. And you're familiar with these most institutional repositories work like this. There are non-institutional generalist repositories, a very good example being Dryad. But in biomedicine now, we're seeing the construction of these repositories and analysis environments that basically do various kinds of data validity checking and normalization in support of major research programs, where you essentially gain value as a researcher by putting your data in there because it is refined and enhanced. That's a very significant development. These things are expensive, but can be very powerful in driving the work of research communities. Another thing you're seeing a lot of innovation around is in discovery. For example, systems like Semantic Scholar are used reasonably heavily in biomedicine. Or actually, you can even just look at the kind of work that NLM is doing with the various services interconnected to PubMed Central to see some of the innovation there. We don't see this in a lot of other disciplinary areas. Another thing I'm looking at very carefully now is how AI is starting to affect various disciplines. Usually, when we say AI, what we mostly mean is machine learning and adjacent technologies, not always, but mostly. It's really interesting to me to look at the divergences that we're starting to see between the humanities and the sciences in terms of how they're starting to apply AI. Machine learning in both the sciences and the humanities is, of course, driven by the availability of big data, well-structured data, training sets for ground truth. It's interesting to note that while open science practices in the sciences have made huge amounts of data available for machine learning in the sciences, that things are much more complicated in the humanities in terms of what data sets are available and what the humanists want in order to do their work. And it would be very interesting, I think, to look at that, particularly when you get humanists who want to engage with the artifacts, if you will, of modern culture, as opposed to, let's say, classicists, where most of the source material databases are pretty firmly in the public domain. Now, here's a little kind of, I don't know, I don't almost call it a caricature, but I think there's some truth here. When you look at what humanists are mostly doing with machine learning, they are trying to either discover things, find things, extract things, or transform things. You see, for example, a lot of work trying to classify or discover objects or things. You're seeing a lot of efforts to try and do things like read handwriting, identify people in old photographs, work of that nature. If you look on the other hand at what scientists are doing, a lot of what they're doing is prediction. In other words, they're trying to go from a collection of observations and facts and predict things. When you look at some of the really spectacular work in science recently, it's things like predicting how proteins fold, predicting various kinds of physical phenomena. It turns out, by the way, there's a very interesting kind of thing that's driving a lot of the science, which is that in many cases, in theory, we know how to do these predictions. We understand the basic physics. It's just that the computation is so huge, so long, so intractable that we haven't been able to do the computations. And somehow machine learning seems to be able to identify patterns, in some cases at least, and usually do the predictions that we can't afford to do the computation for and be right a lot more often than it's wrong. That's kind of how the protein folding situation worked, for example. This is showing up in a number of predictive things that are computationally really intensive right now. So I think that that difference between prediction and transformation or discovery is really quite an interesting one. It's not absolute by any means, but it is striking. I spent some time in October at an online workshop that the OECD did on AI and scientific productivity, which was really, really interesting and gave me a lot to think about. So this started with a discussion of how do you measure scientific productivity and is it getting better or is it getting worse? And remember, when you think about scientific productivity, you can think about this with two different lenses. One lens is on the individual scholar, sort of the promotion and tenure and evaluation kind of thing. How are you as a scholar compared to other scholars? The other is at a national level, how well is our country doing at science and technology compared to other countries? And this is something, this sort of national level comparison is something that it turns out countries spend a lot of time and energy on. They're really interested in this in part because most industrialized nations are spending a ton of money underwriting research in science and technology. They do it in the name of national competitiveness and they really want to benchmark how they're doing compared to how other countries are doing. So you see, for example, the National Science Foundation has a very well-established program in essentially science indicators. The UK certainly does a great deal of this. I would just say, parenthetically, this also led me to think a little bit about could or should we be doing this in other areas of scholarship besides science and technology? How would you even think about national humanities indicators, for example, and how to benchmark those? We certainly evaluate humanists, just like other scholars on the individual level, but I don't know that we pay much attention to it nationally. In any event, so the OECD discussion started around this kind of vexed area about scientific productivity and how to measure it. And then moved on to start talking about how AI is being used in various scientific areas, what difference it's making in terms of the ability to make discoveries and predict. Where it fits in the sort of scientific workflows and the extent, how it's coexisting with people when it's replacing work done by people, when it's enabling work done by people. And then finally, the workshop moved into discussions about how to make policy changes and investments that would increase the level of adoption of AI and presumably consequently improve scientific productivity. That's a very interesting set of discussions and they have made the video of the first couple of days of that available as of a day or two ago and are in the process of putting the rest of it up. I believe I shared out the pointers to the program on CNI Announce that I will be sharing pointers to the video. But let's go back for a minute to this question of individual productivity and in particular the pressures that seem to be getting worse and worse and worse on quality. The sense that overproduction and the pressures to overproduce in terms of publications are getting worse and worse. Somehow, and I have some speculations I'll come to in a minute, somehow it feels like the pandemic environment has taken all of these trends which were already well underway and made them much worse. I was very struck a couple of weeks ago when they awarded the 2021 Maddox Prize. Now, the Maddox Prize normally goes to someone for service on behalf of science. The recipient for 2021 was a person named Elizabeth Bick. I don't know how many of you are familiar with her work. She started her career as I understand it as a microbiologist and basically found her way into forensic discoveries of scientific fraud in publications, everything from reused or doctored images to fabricated data. And I would invite you to look at some of the interviews with her about her sense of just how much of the published literature and in what areas is problematic for various reasons at this point. And, you know, we're hearing other warning signs too. For example, recently there have been reports out of efforts to reproduce the published experiments in a number of what are viewed as foundational cancer research papers. And it's not going very well. You can argue, you know, that the way they tried to do the reproduction is imperfect, but these are these are unquestionably warning signals. The other thing that's striking about Elizabeth Bick is that while she was doing much of her work long before the pandemic, during she got dragged into. In a very nasty way, a lot of the debate during the pandemic, particularly around hydro hydroxychloroquine and whether that did or did not help to cure COVID. Including some nasty litigation with the head of one of the French virology labs that was promoting that drug. And that, of course, you know, shows how what originally was just sort of basic scientific fraud and malpractice is now moving into the political sphere where we're seeing active pressure on the scholarly record in various ways. Somehow, I'm not sure that that's going to go away. I think this is a real thing. One of the other things I would just note in the area of scholarly communication is that preprints have been a thing for a really long time in many disciplines. In fact, the math physics preprint archive that started life at Los Alamos and is now at Cornell, Paul Ginsburg's archive, that's celebrated its 30th anniversary just recently. That's how long those preprints have been a thing. But the area that nobody really wanted to go was about medical and public health preprints. Those, everybody was very nervous about and for good reason, because those affect people in many cases, the decision making of people who aren't necessarily in a good position to make their own assessment about whether the preprint is valid or not. Well, we certainly learned about preprints and public health in the pandemic, and certainly we learned a lot about the relationship, good or bad between responsible journalism preprints and public health. I think that's been a very interesting series of developments, and that's one I don't see going back. Remember, I was saying it feels like we're just overwhelmed with things? There is some evidence that paper publication levels are up, but the other thing that's happened is conference proliferation. I have a theory, actually, that physical presence was a sort of a rationing device in conferences before the pandemic. Basically, you could only be at a maximum of one conference at a time, so you don't feel inclined to spam every conference you can think of with some variation of your current work. In addition, you can only attend one conference at a time, but now that we can comprehensively capture video of every conference that's happening anywhere, I don't know how you all are feeling about video debt. But it seems like there's a ton of video out there that one should be watching. I'll come back to that in a minute or two. I want to just leave the discussion about scholarly communication with a couple of observations. The first is that open access, and in fact open scholarship more broadly, is not a panacea for misaligned incentives and other problems that are epidemic in our scholarly communication system. The other thing that I am very struck by the lack of progress on is we've spent 30 years, at least, coming up with better and better online discovery systems to help you find things, identify things that are worthy of your attention. What we've not done much to help with is the allocation of attention. We still really don't have systems that help you very much with making decisions like, you don't need to pay attention to this. It's not very important. This really is worthy of your time. You have very limited time. I feel like, particularly among researchers, there's an increasing demand for that kind of capability, and it's very striking to me how poorly we're doing at meeting it. Both at a delivered systems level and even at a more fundamental research level that might help us in future to design such systems. Let me talk about some of the new areas of potential innovation, and I'll talk about them both from the social, cultural pressure side and from the technology side. These are all speculative, but they're an invitation to be thinking about things that might surprise us. One thing that's clearly moving quite rapidly is various forms of the Internet of Things and sensors or accuators being attached to that network. A lot of what we've seen so far there is relatively mundane, but you're starting to see this move into an increasing number of areas. Healthcare, for example, more and more. I have a suspicion that we will see continued growth in this, and one of the things that's striking to me is how ambiguous the connection is between research and the Internet of Things. We see some researchers starting to deploy Internet of Things-based systems to gather observations. I think that's a really interesting development, which is going to drive quite a lot of technology. Think about, for example, the kind of use researchers are making of drones or seaborne autonomous sensors or things of that nature. One of the issues for the commercial Internet of Things, by the way, is that these are also a cornucopia of data that can drive various kinds of research in the sciences and in social sciences, and indeed, perhaps even in the humanities. But whether and under what conditions researchers will be able to get at this data, it strikes me as very much an open question. We've spent a lot of time worrying about the inability of researchers to get their hands on a lot of data coming out of social media platforms, but I think that what comes out of the social media platforms is, over time, going to be dwarfed by the broader Internet of Things. Another thing that's really interesting that we're just starting to really appreciate is what I'd call the geospatial singularity. Actually, somebody else used that term, and I just loved it. Many of you are familiar with the singularity where human progress and human knowledge just sort of spikes and transcends. It goes into a self-feeding pattern. This is the sort of thing that Ray Kurzweil and folks like that have been talking about for a couple of decades. Well, what's happened geospatially is that we now have enough observational satellites at low altitudes carrying all kinds of sensor platforms, and in most cases, not belonging to nation states but to private companies, that you really can watch almost any point on Earth almost all the time if you want to. You don't have to think about satellite passes, and maybe you get a look at this point on the Earth that you're interested in a couple of times a week or a couple of times a day. You can really watch for things now, and so much of this data is available commercially. For instance, here's one that's kind of a surprise that's popped up recently in the climate change context. So, you've all heard about carbon footprints, carbon dioxide footprints, and how we need to reduce the amount of carbon that's being emitted. Now, it turns out that another thing that's very damaging, even in relatively small quantities, is methane. And we didn't really have a good way of tracking methane emissions. It turns out that there are lots of things like pipelines and industrial plants that can leak methane. And we now have people watching for these leaks and plumes from these new observation satellites. And it actually looks like one could, and some people are trying to design some very targeted work on the reduction of methane emissions, which can really do quite a lot of good in terms of global warming. I think another thing we're going to unquestionably see with these satellites, when you couple it with more and more extreme weather events, is hopefully much better prediction and warning systems than we have in place today. So, here's another collection of technologies. There's a set of things, virtual reality and AR technologies, augmented reality technologies. EDUCAUSE likes to lump them under the umbrella term of XR. These have been bubbling along for a long time now. They haven't really seen huge adoption, although they have certainly found a growing number of niches in the educational processes and research processes. The equipment is getting cheaper, but the other thing that's starting to happen is that places are starting to go away. There's an awful lot of places that are in considerable trouble right now because of rising sea levels. And I think one of the things you're going to see, in fact, you're already seeing it as a growing interest in documenting places, places that we might lose. I think that this is only going to be exacerbated by the same kinds of trends that may be driving remote access to special collections. Travel barriers, especially internationally, concerns about the cost of travel. It's going to be increasingly hard for scholars and students to visit places that are important culturally, that are important historically, artistically. And I think we may see a substantial rise in investment in these kinds of technologies, which couple quite closely to AR and VR as the consuming side of these place capture technologies. There's a lot of work going on in quantum computing and quantum information science. I just want to flag that. I find that area probably the most difficult to assess in terms of where exactly we are on the technology evolution curve and when we can expect what kind of impacts. But it's certainly got to be on the list. The last of the areas that I'll throw out for potential innovation, I would shorthand as fair. The huge investment that's being made in the fair principles as they relate to data, particularly. And I would tag data discovery and reuse, particularly in the context of machine learning applications as a really growing and significant unsolved problem. Now, one way we're seeing this being solved is by the creation of the specialist repositories that I referred to earlier where certain kinds of data are just amassed. But they're not going to be specialist repositories for everything and it doesn't work for, that's not the right model for many disciplines and many uses. I think this is that really achieving the promise of fair is going to be very difficult. It's a wonderful slogan but people who've actually tried to do it know how hard it is. And I see that as a place where we may see some pretty high profile innovations coming in recent years. There's certainly the need. Now, let me conclude my comments by going back to the question of necessary innovations and which ones stick around. Right now, it feels like we're going through a whole rebalancing of what do we do virtually and what do we do in person. Certainly, I've spent a lot of time thinking about that for CNI. And last week and this week is our first, you know, obviously imperfect attempt to exploit the best of both worlds in a complementary and mutually supportive way. I think for us, that's the right thing to be doing. But different settings are going to need to find their own way on that. It's striking to me to look at what's happened in the broad area of education. I think most of the institutions here in terms of education have basically fallen back to let's do everything in person as soon as they could get away with it. I find that really interesting. I have heard some speculation that when the pandemic ends, perhaps we will just relegate it within a few years too. Bad memories that we don't think about and go back to doing education, for example, exactly the way we did it pre-pandemic. Now, what's interesting to me is that while I think that's where most of the institutions here are heading, that's not necessarily where higher education or K-12 for that matter as a whole is headed. When I talk to people outside of the research university sphere, when I talk to people at, you know, comprehensives, regional places, they're actually looking at more of a balance of maintaining some things online and doing some things in person. And I think it's going to be very interesting to see how that resolves, which institutions fall back to what they were doing before, which institutions actually genuinely try and find the right balance there. I'm also watching some interesting things in microcosm at some of those institutions that have mostly gone back in person. There's usually an asterisk in the, well, we're back to doing classes in person, and the asterisk takes you to a statement except for really big lecture courses. And some institutions are using this as an opportunity to think about, do they really want to do those big lecture courses in person? Or are those better handled as video with discussion sessions that are smaller? That will be interesting to watch, too. And so far, I would say, based on some anecdotal sampling, I don't know of any organized survey here, the results are ambiguous. Look at other areas, though. Conferences. We spent some time on this at the close of the virtual meeting last week. And it's really unclear what the right thing is to do for scholarly conferences. On the one hand, we have seen that virtual conferences, virtual scholarly conferences, bring a much larger degree of global participation, particularly if they're priced properly. On the other hand, this is very problematic for the finances of many scholarly societies who are invested in doing other things. It will be very, very interesting to me to see how different disciplines, different scholarly societies, and remember, scholarly societies and disciplines are related. They're not the same. To see how they find a balance and what that balance is. One thing I do want to flag, though, is that when you look at the amount of video that's being created, either by conferences that have moved virtual, or classes that are being recorded, that are being taught online, that were being taught online, or in these sort of things that sit between conferences and classes like colloquia. There's a lot of really potentially important video material being produced. The rights to a lot of it are terribly ambiguous and unclear. What happens when you take video of or carry out a departmental colloquia that happens once a week on Zoom? Who has the rights to that? Well, it seems to me that if you can figure out who probably has the rights to it, you can probably get permission if you want to collect it. And it's striking to me in the conversations I've been having with libraries over the past six months how little libraries are exploring this area. That's really, I think, a potential important missed opportunity. Related to that, I think if there's going to be more video and there's every evidence there's going to be more video, we need much better tools for skimming and navigating video, and we need to mainstream those tools. How people discover videos, how they find the piece of interest without waiting through hours is going to be really important to figure out. And that's another area where I think we're going to need some innovation. So that's the way I've been thinking about how the landscape is evolving. Those are some of the areas that C&I is going to be watching closely, and you'll see some of them represented in the meeting today, some of them in the virtual meeting, and you'll see more of them as we go forward in future. I've deliberately left a bit of time for comments and questions, and at this point I would invite those. Thank you so much, and it's so good to see you all. I'm Talai Alon from the World Bank. Thank you, as always, for this eye-opening session. I look forward to it every year when it takes place. Do you have any thoughts about the use of blockchain technology for information networks or data networks? Thanks. So this is really interesting. Blockchain is actually very confusing because there are two flavors of blockchain. There is so-called permissioned and permissionless blockchain, and permissionless blockchain basically is very closely linked to cryptography. Permissioned blockchain really is just, it's distributed logs in a certain sense, which we've had for a really long time already, and it's just a way of thinking about formalizing these between organizations and into the public sphere. I think some of that may have a significant payoff, but I think disentangling this from all of the hype and framework around cryptocurrencies is quite a major challenge. I also think that we need to be quite judicious in thinking about where these actually genuinely add value, but it's certainly an area to be watching. I think we had another question. Hi, Lorcan Demsey from currently with OCLC. You mentioned incentives. Just be curious to know whether you have any observations given recent changes about what you might characterize as the research reputation industrial complex. So you have three large companies developing research knowledge graphs and then associated management workflow and analytics services on top of those in terms of Elsevier digital science, obviously part of Hold Spring, and now Clarivate as well with an enlarged group. So just curious what you might, that research reputation industrial complex, then in relation to some of the other things you spoke about, given the alignments that you were saying might be misaligned. That's a great question, and it's one that worries me a lot. There is huge interest and some, frankly, some vested commercial interests, as you say, in trying to reduce evaluation of research quality to this very mechanistic process of counting things. And counting citations, counting publications, et cetera, et cetera. And even the efforts at introducing alt metrics seem to be going down the same path. They're just trying to find different things to count that are less tied into the existing system of scholarly communication. And I guess I am increasingly of the opinion that while doing some of that kind of counting is probably useful in the kind of national science indicator sense that I was describing earlier, or maybe even at the level of trying to assess the relative impact of one major university compared to another. I am increasingly uncomfortable with it at the level of evaluation of individual scholars. To me it's, and maybe it's just that I'm getting old and pessimistic, it's too easy to try and reduce it to counting. There is an inevitable subjective set of judgments that need to be made, and people hate to make judgments. It's much easier to just throw things into a formula and let that tell you what to do. And I think there's been quite a bit too much of that in recent years. Other questions or comments? Yeah, hi, Cliff. I have lights right in my eyes, so I can't see a thing. I know what you're talking about. We're testing. Hi everyone, Jane Greenberg, Drexel University. Something that's been on my mind a lot is the role and thinking about the role and responsibility of major libraries and institutions represented here, and the role and responsibility in terms of AI readiness. And I say this because there's a lot of good data, but we all know there's a lot of bad data, there's a lot of mixed up metadata, and so forth that we've been creating and we've created all these repositories and so forth in digital projects all before this whole AI buzz. So in talking about the roles and responsibility of major institutions to support AI readiness and how does that interact with the services of these institutions? I'm thinking about this. I don't have any answers. I would love to hear your thoughts. Thank you. So this is a little bit of what I was short-handing with FAIR and the set of innovations around FAIR, which get it exactly that. And the thing that libraries, I think, are particularly going to find themselves struggling with is that on the one side you have researchers and others prospectively creating new data today that needs to be described and structured appropriately for that. That's one kind of a problem. The other kind of a problem is the one that you can sort of short-hand under the collections as data issue, how you take existing collections, particularly special collections, and you make them available and describe them in forms that are hospitable to various kinds of machine learning, which feels like it's sort of a different problem, but a very real one. One of the related problems there is that we really, for all the talk about machine learning, we still don't really make it easy for end-scholars to do that kind of thing with a collection. I think we're going to see a generation of tools in the next few years that really are going to start opening that up. And as those tools start coming along, I'm hoping that will give us some additional guidance in the way we want to do those kinds of descriptions. I wish I could give you a more concrete answer to that, but they just, at least I'm not seeing anything other than a lot of experimental projects at this point, which are informative but in most cases hard to scale and hard to generalize. Thank you. Hey Cliff, Gina Seizing, Bryn Mare College. So good to see you. Thanks for your reflections and questions and hypotheses. You mentioned two trends that are kind of at odds with each other in certain way. One is all of the data that's coming through sensors and instrumentation in that Internet of Things family, things that are often sold by vendors and coming on to our networks in service of particular kinds of research that often scientists are doing. And then you also mentioned the sort of increasing threat terrain and what institutions are dealing with around fiber security. And often those devices are serious vectors for us in terms of those cyber threats because they often go unpatched. They often go even unmanaged generally speaking. And so I'm thinking back to your frame about how we innovate not just at the institutional level but at the community level. And wondering if you're seeing anything that takes us out of just a very localized instantiation of the battle to stay ahead of cyber criminality at local spaces to any kind of incentive program or advocacy or innovation happening more at the national level that would hold the vendors of these IoT devices and software entities more liable for attending to security. So anything in that space that you've been seeing? That's a great question. I mean, I've been following the discussions among the networking folks at our institutions about what to do with all these wonderful devices that the students bring with them to the dorms now. And it seems like the solution in many institutions is you basically treat the entire dorm network as this sort of swamp that you carefully fence off from anything important. You basically figure it's just this gigantic compromised mess. And that may be actually not unrealistic given the current state of play of many of these IoT devices. Although if that in fact is an accurate assessment and these are pretty smart people making this assessment, this should not leave you with a warm and fuzzy feeling about what's going on in your home. Particularly if you don't know how to fence off separate sub networks of your home network. I would be very, very nervous and unfortunately it's getting really hard to find appliances and things that don't want to get on the internet and be insecure. But to go back to the sort of collective questions, I think that there's already quite a lot of good thinking among universities and among the research community about how people should do internet of things systems that are responsibly designed. I think most universities who are buying these things institutionally probably are trying to do those things. I think the problem with the consumer stuff is that really the universities can't fix it. They're such a tiny proportion of the market. The place where that's going to get fixed is probably legislatively and you know from my point of view it can't come soon enough. It is interesting that the Computer Science and Telecommunications Board of the National Academies did do a session probably two to three years ago looking at essentially responsible design of distributed updates. Updating of highly distributed software such as you would find in internet of things kinds of applications. So there are best practices out there. It's just that the commercial folks generally have no incentive to follow them because they have no liability and they add to their cost. I think we could take one more question or comment if there is one. Sure. Thanks Cliff. This is Rob Hilliker from Rhone University and so I was just struck at the moment in your talk where you addressed the difference in the use of machine learning and the humanities and the sciences and particularly in the sciences the focus on predictive analytics and the absence of that in humanities and it set me to Googling because I knew there was a project and I found it. It was a project Cassandra that had tried to use the analysis of narrative literature in order to predict civil unrest in different environments. I think it's interesting to look at that and it maybe raises some of the issues as to why the humanities typically are not using predictive analytics. But I was curious to approach from another direction and ask you whether you think there are mechanisms or ways of encouraging that kind of use of machine learning and humanities. In other words are there applications of predictive humanities research that might be of benefit to encourage. That's something I would love to hear more about particularly in the things that are firmly in the humanities category. For time reasons when I give you that sort of caricature contrast between AI in the sciences and the humanities I left out the social sciences as some of you might have noticed. And I did that because there's almost too much to talk about there. What you've actually got in some social sciences is you've got people who really are genuinely now trying to do predictive social science. They are trying to do it in some cases with proprietary data. I mean if you think about what the social media platforms are doing that is a kind of a nasty form of predictive social science that's being driven off proprietary data. And arguably it works pretty well for their ends. I'm not going to get into a discussion about the ethics of it but you are certainly seeing that. People who can mine social media streams certainly can use them for some predictive things and are using them I think for some predictive things. Whenever you get into broader predictive efforts in social science particularly you immediately get into a very complicated set of issues about bias about quality of data about what objectives you're trying to fulfill what ethical things. So you are seeing some of that but it's a very wary terrain to navigate. I know that the intelligence community has been fascinated by this question of can you use things like news reports in a predictive fashion. And they've certainly been funding work in that area for oh gosh probably at least at least 50 years probably more. And my impression is that most of it has not been terribly successful although you'd probably also if it was very successful we probably wouldn't be hearing about it. And I suspect that most of the successful stuff tends to run more on the tactical side than the strategic side. In other words you probably can't predict a rebellion two years in advance in country by machine manipulation of the news. On the other hand you can do and they do do things with tactical signals intelligence and things like that that that are quite predictive. We've gone a bit past our time but that's OK because we have a generous break before the next session starts. But at this point I'd like to draw this session to a close. Thank you so much for indulging me in these reflections. I hope that they are helpful to you in your thinking about where we should be going from here and where your own institutions should be going from here. I really would urge us to look for opportunities as we examine the landscape for those things that are either institutionally or as a community transformative rather than incremental or necessity driven. Those I think are going to be where the future waits for us. So thank you again for being here today.