 Okay, I'll just quickly say something. So welcome to this session. First one of the afternoon sessions, we have two half an hour sessions. So what we'll do, or what the presenters want to do is take about 20 minutes to talk and then 10 minutes for questions right after each of the sessions. Is everybody happy with the session to be recorded? Yes, just to double check. Great idea. I think I'll give it to you, a way to you, Igor. Thank you very much. Good afternoon, everyone. My name is Igor Lesko. I come originally from Slovakia, but I've been living for the past 22 years in South Africa and that's where I'm coming from for this particular conference. I'm from Open Education Global, which is an international member-based organization dedicated to opening up education worldwide. And on that note, I think I'm compelled to say that I hope to see many of you at the Open Education Global Conference at Monter in October this year. But today I'm also wearing a different hat. I'm also a PhD candidate and a member of the Global OER Graduate Network. Global OER Graduate Network is a network of PhD researchers working on different topics related to open education, policies included. They are back here in the audience as well, who is part of the management team if you want to know more, please speak after the session. And today, this presentation is part of a bigger PhD research study or research that I've been working on for some time now exploring how international organizations have influenced the development of governmental OER policies from 2002 until 2019. The goal of this particular presentation today is to provide an overview of the OER policy instruments that they use by four intergovernmental organizations to influence the development of governmental OER policies, highlight the similarities and differences in their approaches, and then also report on their perceived or observed successes and challenges with their efforts. I already said what the main research question was. So I obviously have carried out a literature review, but more importantly, what's interesting for you to know is that I conducted a semi-structured interviews with international organizations and also with governmental policymakers and advisors and experts in different countries. So the data collection process was structured into two different phases. The first phase involved international organizations and that included all together these players, so in different groups. So intergovernmental organizations, the four of them, UNESCO, OERCV, Commonwealth of Learning and European Commissions, and four international and governmental organizations or associations, which was Open Education Global, Creative Commons, OER Africa, OER Asia, and also foundations, which was U.S. Foundation, Shuttleworth Foundation, and the Open Society foundations. There was a rationale for including foundations as well, which is not something that I'm going to be speaking about right now, but they are included nevertheless because of the activities in the space since 2002. During the interviews with international organizations, I mean, this on average lasted about two hours, one and a half to two hours, two respondents per organization, I focused on the rationale, so why they're actually planning to or why they're trying to support the development of governmental policies as one of the lines of action. What kind of OER policy instruments they applied and what are the sort of organizational priorities with respect to these instruments? What are the sort of collaboration patterns with other international organizations in the space? They're perceived to observe successes or challenges and their future intentions. And because the OER policy instrument is a term that I use throughout the research, just to give you a little bit of a definition. So OER policy instruments, as far as they are defined for the mind, PhD research are multiple mechanisms of influence by international organizations that can directly or indirectly affect governmental OER policy processes. And by governmental OER policy processes, I'm talking about the sort of policy cycle. So if you're thinking about agenda setting, policy formulation, and nutrition implementation, and so on. And these mechanisms of influence here are referring to discursive dissemination, standard setting, financial means, coordinate functions, and technical advice. And I will explain what that means in subsequent slides. So this is just to give an overview of the first phase of data collection. And briefly, I'm also going to tell you about the second phase of the data collection. So then you have got the most great picture about what I did in the research. So during the second phase, I have conducted interviews with governmental policy makers and advisors in 35 different countries, states, and provinces. This was close to 40 interviews. It was 11 countries in Europe, 11 in Asia, four in Africa, four in Latin America and the Middle East, and then two in North America, which includes the United States and Canada, three provinces in Canada, British Columbia, Alberta, Ontario, and the United States. It was Washington and California states and the Federal Department of Education. During these interviews, the focus areas were on the governmental OER policies that were adopted in these respective countries, and the perceived influence of international organizations on these policies that were adopted. Also the perceived effectiveness of the policy instruments that are used by IOC in this regard. And also, like I was asking about the recommendations. So in what way could IOs or international organizations support these processes more effectively? This is not what I'm going to be focusing on during today's presentation, but it's just to give an idea of what happened. All right, and just to kind of contextualize this behind the literature or the theoretical underpinnings, this particular research is positioned within the literature that really examines the effect of globalization on educational policy processes. The main argument there is that public policymaking is no longer confined within national boundaries. And the policy makers are increasingly interconnected with a range of different actors beyond the nation state. So this could be international organizations or other policy networks. And then the policy text production is increasingly affected by globalized discourses and process of globalization. And this is a phenomenon that is often referred to as a global education policy field or community. Now researchers did lots of case studies in this area. Revealed that international organizations affect higher education policy at global and national levels in different ways. So for example, they engage in discursive activities. They construct and disseminate ideas about higher education issues. They also assemble and maintain transnational policy networks. They also sometimes prescribe policy directions which can have both binding or non binding for the states and then monitor compliance. They provide funding. They provide direct or indirect policy or technical advice, technical assistance as well. So there's a range of actions that IOs use to realize their intentions. And all of these that have affect governmental educational policy processes as I have already explained previously, gender setting, policy formulation or implementation. So overall international organizations really have five main categories of policy instruments in hand that they can adopt or apply to influence governmental policy processes. This would include discursive dissemination, standard setting, financial means, coordinate functions and technical assistance. I'm referring to them as OER policy instruments. This framework is based on the work of Jacobi and Sanagian that I've also then adopted for my PhD research. Discursive dissemination in terms of activities. Here you can think about, again, these are IOs construct and disseminate ideas or call for support through their activities which could include policy reports or proposals, various kinds of case studies or publications. They also organize conferences or meetings or networks. They also issue policy communications which could be declarations or various kinds of statements. And I will provide some examples for each of those subsequent slides, the findings slides. In the standard setting instrument as I've already mentioned is these include conventions or recommendations, for example. I think that most of you here should be familiar with the UNESCO recommendation of 2019. Is there somebody who's not? Okay. So this is pretty much the only standard setting instrument in relation to OER that was adopted by an IGO or Intergovernmental Organization. Then you have got financial means and that's really mostly provisioning of funding. The European Commission, for example, has been doing a lot of that. The coordinate functions, that's the monitoring of the policy compliance and it's normally attached to standard setting instruments. So this can be both formal or informal monitoring procedures. So again, when we are thinking about the standard UNESCO recommendation of 2019, the monitoring process there is related to the fact that the member states need to report on the progress with the implementation of the UNESCO recommendation every 45 years. And this process is currently on point because it's now, it's been more than four years now. And the technical assistance that's various capacity building activities that IOS can engage in or they can also provide technical assistance in a sense of direct or indirect policy advice. So direct policy advice will be speaking to policy makers. For example, provisioning of model policy documents and the like, providing analytical advice. Indirect policy advice is typically incorporated within different types of publications that IOS produce. So in terms of results, right? So again, just coming back to the original questions like what kind of OER policy instruments the selected intergovernmental organizations applied to influence the development of governmental OER policies. The international activities in this space ranged from constructing and diffusing ideas about OER and related policies to providing funding, technical assistance and then also adopting standard setting instruments. But in this particular case that only refers to UNESCO. And just to give you some of the examples. So under discussing dissemination is one of the OER policy instrument category. The main focus there was on constructing and disseminating ideas about OER and related policies. They also facilitated exchange of knowledge and practices through various OER policy proposals and communications, publications, policy reports and advocacy. They also organized international conferences, meetings, networks, fora of all kinds. And just to give you an example in terms of the policy proposals or communications, you can see the Paris OER Declaration of 2012. So that's a policy communication and discursive dissemination. All the regional forums that were organized prior to the period leading up to the actual adoption of the Paris OER Declaration. In 2013, it was the opening of education communication by the European Commission. Another example is the various surveys, guidelines and publications that were done by these IGOs collectively between 2010 and 2019 specifically. Technical assistance as another category of OER policy instruments included the provisioning of indirect policy advice through publications and direct policy advice to governments and educational institutions and then also capacity building activities and the provisioning of technical infrastructure. Here the example would be, for example, UNESCO and Coal following this policy communications from 2012, they engaged in a prominent policy advice in different countries and they often collaborated together in different countries. The indirect policy advice was incorporated between different publications. For example, guidelines on the development of OER policies which was issued in 2019 or the OECD in 2015 issued another publication, OER catalyst for innovation, which is currently actually dedicated to policy. Then you have got going open policy recommendations on open education in Europe, but that was issued by the European Commission in 2017. Here I have to actually qualify, qualify that European Commission is not just looking at OER, but they're looking at open education more broadly. And so in that context, not just about OER policies but open education policies of which OER is a component. This is just a snapshot of examples. The range of instruments, examples is pretty much spans across like three or four pages in my results, but it's just a given idea. Under financial means instruments, again, this includes providing funding for projects and initiatives. As example, it is the European Commission, the Erasmus Plus funding, which has a significant amount of money available for different initiatives related to OER. And as far as the standard setting and coordinate functions instruments are concerned, much of what's going on, there is a update. I think it's going to be. I think it's the entire university totally to send the Erasmus to you. Oh, oh. Not the mic. Yeah, the period of the moment. And so the standard setting and coordinate functions instrument that includes again, the OER recommendation from 2019. So these are the examples of the OER policy instruments that the selected IGOs applied during this period to influence the development of government OER policies. In terms of similarities and differences between these IGOs, as you can see here, on the left hand side, you can see the type of OER policy instruments. And going to the right, you can see like which IGO used, which OER policy instruments are. So here you can really see that the selected IGOs collectively used all OER policy instrument categories, although like each IGO could not use all of them. So again, as an example, only UNESCO uses standard setting instrument in this arena. And those differences are dependent on the mandates of these organizations, the legal arrangements, organizational priorities, which can be changing as well. So that's really dependent, context dependent for each IGO. And for example, I don't know, European Commission does not really have the mandate to prescribe policy direction in education. It's got the support competence. OECD rarely does this as well because they really focus on the power of persuasion, so of peer pressure, rather than describing policy directions. So in terms of the organizational priorities, as far as this selected OER policies, identified OER policy instruments are concerned, between 2012 and 2016, is IGOs classified most of the OER policy instruments as having a major intermediate organizational priority. During this period, they all expanded their activities on disseminating ideas about OER, right? And this is specifically true following those communications policy proposals of communications from 2012 onwards, right? And I should also maybe just say that the OECD, there was also an intention or there was a proposal for like a stronger instrument in 2011, for potentially a declaration or a recommendation on OER, which was not successful, but the positive outcome of that was that it restarted the line of investigation into OER, which then ultimately led to, because policy makers were asking for more evidence about some of the claims related to OER, and so that's what then happened. And then in 2015, they led to the publication of OER catalysts for OER patients. Again, and so following this policy communications in 2012 and 2013, UNESCO and COAL started to focus prominently on providing direct policy advice to policy makers around the world on capacity building activities and then evidence-based advocacy, European Commission focused on applying funding OER policy instruments, technical infrastructure and research activities, and OECD as a consequence, as I already mentioned, they expanded their activities on creating ideas about OER. So these are some of the similarities and differences as far as these organizations are concerned. Now perceived, thank you. Now perceived are observed successes and challenges as far as the successes with the efforts. So here you can look at two different scopes. One is an international scope and then the other one is a national scope, government lawyer policy-making processes. So the respondents identified as far as the international scope is concerned, the respondents identified their OER policy proposals and declarations in 2011 and 2013 as managing to set international OER policy agenda and also it led to additional lines of actions for these IGOs, as I already mentioned, right? Which was a positive development. And in terms of the scope, the national scope, so here we are talking about the perceived successes of their actions on national policy-making processes with respect to OER, right? So the first example is the agenda setting, which is the first section of the policy cycle. So here, again, the respondents found that their policy proposals and also that the international regional consultations, forums and conferences and meetings that they organized that managed to set national OER policy agenda for policy makers. But the one thing that I should also mention though, that they could only establish these kinds of successes through inferences to a few national policy texts where the instruments were referenced or, you know, but they didn't really tell them much, you know, or it was through some sort of anecdotal feedback. And that's what the second part of the research that I told you about as far as those interview with government or your policy makers are concerned are so important to actually see how they perceive the effectiveness of those instruments, right? And as far as the policy formulation and implementation stages are concerned, the respective IDOs felt that the direct OER policy advice that they provided to different policy makers and the capacity building activities led to the development of national OER policies or support implementation processes of the developed or adopted national OER policies, right? And then finally, in terms of their, I'm seeing, okay, let me try this. Think like the computer for us. Luckily. It's fine. Thanks. Just have one session. Oh, no, no, no. Okay, so my only story is five plus two. Five plus two, so I'm still left then. Yeah, I'm saying a little bit. That's this one. There you go, thank you. Okay, these are the last two slides. I still have 10 minutes left because I'm starting five plus two, so I'm still good. So as far as the successes and challenges, I mean, as far as the challenges are concerned, the perceived challenges with their actions, this could be grouped into different categories of challenges and they all have like specific outcomes or consequences as they describe them. So here we are looking at organizational mandates, for example, as one challenge or changing organizational priorities or new emerging trends in education. These are some of the challenges that the policy makers or the representatives of IDOs described and their consequences that limit the ability of their respective IDOs to adopt legal OER policy instruments, as I've already explained earlier, to discarding policy directions for member states or to affect the continuation of their work on OER in general. The second group of challenges are related to lack of data or evidence about the uptake and impact of OER and open education in different countries. So researchers here take note, please, because this is something that is really a continued need for information for policy makers. And then there are even OER developments in different countries. And so what the consequence of this is really is that IDOs have got lack of clarity about the appropriate OER policy instruments they should apply, that's one. And the second is that it actually requires very nuanced and differentiated approach to policy advice in different countries, which is fine, but it requires resources. It's not something that IDOs always have. And then finally, I think this is nothing new, perhaps, in a way, those who are familiar with sort of policy work. So changes in government, bureaucrats, governmental priorities, that was another challenge identified, or misconceptions about copyright, open licenses, infrastructural challenges in different countries, or the lobbying activities by the publishing industry. So all of these need to have their own sets of consequences, for example, that the developer OER policies were not implemented or the developer OER policies were not addressing OER. And then I just want to think about the learning activities. Yeah. And that's about it for now. Thank you very much for your attention. I think we have a few minutes for questions. So a lot to digest, I know. And you make it good. Let's go ahead. So in the national policies that you looked at, what are some examples of some countries that have particularly interesting or exemplary national policies? I would be hesitant to say which ones have got exemplary ones. Because I was not actually doing analytical work about policies themselves. I'm merely looking at the inferences or the role that international organizations and other stakeholders or advocates play on these kinds of processes. And in what way they can actually make the processes more effective in open, right? But on that note, I'm looking actually, as far as the policy, governmental policy is concerned, I think that it also requires a bit of a more nuanced way of talking about policy. Yeah, because everybody keeps asking what the heck is policy? Like is this policy, is that policy, is funding policy? And I think that looking at it in terms of policy instruments is a better way of understanding like what governments are actually doing. So if you are looking at kind of stick summons, right? That these are all specific policy approaches by that the governments can take with respect to a year of an education abroad, right? So they can prescribe direction in a certain way. They can provide funding for specific activities. They can issue communications. So this would be the summons. Another example would be, I don't know, the NATO framework, not the military NATO, but which is modality instruments, authority instruments, strategies, organization instruments. So when people are looking at policies and trying to analyze them in different countries, I think they should kind of try to take the range of these instruments into account. Because I think often people look at like, is there a legislation, which is rare anyway? But as far as like some stronger policy statement, you should have a more differentiated approach to how you're looking at these different actions that governments are taking. And I could probably point you to a lot of examples that are of policies that exist out there that have a lot of room for improvement. I think... I think... But, yeah, but of course there are some successful challenges. And actually, Robert and I were speaking about this during lunch break, there are certain governments that keep supporting the movement over a sustained period of time through different needs, adopting different policies. Truman and Netherlands is one of them. Since 2006 and up to 2013. And I think the question there is, like what's going on there? Like why does the government continues to support or are open education related efforts for the justice? So I think these are the kinds of questions we can maybe start to explore in terms of when we are looking at successful policy approaches in different countries. I think British Columbia could be another one, I think. Their government has been providing funding for open education related efforts for a number of years now. So it continues to be on the agenda of policymakers. I think the question is why? I think this is something that we need to start looking at in terms of successful examples and what's working there and why. I think... Well, have you noticed and maybe had any correlation between countries that have these policies and countries where students don't have to buy their own textbooks? No, that's not something that I'm focusing on. So I would refrain from commenting on the table issues right now. And also remember my geographical spread is really wide and the policy context is really different from one country to the next. So yes, I know in the U.S. Canada, the focus is very strong on textbooks. In other countries, the policy context is quite different. And that also requires a different analytical approach is course analysis and the like. These kinds of documents. I retract my question. Do you see any sort of influence on the amount of sustainable development goals? Is that sound to feed into the talk process? In terms of... In terms of implementing the direction of policy. I mean, you can have a policy from a social justice perspective or from a business perspective. You can both want OERs, but there's a different underpinning ideology, if you will. Yeah, certainly the sustainable development both play a role in there. And also the IGOs are contextualizing their actions or justifying the further involvement with this space in the context of sustainable development goals. And where you see new developments, I think in Germany, when you look at the latest OER strategy, for example, I think sustainable development goals are featured within that specific policy document. So I think that overall, maybe just a part thing I would like to say here is that when we are doing this kind of work, when we're trying to convince governments to adopt policies, or supporting policies for our open education, I think it needs to be kind of aligned with their priorities, their national priorities. It really does require a more of a differentiated nuanced approach to what we're trying to do. So it's easy to have some model policy documents, right? And that's helpful. But I think that in terms of justify and why government needs to do this, those kinds of genetic arguments like do this because it's the right thing to do, it just doesn't resonate. It just goes blank, you know? So it does require more work on our parts also, whether you are an advocate or whatever your role is and you're trying to convince your policymaker to do something about it, like just do your homework before you study at work. Yeah. Just looking at this from an outsider looking in, is there any research or information that shows which countries are the champions for implementing policies? And would that help? Because the biggest opposition is the lobbyists against it that are about the printing of learning materials. And from an advocate standpoint, from everybody at this gathering, the lobbyists saying these countries are champions at it and these ones are excelling. And that would help the politicians want to be in that competitive race to be equal to them and help them push it to the next level. Those countries are selling, yeah. That's a tough one. I think in many countries, you do have to look at your contracts and you can even fit right to higher levels. And so it's very difficult to kind of step away from it. But I would say maybe ask for a little bit of patience due to answer your questions because at this particular moment, as I said earlier, the UNESCO is embarking on the monitoring process with the UNESCO recommendation. And I'm not sure if you read the document, but there are five main action areas. And one of them is about calling on governments to create supportive policies or policies. And so they're actually supposed to also report on these action area. And it's a survey instrument, there are always issues with survey instruments, how reliable they really are and who is filling it and all that. But in the next few months, I think we could have a fairly good overview of the most recent developments in this regard. So just as far as I know, that should be presented, the report should be presented to the General Conference in October and it should be available short afterwards for public consumption. Great, I'm afraid we'll have to cap it here but another round of applause. Thank you. Oh, I forgot to say that the code here, just in case you're wondering why the penguins are standing here on the table. Globally, our graduate network has got the penguins as the mascot, as the official mascot. So if you see any penguins floating around in the space, you can't pass the space, or you know why. Thank you for your attention. Thanks. Over to you. Hi, everyone. Thank you for joining me this afternoon. Today we're actually transitioning the conversation a little bit, we're talking a little bit of like going against the policy, I guess you could say. So a different kind of route in terms of conversation. So the topic of the conversation is opening up research through self-archiving practices. My name is Inilda Romero Hall and I am currently at the University of Tennessee Knoxville and I am the graduate coordinator of the Learning Design and Technology doctoral program. My co-authors who couldn't be here today are Josh Rosenberg who is the coordinator of the data science program at our institution. And he's also in the learning design space. And last but not least, George Bellachianos who is a full professor at Royal Roads University in British Columbia, Canada. And he is the Canada Research Chair for innovative teaching and technology. I always joke with him because I forget his really long title. So the plan for today is to talk a little bit through the introduction of the topic, then discuss a little bit of prior research that has been done related to self-archiving and the practices of sharing that scholars who in education and then discuss a little bit of our work in progress research that we have going on. Some of the results that we have had so far if you're still going through the validation process of our results. So I'm gonna talk a little bit about that and then just having a little bit of feedback and discussion. So I should start by saying that this work was actually inspired by the work of my co-authors in 2017, George did a presentation at a different conference, AECT, and he talked about whether researchers make our research available to the public. We are, all three of us are in learning design and technology and in this particular case, George was talking about how instructional designers and learning designers rely significantly in the research and scholarship that faculty publish, right? And if that scholarship is not available or freely available, it just limits their practice. So he did some initial work and was able to find out that 48% of research publics was also available to the public. So it was kind of like an initial part of the research. Then just last year, my colleague, Josh Rassenberg actually proposed to the Open Education Conference this topic of academic, how academic journals are shared in the open. And it was a very small sample of 50 journal articles that he selected randomly and he looked at whether they were available openly and he was able to find that 68% were available in their final published form, only two were available in their pre-print form. He was not able to present this research which we were really shocked and he wrote a blog post about it and this is George and I reacted to that blog post and that's just how our collaboration sort of got started. So overall, we can say that policy makers and other entities, our colleagues in universities are constantly encouraging us to search, to share our research efforts in a public manner. We have seen a significant amount of this when it comes to granting or funding agencies. So in the United States, one of the biggest funding agencies is the National Science Foundation. There are other big funding agencies in Europe, Canada as well, and one of the biggest part of grant funding is writing your dissemination plans. And a huge part of those dissemination plans is having some sort of open educational resource available for everyone, right? But there are various systemic issues that we are facing in academia that really prevent us to engage into this open scholarship practice, right? First of all, we have the issue of focusing on this prestigious journals. We are always pushed as faculty member, I am too, to go for that R1 or, I don't know what event school journals that we have to publish in because that's a criteria that our institution wants for us. And then we may be a scholars in different type of stages. So if I am an assistant professor or I'm an associate professor, I'm a full professor, I can make various determinations as to where I want to publish. If I'm gonna be judged by a tenure committee, I have to sort of like rely on what that tenure committee is what I want for me, right? So that raises some sort of, I guess, systemic issues for scholars. Then there's a tenure process and the unequal access to open access journals. So in some instances, there may be a fee that you have to pay. So for example, AERA, the American Educational Research Association, which is one of the biggest professional organizations in the United States has a journal that's called AERA Open. Their publication fee to publish OER is like 300, 400 dollars. So where are you gonna get that money to publish in that journal? That's just one example. There are others that charge even more. So again, issues that challenge whether we publish open or not. Looking at some of the limited literature that we were able to find on this topic, we were able to identify this publication by Perkins and Lowenthal in 2016 that show that as scholars, we primarily focus on the journal that we're gonna publish in based on whether, what's the aim of the journal? So if you ask any scholar, where should I have this piece that I published that I want to publish on feminist, pedagogy, and all learning environments? Where should I publish? They're gonna tell you, you have to research the journal, you have to look at their aim and that's how you determine where you're going to publish. But Harley would say, look at the aims of the journal and also consider looking at whether it's gonna be open access or it's going to be restricted to just individuals who have access to databases. So that moral and ethical concern is really not there as a main issue for scholars. It's really like a second thought or something that we think about later on, dissemination and being open is not the number one priority. We see civil arriving as sort of like a way to help mitigate this issue of publishing in journals that are restricted, right? So myself, George and Josh all maintain our own websites. And one of the things that we are often told, all three of us is you're so generous for sharing your words so widely in your websites. But we want to make sure that our scholarship is available to all. We want to make sure that individuals who do not have access to a database are able to access our work. But prior research has been conducted and I want to highlight the fact that this prior research was done in 2010. So it's been quite a long time since this topic has been looked at. It's a touch screen. Oh, is that what it is? Okay. So it's been quite a long time since this topic of civil arriving has been looked at and what they were able to determine back in 2010 is that civil arriving was not something that everyone does. In back then about 20% of scholars practice civil arriving. So I'm curious to know based on those who are here, how many of you practice self-archiving? Okay, so that's some level there. That's pretty good. Thank you for doing that. So with that, we decided to move forward with our research, which is, again, working progress. So we took some of the work that Josh Rossenberg had already started in which he selected 50 journal articles on random. And instead, using his data science background and art coding and other things, we continue to look at the journal articles that are published by ARA, which are seven of them. And the first one that you see right here is ARA Open, which is the one that's, you have to pay a fee, but it's published open to anyone. All others are restricted just access to institutions who pay their fee. What we did is that there was a third party that was involved in the collection of this journal articles. And we accessed Google Scholar as the main database to look at the journal articles. There was some programming that is involved in there. And we used this SIR API in order to search Google Scholar and access all the journal articles of all those seven journals to determine how many of them were actually available freely on the internet through Google Scholar. So I'm gonna make the slides available, but if you want to take a look at the code or access to the code, Josh has very graciously posted the code in his own blog. And the process was actually a little bit painful. He would go try the code and then we would meet. We will look at the output that was generated from the code and we would make some determinations of things that were new or random. And then we would go through the coding process again. So what we were able to determine was that from 2010 to 2022, so we decided to select 2010 because that was at that point where in the literature they had said that 20% of journals, 20% of scholars self-archived their work. So we selected 2010 and we waited until the very end of 2022, so we ran this data earlier this year. We were able to find a total out of 3,055 journal articles, 21 and 26 were available to reserve archiving processes or methods. So a total of 70% of the journals that are published are actually available to reserve archiving methods. So that from 2010 to 2023 has increased tremendously. It shows that scholars are very interested in disseminating their work and sharing it with the public. We were able to determine through the code, sorry, let me get through this side. We were able to determine through the code for each specific journal how many articles were available. We left ARA open in there because we were aiming to have out of 539 articles, have 539 articles available, but someone there was like a discrepancy there. We're still kind of working through that, but overall you can see that there is like a variation in some of the outcomes, but not a huge discrepancy based on the different journals that are published by ARA. Another important element for us as part of this research was to see if there was a change through the time period. So perhaps we were gonna see this number be a little bit smaller here and perhaps grow a trend and growth, but the reality was that there was not really a change through the different years throughout the number of articles that have been shared have stayed pretty similar. Again, not huge discrepancies. We also thought that maybe in the year 2022, we're not gonna see that many articles that were going to be shared because it's such a recent year, but that didn't hold again. Articles were shared widely. And then we looked at specifically where people are sort of archiving their research. And of course, Research Gate is right at the top. Then we also saw quite a few publications through Sage Pop, academia.edu. Although we were, we had quite a few issues with academia.edu, we were not sure if it was because neither of us, none of us have an academia.edu account or there was some other issue with it, but maybe that could be it. And then this were the remainder of the sites where individuals were sort of archiving their work. I changed my function. Sorry? You said self-archiving, and I thought you meant having your own website. I don't have where I use Research Gate. Do you call that self-archiving? Yes, absolutely. I was sorry, I didn't understand. Yeah, so self-archiving really is any practice that allows you as an individual to search or share your own work, whether it would be your own website or using a different tool like Research Gate or others. I miss them, so I thought you meant this for yourself, you had your own website. Yeah, thank you, I appreciate you. I don't suspect more people than we have touched on and. Okay, how many of you, no, I'm curious to know how many of you use Research Gate? Okay, so, okay. Thank you, I appreciate you. Thank you, it's well called. Sorry? Anacademy.edu. Okay. I'm sorry, I'm sorry. So knowing that I suspect a few people might have- No, I appreciate you mentioning that, thank you. So in order to determine whether our programming was actually working the way we want it to work, we engage in a validation process of a hundred links because of course we can do the programming, share the results, but is it really the output that we want or is there something else that it could be giving us? So for the validation process, out of those 3000 plus articles that we were able to gather, we engage in a validation process of a hundred links, randomly selected. And that actually helped engage the assistants of our research assistants. So we did the validation comparison of the manual URLs and I'm gonna show you the spreadsheet here in a minute. So literally someone entering in Google Scholar the name of the article, do we get the exact same websites? And then we also did the validation to check what was behind those websites. Are we really being taken to a PDF of the final version of the journal or a preprint of the journal, right? Can I switch in between? I can, yeah. Thank you. Okay, so this is the output that we get through the programming that's done of Google Scholar. So running the program through Google Scholar. And here we have the source of where we're getting this article from, the format, the link, the original link that we're getting through the programming, thank you. Then we have the manual coding that was actually done by my graduate assistant Danny. Sorry, the manual looked searching Google Scholar, the article name and then the journal, the year. And then we take notes on whatever happens when we do that manual search. So through that validation process, we were able to determine in the manual search of Google Scholar that there was a 69% match between what the, when we run the code versus the manual search on Google Scholar, 31% were not the direct match. But what we found is that eight of them were actually a direct download link. So the link that was in the spreadsheet is unlikely to ever match because it's the download link to whoever's using whatever computer, right? And then in the actual search of what is behind the link are we actually getting the journal articles? All the links that were looked at were for ARA Open. So we were expecting 100% to find a link to the actual journal article. Seven of them were academia.edu links and they just provided an error when we looked behind the article. One was view at CTU, which we're not 100% sure what that stands for, but it also did not provide an article. And one led to a PDF, but that PDF was actually like a plan for the research study. It was not actual, the actual research study or the actual publication. So in total, nine of them, I hope we were not able to find the people who were the final version of the article. So overall, in conclusion, we have found that from 2010 up until now, the practice of self-archiving is increasing tremendously. Looking back at the percentage from 2010 to 70% of scholar trying to disseminate and make their research available to others, we find that to be something to be proud of as a community of scholars. We're trying to make the public aware of research. And that is tremendously important, giving all the issues that we have with misinformation and other things that are happening in our political environment. Questions that I wanna leave you with is do we continue publishing in closed journals? I'm using surf archiving as a workaround. Do we really want to continue doing that? Or do we publish in open journals without no need for doing this workaround? And we don't really have an answer to it. Those are just questions that we are currently kind of reflecting on as we think about writing this for publication in an open access journal. Here's our contact information. Again, my name is Anil de Romero-Holl and my colleagues are Josh Rassenberg and Josh Valentino. So thank you so much for having me here today. And I'm open to any questions or comments that you may have for me. I'm Kathy Esso and I've followed your work with AST for a while and I appreciate it so much. I'm a library and academic librarian and these conversations are actually happening a lot of places in librarianship and there's a lot of contemporary research that's been accomplished. And you're experiencing the same thing in distance education scholarship as people are coming in and saying, oh, now online learning is starting. You're like, oh, no, thank you, AST. So I, and any of you that are fascinated about what she's saying, if you'll connect with your scholarly communications librarian and your institution, it's like yours might be Rachel Caldwell. Has she worked with you on this at all? No, Rachel, no. Okay, because they are all over it, the Spalcom people and they're gonna have resources and be able to point you to really the sweet spot on the literature and what you might be looking for is scholarly communications and librarianship, key words to drop in there to increase how much literature and they also will have resources that will help really even optimize your search, umpaywell.com and stuff as you're trying to find things that are accessible. They will also point you towards your institutional repositories. I don't know if that came up at all. Well, one of the things that we have struggle with is doing the search while we are in our systems. And that's really hard to be completely disconnected from our system. So when we get in meetings, we're like, well, you try it, you try it, you try it because we may not be in the system. Yeah, your library can sit down with you and help you square that away. Okay, thank you, I appreciate it. Yes. As a follow, there's another librarian or resources librarian here. I suspect part of the answer to your second question is, and it's all about money. Tom mentioned the yesterday, which is true, that there is nothing really free, you know? And I think, self, you're going to find that some institutions can afford to publish an open access because they can afford to pay the processing charges. Some institutions can't afford it, you know? And their authors and their authors can't afford it out of money at the grants. And so this is the difficult way out. There's an inequity there between those institutions that have large amounts of money and can afford to do this for funding. And journal articles are minuscule in costs in terms of what's happened to more graphs, yeah. And that is part of the answer. It's not the whole answer to your question, but it's part of it, which is why I'm pleased to see that your researchers are doing the self-reporting process because it's very much a route that makes it more affordable for them and for their institution. So that's great. Thank you. Yes, and the money, yeah, always comes back. Sorry, the question always comes back to Manny on Versailles. Yeah, OK, I think we have time for one more question, but I don't have two. Nice, Phil Johnson retired academic and then came up against this when I could be a couple from the institution that couldn't get things anymore. I felt as if I had been thrown out of the gang. But that's it. Well, seriously, I love the idea that people are increasingly active in making this stuff available to anybody that likes it. And it strikes me that you're very interested to get your take on other people's.