 The next presentation will be by Julie Voss, Tunde Vagatkin and Michael Lavin, and their presentation is entitled 30 Years of Research in Learning Technology. And there's three of you, so you're going to have to fight over the two mics. I think my co-presenters are going to stand the audience for the time being. So hi there, I'm Julie Voss, I'm head of digital education at City University, and a team of us have been working on 30 years of research in learning technology. And this idea came about probably about nine years ago when I did a MOOC on corpus linguistics. Yes, I did complete a MOOC. I know not many people did, but it was a fascinating MOOC and it got me inspired. And it's taken me this long because I've done a PhD between then and now to actually get to the point of doing this research. But I raised the idea with Tunde and the editorial team and they've kindly come on the journey with us. Okay, so what is research in learning technology? Well, hopefully you all know, but it is the peer reviewed open access journal from ALT. And this research project was to look at all the journal articles from 1993 up to 2022. So we've done a content analysis, we've looked at an author analysis in terms of the locations of the authors, and then a structural analysis. So looking at how the editorial board and the role of the editorials have changed. Okay, so the journal was launched in 1993. Some of you may have known it as ALT J. There's some examples on the slides of some of the previous copies, and I think we've seen some through the Museum of ALT as well. There have been 785 articles published between 93 and 2022. The majority have been what's been termed original research articles, but there's also been a flavor of other things. So editorials, these have been articles written by the editors, and these are going to be covered in Tunde's section. Then obviously original research, but also two other types. So we've got discussion. So these were typically responses to published articles. They started in 1998, but the majority were between 2002 and 2007, and they were followed up with a response from the author. So this is quite interesting. So I think there's one piece where Jilly Salmon wrote an article, and then somebody critiqued it, and then she wrote a response back. So they're quite nice to have a look at. And then there were 12 what were called reviews, and these were typically literature reviews, but at one point in the journals history, there was a dedicated editor for book reviews as well. And just looking at where the journal articles are focused, we did a sort of sector analysis, and the majority have been about higher education. And I think in the early years of the journal, it was primarily aimed at higher education, but we've seen a change over time. So about 68% overall have been from higher education. And this has been fairly consistent. So between 60 to 75% through each of the decades that we've looked at. There have been increases in schools and FE, but there's been no FE focused articles so far this decade. So there's still a few years left, don't worry about it. And we'll talk about how you can get published in the journal later on. So we're going to try a quick poll just to find out what your experience has been with the research in learning technology journal so far. So I'm just going to see if I can set that one up. So it is opening the poll. I don't know if anyone's managed to get in so far. Okay. Let me try and open it on my one, because that's showing us no internet access, but I'm assuming it is working. Yeah, we need to interlude music, don't we? Okay, let's view this one. Right, for some reason the internet's not working on that one. Okay, so have you been an editor or a member of the editorial board contributor, regular reader, occasional reader. Have you heard of it but never engaged with it? I know you can select all that apply. Oh, it should be, you can select all. Okay, so I think the majority are occasional readers, you've read fewer than 10 articles per year. And some people have never heard of RLT before today, so welcome. This will be an exciting journey for you and hopefully you'll be inspired to publish at the end of it. Okay, let me just go back to the slides. This is the section on content analysis. That was myself and my colleague Ivana from City University. And we've looked at the articles, the research articles themselves. So we took what was called a corpus linguistics analysis, so based on what I learned during this MOOC of 680 research articles, and we primarily focused on the titles and abstracts. We didn't do the full article. We did some other studies in this sort of area, just focus on titles and abstracts because that's quite a lot of content already. So what we did was we created some word lists and these were based on sort of the frequency. So how often words occur, but we also excluded common English words. So things like A, the, as well as some of the words that would crop up in a journal like paper, study and research. So these are our top 10 words. It's good to see learning and students at the top. Always good. And then technology online education and we'll look at how these have changed over the decades. We then followed up with what was called key words in context. So looking at words that appear with alongside other words. So what words appear next to learning now. It's the Association for Learning Technology. You'd like to think technology appears quite commonly next to the word learning. So it's sort of co located words again words that are co located regularly next to each other. And then we look to the dispersion so whether some of the words are sort of in clusters in a particular decade or just a particular group of articles, or whether it's spread throughout the whole of the corpus so throughout the decades. And we use this tool called and con can I have got some hidden slides if anyone is interested in looking at this tool later on. But it's an interesting tool. It's very simple and very quick to use and to generate these frequency lists to look at. So looking at the top 10 words by decade we can see that learning and students have always been at the top and again that that's nice you know you sort of feel that the focus on student experience came about around sort of 2012. When people started going students consumers are actually paying for something now we need to do more but actually there's always been a focus on learning and students. Some of the other changes in language that you can see, you've got like computer is quite heavily used in the 1990s, but less so it sort of drops off the list from the sort of 2000s onwards. Technology we see rise up, but you also see the rise of terms like digital and online coming through that weren't really present in the 90s. So what we've done is we've taken a couple of words to look at the language change over time. So these are, you can see computer again strong in the 90s but it really drops off down to sort of 2000s and 20s. Technology is the sort of blue line at the top that's just fairly sort of standard, but you can see the rise of terms like online so that's the yellow line, so coming up from very low down in the 1990s up to the 2020s. So that's the percentage, it's a very, very small percentage, because actually when you look at a large corpus the majority are your common words. So, you know, the frequencies we're looking at our I think computer appears probably about 200 times out of a corpus of, you know, like a well over sort of 10,000 words if not more. There are very, very small percentages. So what we've got is another poll, which is just to find out well the Germans called research in learning technology but do you have a preferred term to describe the field. I'll just move on to the next one. I've got a few responses coming through so learning technology seems quite popular ed tech or educational technology, digital education enhancement. Yeah. Class learning. Sottle. Yeah, quite a few different ones here. Digital education, digital learning so quite a variant on a theme to the sort of education learning digital. So what we've done is we've had a focus on the terminology that people have used. And again, looking at the frequency that these words have appeared. What we've done is we've combined some of them when we talked about sort of computer. The whole range of terms that we used in the 90s it could be computer based computer assisted computer aided mediated computer mediated tend to be used in conjunction with the term communication but all the others were used in terms alongside other terms like instruction assessment, a variety of different uses. So we saw computer was very strong in the 90s. In the 2000s we've seen a learning sort of pops up and becomes the most prominent one and there's been a rise in the use of sort of it, ICT and see an IT so information and communication technologies. And one thing we found when we were doing the analysis was, we saw a pocket of articles which suddenly referred to something called information and learning technology. And when you look into the detail it ties it back to a small group of articles who all got funding through an FE strand, which was called that. So it's interesting to see where these pockets just pop up and then disappear off again. And we've seen learning technology actually seen the decline since the 2000s, which is interesting, you know, given the strength of Alt. You know, it's a term that seems to be dropping down but we've always had this sort of mix of terminology and you see the terminology changing. We're going more towards digital technologies. So in the 2010s, technology enhanced or technology enabled learning was one of the most popular terms. And then the 2020s we've seen most of the terms that we had previously declined in use in favor of digital. The primary term that we found is digital technologies but interestingly, no one so far has written about digital education or digital learning. Many of us have those in their job titles my team is a digital education team. So it's interesting that that's not coming through in the articles at the moment. We also looked at learning in general so we looked at the top five words that were co located with learning. We've again you're looking at terms like collaborative and distance so they were quite big in the 1990s but a smaller percentage but then you see this e learning shoots up again. We also had the rise of the term online learning. But so far in the 2020s we've only had a third of the articles that mentioned it. Sorry, a third of the mentions all came from one article. So this is how some of the data can be skewed because we've only had three years of data. It can be skewed by one article so that's the challenge of looking at it when you haven't got complete decades. But we've mapped the on the chart the five overall learning terms that have been found. And you can see how they've changed over time. Another thing we did was we we mapped it according to what the journal says are some of their top themes. So things like online learning distance learning mobile learning. We can see that mobile learning peaks in the sort of 2000s and 2010s. And the majority of these mentions are between 2007 2012. And then it sort of drops off again it seems like, you know, as in the 90s computer was the main thing, then suddenly we'll all everything's mobile and now it's just digital stuff. You know, we're not so worried about the devices themselves. In the 90s we saw flexible and distance learning was quite high but flexible is sort of declined quite rapidly from the 2000s. So it's interesting to see how the terminology has changed over time and and you do still see pockets of people talking about computer based there were I think the the latest point we saw that was in 2018 so you're still seeing these references throughout the literature. I'm going to hand over to Tunde who's going to talk. Sorry, Michael is going to talk about the author analysis. Just the next slide. Yeah. Is he just here. Yep, just. Thanks. Together with James Brunton is from Dublin City University and he did the bulk of this work. We took a look at the authorship of articles since 1993 it was sobering. If you take a kind of cursory look at this pattern it looks broadly healthy that scholarship from outside the UK. As a proportion of the scholarship we publish is on a upward trajectory well that's true. They're not exactly like and like categories. One is the United Kingdom. Two is the entire rest of the planet. I think if we disaggregate this data. We find that we're a heavily or historically have been a heavily UK centric publication in terms of where the publications come from. And this is problematic because research and learning technology describes itself as an international journal. We've got the do nothing option because we can say well look the general trend seems to be improving. I think we struggle to say that's quite adequate. I think we need to do more. I think one of the that's going to be all kinds of points of value for the exceptional work that Julie's been leading on. But I think once we have the finished article we have to take the data from that. And then use that to inform our next strategic iteration of the journal so that we have a healthier set of statistics next time around. Julie you're okay to pop the next one on for me. We take a look at it here again the overall trend is encouraging the number of countries that contribute scholarship is increasing just a quick methodological note the way we work this out. What we looked at the first author of each journal article we looked at their academic affiliation and took that as the country of origin, the workplace of the first author. Again the overall picture is encouraging but if we think about two categories there's still definite room for improvement. And if we look other than the UK at the countries that are producing more than 10 articles. We still see we're not doing enough I think to develop technology enhanced learning scholarship from the developing world and in our next strategic phase. I think that's really an area that we need to focus on. If we were toward our self a mark I think we're at the B minus could do better mark but I think this data set is precisely what's going to inform proper strategic steps to put us in a healthier place with the next generation. Thank you Julie. Thanks Michael. Hello today. Yeah, thank you so with Sarah honey church and Liz Bennett we looked at the structural analysis of the journals so that was looking at the mode of publication the name and also the editorial board. I think Julie already said that a big pivotal change in the journals history was going changing name and I guess reflecting the focus in 2011. And then in 2012 going open access and I think the edits one of the editors responsible for that marvelous changes in the room as well. So, I think that I will come back to that because what that meant, obviously that everyone had access to the journal and this is true going backwards as well. So it's a very welcoming and open space for any of you who haven't published yet to consider that. So that change also gone hand in hand in hand, sorry, whatever with going from discrete issues to rolling volumes and I'll come back to that in a minute. And the other thing that was very interesting for us to look through the special issues and the conference proceedings that was a big history of this conference and people presenting and you presenting your work then translating that into journal articles. So I think that was that's really reflected in in us looking through the last 30 years. And we also looked at the editor editorial team and editorial board composition, looking at the country representation, and also just what the roles were. So for those of you who might be new to publishing and just in terms of the editorial team and the editorial board members responsible for the strategic decisions and influence about the journal and editors are also responsible for welcoming in the and allocating them for review and also for desk rejection. So that there's an important and volunteer role for this. So looking at the structure of that already in 93 we had an editor and two associate editors and that hasn't really changed that much with some, some periods where there was some specialist roles for editors like as Julie mentioned the discussion paper. There was a public editor looking after those papers in that period. Perhaps what you notice is that today we've got editor in chief, Michael, and seven editors supporting him with all the work and that just signals that there is a really healthy submission and and traffic in to the journal which is great. In terms of members that members of the editorial board I think that reflects what Michael was talking about representing, we've got really good representation from countries but also there is scope for, for looking at that and widening. And just finally, maybe a remark around so female representation on the editorial board is very pleasing because we have got about 70%. Yeah, thank you. Okay. Yeah, the next one I think is that. Oh yeah so we also looked at. So that's probably for the period when we had the issues rather than the rolling structure, when there was an editorial from a piece of writing to headline the issue that particular issue. So looking at looking through that they, they took up a really important space I mean words are power as we know it. So they served various purposes so for instance some were about highlighting and focusing some really important areas for research things that we should care about as researchers. For instance around inclusion or accessibility. Other editorials were close to action around establishing research centers, and then also this seems to be a recurrent theme around navigating the tension, but do we report our practice. So I mean as we were dealing with various theorizing our field. So it's how we balance those, and then also those editorials took a role in shaping the values again that we should care about to maintain the quality of research in our field. So on the round looking outward to external pressures, such as ref or, I mean, if some of you may remember the going of back to our or so to some of the external pressures that were impacting our field. And some of them were involved looking for instance, when we were explaining about the changes about the open source and why those. I think. Yeah, and then Julie you covered the discussion articles. Okay, I think I think that's probably enough to go on to. Yeah. And so just a final bit one of the editorials. You know, was reflecting on technology, and says you know it may be that having given students the tools in the form of learning technology with which to access and manipulate digital edge information we need to rethink our examining criteria where it comes to project work, at the undergraduate level now. This sounds quite current doesn't it but what year do you think this was from. It was, yep, 93 it was issue one volume two and it was one of the editorials, but it's funny how the conversations come background and it's been fascinating looking through some of the older journal articles. So just hand over to Michael who just mentioned if you are interested in publishing in the journal. I had a look at the very first edition of the journal as part of the work I've been doing. And it did notes the something to keep an eye on was this emergent technology called word which seems to be quite popular. It's worth revisiting I think somebody that was onto something there's a perspicacity for you ladies and gentlemen, quick word about the journal okay a positive piece of news. The current editorial team has been in post citations of reason what this means for those of you who may not be familiar with the journal academic publishing in general, is that other scholars use research and learning technology more than they used to we're on an upward trend. Now we can simply do nothing and hope that continues. I think we want to do more to try and push that forward at a conscious level and I think one way to do that is to think well what do we do that's distinctive if you do any research in this field you'll you'll know you don't have to look far to find a learning technology journal. So if we were to try and advance our journal what do we do that's distinctive. I think one of the things that really kept me coming back to all conferences over the years is talking with people who do the job. If I want to talk with fellow academics I can just stay in my department at King's, but I don't. I'd rather come here and listen to people who do the job who work in learning design who work in the back office of technology, who are real. If you like cold face workers in this space because I think the, the knowledge they grew, and the insights they produce are really lucid. I think to some of you who may not be people who publish in academic circles what can we do to support you. So in I think in the next iteration we're going to do more with the old blog. If you haven't done much academic publishing before speak to us about writing a blog piece will support you that blog pieces impactful and having it speak to us about what we can do to develop and just state that work into an academic article. On the other angle, if you write an academic article and we accept it, we will invite you to write a 500 word summary piece for the blog. I think we're going to use the editorial board of whom we haven't spoken much day and at the stratum above us editorial team to think well what can we do at the strategic level Julie's article is going to give us the data on which we can devise the strategy, and then gets percolated down to the editorial team, which then we use the journal and the blog, and let's rather than cross our fingers and hope for enhancement let's build it into what we do but it will be predicated on the outstanding work that Julie has done. Thank you very much. Thanks to both of my co presenters and the rest of the team we've been a fascinating walk down memory lane for some of us. See if cables. That's why we're all here we're learning technologies cables everywhere. We do have around three minutes just for questions and I. I did see your hand go up first. So, let's fight to the death. Thank you very much. Thank you for that. I am interested to hear how do you position yourself against other journals that do pretty much the same like, for example, be jet, or I mean what in terms of impact factors, I mean, tell us a little bit about that. And the editor in chief. Thank you. I'm not on the editorial team so over to you Michael. I mean, one we're not suing a rival strategy. Because I think we know we're all operating the same field we support each other in my own published work and was likely to use BJ as I am to use rlt I think I have no brand loyalty as a scholar, because I think you're just looking for where the most relevant thing is. Right there what do we do. I do think if we can just find our distinctiveness which is there in the practitioner level and look how we develop practitioner into writer I think for me that's where I think we should be heading, but I'm one voice amongst an editorial team. So yes, I mean you really don't have to look far to find a journal that works in learning technology. I mean, it's interesting how you see these kind of waves of what's fashionable coming in. We're starting to get a lot of AI related submissions because of how high AI is at the moment. I'm currently peer reviewing a paper for computers and education and guess what it's about AI. So I think those waves of what's in and what's out will come and go, but I think where we have a real you. The whole unique selling point is that yeah sorry I don't feel comfortable with myself and I use that language I feel slightly sully now I'm really sorry. But I think our USP is that practitioner core and I think that's what I'd like to see us develop over the next iteration of the journal it's next strategy phase thank you and strategy phase is awful as well. The light is rising I think we're now in Q1 so we're quarter one. I think by the time I hand I step down as editor in chief and my own view is no one should do more than five years in the role because you start to ossify a little and it just becomes part of what you do instead of driving it forward. The real target I'd love to see us in the top 15% by then, but you know there's the cross your fingers and hope strategy and there's the actual take step strategy and we have to pursue the latter. Thank you. We've got time for one more question were three hands previously, but some people have slipped down. She was first and running. They never said it'd be this stressful to chair. I'm not going to be able to answer this question very quickly but talking about the corpus linguistics methodology. Obviously you've done the first stage where you've looked at the words and the data the concordance lines etc. But what about the second stage what conclusions have you drawn from that data. What do you think that data tells you about the development of the field since 1990s. That's the next bit the research that we're going to be doing. We've been looking up the paper and mentioning BJ they actually did a study of their 50 years, and that that was one of the sort of inspirations for this as well so we've been looking at some of our themes and how they compare there. But yeah the next bit is looking at the trends. I do work on the size of technology enhanced learning survey so trying to tie in with some of the data that we've got there as well. So, yeah, maybe we'll come back next year and tell you. That does bring us.