 So please give a warm welcome to Rob Farrow who's going to speak to us about research methods in open education and share insights from the global OER Graduate Network. Welcome Rob. Thank you very much and good afternoon everyone. Thank you for coming to this session. Good to be with you today. So my name is Rob and as Marian said I'm a researcher with the Open Education Research Hub at the Open University in the UK. And one of our projects is the Global OER Graduate Network. And so what I'm going to be talking about today is really about some of the outputs and activity that we've had in GoGN as we call it. So just a brief overview of the kind of terrain that I'm intending to cover. First of all just a bit about GoGN in case you're not familiar with it. And then I'm going to talk about the research methods handbook as an idea and kind of where it came from. Then how we put it together. Some thoughts on how we decided to try and present material to the network members. And then I'm going to say a bit about the kind of insights that we generated. I can't give you all of them necessarily because there's quite a lot and they're quite detailed. I'll indicate what's in the actual handbook and how it's intended to be used. And then I've also got some thoughts to share about how we could improve the handbook in future editions and treat it as a kind of ongoing sort of living document that is iterated each year. So first of all GoGN. So if you're not familiar with GoGN, the network was first started in around 2013 with Fred Mulder as the main lead. And since I think 2018 it's been administered by the team that I work with at the Open University. It's funded by the Hewlett Foundation. And the aim really is to provide a community of practice, a network of support for people who are pursuing doctoral research in open education. So the aims of the network are to raise the profile of this kind of research to offer support to the people who are doing it. But also to come to a better understanding and promotion of openness as a practice and as a kind of function of research. So the network has grown quite substantially in the last few years. We now have more than 100 researchers. And then there's a wider community around that of experts, people who supervise PhDs, people who do mentoring. And also people are just kind of advocates for open education and OER who want to be connected to and offer support in that way. So why have we got a research methods handbook in the first place? I think if you start off with just the idea of a research method, there's a sort of deceptive simplicity to what we do when we do research. And we try to create new knowledge or find something new out about open education. And when we talk about methods, we talk about essentially the techniques that people use when they do research. How they collect data, how they analyse it, how they share those findings with people. And it can be done in the format of testing a hypothesis, but not necessarily. But generally speaking, the idea is always that through the actions that you take, you support some subsequent claim to knowledge that you've gained through those activities. Even if you've just proved a null hypothesis or something like that and said, well, we know that this doesn't work. You found some sort of new knowledge. So why do I say it's deceptively simple? Well, if you will, it's the sort of tip of the iceberg. Because once you start to interrogate the idea of knowledge creation and the idea of taking certain actions to produce new knowledge, you end up kind of going deeper and deeper into the kind of philosophical assumptions that underlie those claims to knowledge. And it's quite possible that most researchers will not necessarily engage with this sort of level of theory, certainly not before they get to a doctoral level of research. And it can be quite difficult to sort of navigate this territory. And one thing that we found when we were getting feedback from our members was the methodology and research method is an area that consistently people were concerned about. And we're looking for additional guidance. The way that GOGN works, everyone who's a doctoral member already has a PhD research supervisor or supervision team depending on where they're based. And we kind of offer support alongside that. And sometimes what happens is the people who are supervising the PhD in open education or OERs or MOOCs or anything like that don't necessarily know much about that specific area. So sometimes people come to us because they want support specifically on working and either on the subject of education, or they were interested in pursuing a sort of open educational approach or open educational practice with regards to their research. But even if you're not doing open education as part of your research method is kind of hard, right? And you can quickly feel like you get lost trying to justify what you're doing. Even aside from that, people are very well versed in these kind of things can experience imposter syndrome or feel like they don't understand other methods outside their own. And generally speaking, we found that people were more comfortable expressing these kind of concerns to us in smaller groups or one-on-one rather than in a big group where we often kind of discuss things in a sort of open way. And I think related to the fact that not everyone can access expert supervision in open education. The fact that it's a kind of emergent field of study and often involves taking methods that have been developed in other contexts and then applying them to the field of open education. It's not always an entirely comfortable process. And in addition, those kind of open practices around how we collect data, how we analyse and share it all can be seen to be in tension with some traditional assumptions about how research works. And in addition, I think different disciplinary backgrounds because openness tends to bring people from all kinds of different fields together. There's also a kind of mismatch between some different disciplinary backgrounds and different academic cultures. So we were very much led by sort of demand from our members in terms of coming up with this approach. And the goal of the handbook, because we knew that we could never explain every research method in adequate detail in a way we weren't trying to. But we could try to contextualise some of this stuff within an open education context and to talk about how some of the feelings and experiences that people have around this kind of stuff are perfectly okay and also quite common. And so part of the way we approached that was to collect experiences from our own researchers about their DHD research and the kind of methods they used, what worked well, what didn't work so well and what kind of insights they would like to share with up and coming people who were interested in doing research in this area. So the approach is very much kind of reflective and critical. But we were also trying to keep things accessible and offer a kind of easier routes into method than some of the kind of foreboding text that you can capture out there that are very technical and very make a lot of assumptions about how well people understand the difference between different research paradigms or epistemologies and that kind of thing. So we wanted to explore some of the philosophical foundations, but not necessarily go, you know, writing chapters and chapters about ontology and metaphysics and that kind of thing. But more to sort of allude to these things and sort of explain that this is where the differences come from. So we had this idea, we had this concept. And we wanted to partly draw on our own expertise as we are hard, but also to draw on the expertise of the wider membership, not least because everyone works on different things and with different approaches and you know we don't claim to know everything about research methods. We were quite interested in drawing on the whole network as much as we could. So back in January, we first put an announcement about this project and we had a webinar in February where we had a kind of open consultation, we had some input into what kind of things people wanted to see in this handbook. We also had a survey that ran from January to March where we asked people about research methods that they use in their own work and what they found work well and what didn't and so on. In April, we were due to have a face-to-face workshop, which was to coincide with area 20. In the end, for obvious reasons, that couldn't take place. It ended up being sort of reduced in scope a little bit to a webinar rather than a sort of half-day workshop. But we still had by that point, you know, some sort of outline structures and some kind of ideas about where it was going. So we got some more feedback then and spent kind of May and June drafting and editing. We had an open Google Doc, so anyone who had contributed was free to come in and just do some critique, offer some suggestions for things we could improve and hopefully improve the quality of the manuscript. And then we published in July. So the report's out there and I'll share a link at the end. It's all CDC Vibe so you can access it. But I wanted to talk a little bit about the presentation style in more detail. So last year, we went through a kind of rebranding of our visual identity as GoGN. And as part of that, we met with Brian Mathers from Visual Finkery. And this is his mind map of the discussion that we had about the kind of things that we were trying to convey with the GoGN concept. And we're very keen to emphasise kind of empowering people, the idea of supporting people's academic journeys, supporting social justice, supporting people whose voices are often more marginalised in research and that kind of thing. And where we ended up is with this idea of a kind of golden age of travel or a kind of travel guide. And this identity has been sort of running throughout our recent work, you know, tickets to travel to webinars and our annual seminar and that kind of thing. And this was also integrated into the style of the methods handbook. I can see there's some penguin chat already emerging. So penguins are the unofficial semi-official mascot for the GoGN network. Partly because they're from the global south, partly because it was a bit of a coincidence with some penguins. It's a long story. But they're very popular with our members. We have our own penguins that we send out to people. And so the journey of the penguin became the kind of idea behind the sort of visual design of the report. So with this iceberg and the penguin starting off with methods, not necessarily realising how much there is sort of under the surface. So keep in good that idea of travel. The only part of the report or handbook I should say details the philosophical foundations of different methods. So ontology, epistemology and axiology study of values. And these were seen as these are presented as kind of like things that you have to know. You have to kind of just get your stamp and get on with it. But it's your starting point and it gives you a foundation and a place to work from. And gradually we work through sort of more and more complex paradigmatic comparisons and philosophical foundations for different approaches. This is a diagram which shows you a kind of spectrum of different, I suppose, metaphysical commitments in a way. Whether you think that when we do research we're uncovering something real and true. Whether you think from a more constructive perspective that all knowledge is relative, for instance. So there's a kind of spectrum between those things presented here. And at the bottom you can see different methods which are characteristically associated with these different positions. It's complex stuff, right? And having some penguins lessens the blow a little bit. It's also interesting seeing how people come to these kind of things with different assumptions and different kind of perspectives themselves. There is no sort of view from nowhere, if you like, with research method. But I think trying to map the terrain out and show this is the kind of spectrum of possibility is quite useful exercise. And certainly the kind of thing that most people would benefit from as researchers. We also did some re-drawing of existing resources. So for instance this is about the research design process and the contents actually taken from a university course which is no longer being presented, E891. And here it's re-drawn as a kind of map for you to find your way. But it kind of takes you from the idea of I've got a question I'm trying to answer through the kind of philosophical elements through the research paradigm and into a sort of design process. So what kind of things can you do to find data around that? In terms of the structure of the book itself. So we have some discussion around the philosophical foundations. We've also got some stuff around research paradigms in there. And then we talk about paradigmatic methods. So what kind of methods are associated with different approaches kind of classically. But also how do these kind of map on to open education research? Where does it become pertinent to think about these kind of things with the last particular question and that kind of thing. So similarly here you have, and this is again, this is a redrawing of someone else's diagram, but with some changes. But you can see here the penguins are kind of taking you through that process starting with a sense of values or a sense of what your commitments are as a person, as a researcher and so on. Through your sense of what there is, the ontology, how you can find out about epistemology, how you can actually do some research, which would be your methodology and your method, and where you get your data from. So trying to summarise how these things work as a whole, as concisely and excessively as possible. But also throughout we kind of give quite a lot of links to other resources and to other interpretations of how to present this stuff. So there's another part in the handbook, which is primarily written by the OER Hub team. So some stuff around open research, what does an open research cycle look like, what open practices and research look like. And so a lot of that bit draws on Martin Weller's book, Digital Scholar. But also we looked at the process of how do you go about designing a research project. Not just in terms of the research design, but some of the stuff that people don't necessarily think about, like planning it out, thinking about the ethical issues, thinking about risk, thinking about what technologies you might need to use, I suppose what you might need to learn to be able to use them. And kind of self-management and self-care as well, which are not kind of things that you would typically find in a guide to research methods. So if you're like, that's the sort of first half of the handbook. The second half is where we present the insights of the GoGN researchers themselves. I did think about putting some examples out for this, but there's just too much content and too much stuff to actually go through and there's a lot of detail there. So it's probably better if you're interested to go and have a look yourself. But these are the kind of things that we cover. So action research, doing case studies, doing content or thematic analysis, design-based research and interventions, discourse analysis, ethnography, evaluation, experimental research, grounded theory, doing interviews and focus groups, doing a literature review, mixed methods research, narrative research, doing observations, doing a phenomenographic work or phenomenological work, social network analysis and surveys and questionnaires. And for each of these sections, we have a description of the PhD project that someone is working on and also how they used a particular method in their research, what they found worked about it or didn't work about it and any advice that they would have for anyone following them if they wanted to use that method. Which is, again, not the kind of thing that you would normally get in this kind of research methods guide. And we take it to be a kind of open practice, this kind of sharing of things that people don't necessarily normally share in a traditional kind of scholarship. So one thing, I said at the start, one thing we were interested in is the emergence of characteristic practices in open education research. And I'm not sure that we quite got to be an answer to all our questions around it. Certainly not to the point where I would feel sort of like insisting that everyone was doing these things. But some of the things that came to associate with people working openly were these kind of more agile ways of working, often a more kind of intimate connection between what they were doing and the kind of practice that they were trying to change or influence, I should perhaps say. The idea of transparency and sort of how people work, how they get there and share their data is definitely associated with open practice, as is this idea of an enhanced social media presence and online presence. And leveraging the network itself and using personal networks as a way of collecting data is another kind of characteristic activity. I think sharing data, sharing tools, developing research instruments in collaboration, publishing results for open access, and I think also kind of explicit interest in working towards social justice or having some sort of ethical component for what they do. These are sort of, I would say, emergent. Maybe you could develop this a bit more into a kind of sort of template for how people do these kind of work. But in a way, part of the whole approach is that openers empowers people to do the right things for their own kind of context and their own needs. On social justice actually, I mean, when we talked about axiology as a kind of fundamental element of research method, that's not necessarily something that everyone would include, but it does foreground the idea that people's ethics and values are a core part of how research happens and why you choose to look at one question rather than another, why you choose one method rather than another. Again, it kind of challenges the kind of received view of research as something very analytical and dry and cognitive rather than physical and embodied. So off the back of this, some of this stuff around open practices. We also include in the book in the handbook some prompts to try and encourage people to think more, reflect more about the assumptions and their kind of goals and what they do. And that's supposed to be, I guess, partly about personal practice, but also about how to refine your method in the right way. So thinking more about what are you actually trying to do with this research? Are you trying to just identify a pattern? Are you trying to challenge a narrative or support some sort of professional practice and so on? But then also thinking about what differences open making in your research, and that goes back to some of the things that I was just going through, influencing practice, making things for other people to use, sharing this stuff as widely as possible and so on. So the handbook has been live for almost two months now. I think we've had something like 4000 downloads, which I thought was pretty good. And also I think reflects a desire by not just PhD researchers, but a wider community to be interested in having this kind of support and this kind of guidance. So on the Go Jam website, you can download the report in its entirety. It's licensed CC by. We also have a download for the images. So the images also CC by and you can feel free to play with them and put penguins around everywhere. So one idea we had around this handbook. So the original conception was that this handbook would be part of a series. So the next one we have planned is to focus on theoretical perspectives specifically. So sometimes the boundary between a method and a theoretical perspective can be quite blurred because the assumptions that particular theory makes might inform a method. So sometimes people said, well, I'm working in this way, but it's more a theoretical perspective than a method as such. So we have another volume plan, which will be focused much more on theories and how to use theories, how to understand them and so on. But for this resource, we're also thinking that we could do a different edition of this in the future as more of our researchers come through and finish their doctorates. And they can contribute something to the handbook about their own experience and so on. And we also think there's some stuff missing in, you know, there's so many things that are used by people outside our network, open textbook research, adoption studies, Coupe framework stuff like, you know, so we're thinking about trying to maybe commission some more people to write about methods that they use maybe outside the network. As I said, we want to keep developing this sense of what open research is and how we can understand it. And also, I think we could add more detail around research design and it adds a lot of complexity potentially to give kind of detailed guidance and all the kind of possibilities. But that seems to be where people also need some more support. So not just understanding methods and how they work, but also for my project, how can I make this better? How can I plan this better? If you're interested in penguins, there's a paper based on the kind of visual aspects of this handbook forthcoming in a special issue of the International General of Management Applied Research. I think it's coming out this year. And one last thing I wanted to draw to your attention is we just had another publication come out yesterday. And in this one, it's our research review, which again is a regular series every six months. We're going to get our members to review recent research in open education and give us their kind of critical reflections. So that just came out yesterday. Again, it's CC Buy. You can access it on the GoGM website. And in order to just like to do to finish, it's just to say thank you and acknowledge the contributions of all of our members who contributed, copied for the report and were involved in the editorial process. We couldn't have done it without you. So thank you very much. And if you could all put your hands together for a really amazing presentation. We've just got a few more questions before we break for lunch, but a few more minutes rather for some questions. And I'm sure Rob, there'll be many. But thank you very much for making sure the summit is overrun by penguins. So I think both on Twitter and in the chat, that was the dominant contribution. So that's been amazing. Thank you so much. Thank you. Thanks everyone. So we can use the mic and give you the mic if you raise your hand if you want to ask Rob a question in person or please post any questions you have in the chat and we will pick them up from there. We'll give you a few minutes. In case there's any particular points, I'm just going to scroll through the chat as well to see if there's any particular questions that came up earlier. I'm going to jump in with one question as everybody seems to be busy typing still. But you were just mentioning kind of the next step and what a really positive response you've had so far from the publication. Is there anything in particular you feel you want to hone in on for the next part? One thing I think would be good to have more of is an overview of well established methods from a sort of North American context. So a lot of the work that's been done around textbooks, as I said, and Coupe Framework stuff, where there's this very big library of papers now doing very similar sorts of research. I think it would be really good to have that captured in what we're doing. And I think in a way there are still many more methods that we could look at and include. I think in a way the key thing is getting the balance right where you don't really want to be writing an encyclopedia. You want to be kind of producing something that people can still find accessible. So yeah, I think it would be really good to keep adding to it. I think in a way that the real benefit is over time as it gets improved each year, I think it could really be quite unique actually. We have quite a lot of positive comments, but also a question. So Lucy has posted that in the chat, I believe on behalf of Maddie. How did you use the MindMap to inform your research? So the MindMap was really about the GoGM brand, if I can put it that way, our visual identity. So we had various ideas about what we were trying to convey with it. So it was only really indirectly that it influenced the handbook in the sense that visual identity came first. And it was a little bit ironic that we had this whole sort of travel motif partly based around our annual seminar where we bring members together and normally coincide with either Open Education Global or the OER conference. And obviously that hasn't really been happening this year. So a lot of our ideas about the travel motif, they ended up being quite abstract because people are still at home. But that idea of the journey and the support, the excitement of going on this journey, maybe it's almost a little bit intimidating as well. All those things kind of came together into the kind of visual style and the kind of orientation, I suppose, for the reader in trying to understand this stuff. So not exactly the research, but more the kind of presentation. Thanks very much. It's really interesting. As you say, it's a specifically challenging time to be a global network at a time when people can't travel. So I think we are at our full time now and it's two o'clock. But I'd like you all, before you head off for lunch, to just put your hands together one more time and give a big penguin. Thank you to Rob, who's presented so wonderfully here at the Summer Summit. And it's been a pleasure, Rob, and also Lorna. Thank you so much for leading an Open Education session that's inspired us all. We have a one hour break now with live sessions and we'll be coming back at 3 p.m. when the Alzheimer's Summit welcomes Angela Saini for a very special Q&A, where we'll particularly have been focusing on racism and academia. So I hope many of you will take the opportunity and join us for Angela's session then. Thank you very much, Rob and Lorna.