 Hello, I'm Melanie Nind. Welcome to this video for the National Centre for Research Methods. This video is about teaching research methods. It's one of a series and the focus of this video today is on the methods that we used in the study which was around the teaching of research methods, so lots of methods about methods. I worked on the pedagogy of methodological learning study with my colleague Sarah Luthwate and her ideas are very much in this video as well as mine. So because the study was called the pedagogy of methodological learning I just want to get pedagogy out of the way first of all it's a word that some people are comfortable with and some people aren't and we need to understand really that pedagogy is more than just teaching. Pedagogy is what people do in teaching and learning situations but it's also about what they understand to be meaningful, important and relevant to that teaching so it's some of the stuff that goes on behind the teaching and learning that you may or may not be able to say. And I would say about pedagogy first of all is that it's you know Brunner said it was never innocent. There's always values and meanings and perceptions and cultures and things behind the teaching that you see. I've done a book recently with my colleagues in Ireland, Cathy Hall and Alicia Curtin around research methods for pedagogy, how we research pedagogy. And we talked about their pedagogy as specified, how it's written down, pedagogy as enacted, how it's explained and carried out and pedagogy as experienced, how people actually experience what's going on and we also talked in that book about pedagogy being hard to know. It's not always easy to be able to research pedagogy because there are these layers to it. This video is the first I hope of a whole series of practical videos about teaching research methods but this is the one that sets the context. It tells you how we came about the findings around teaching research methods so it tells you how we went about the research. In the first place and explains I guess the evidence base that we have here for talking about teaching research methods. So a whole range of videos will follow this one and the thing I would say about the series is that these aren't tips for methods teachers. So if you're a teacher of methods and you've come here for tips, you're in the wrong place and I say it's not tips because I think the whole notion of tips really downgrades, downplays, the whole skill, knowledge, competence, reflective ability of methods teachers. So they're not, you know, when we teach methods we're not following a formula, we're not just picking up on some tips. We're bringing a lot to bear and our study really tried to get underneath that and to probe it but nonetheless these are practical resources. I think they offer guidance for you. The idea really is they stimulate discussion amongst teams of methods teachers about how you go about what it is you do and how you might think about it slightly differently. And the videos are accompanied by our quick start guides which I hope will help you. So why do a study about research methods teaching, you know, why the teaching? I think the teaching of research methods has been really underexplored. There's an assumption that we need to enhance the capability of people to carry out research therefore we have to teach them how to do that. But methods teachers often kind of are working a bit in a vacuum that they want to be able to talk about how they do their work. One of our research participants in the study, Charlene Hesse Beiber, who we involved as an expert teacher, if you like she said, you know, I want a space where I can talk about my own pedagogical challenges. I want to reflect on them with a group of people that share that challenge with me. I want to share resources, ideas and have this kind of pedagogical culture and that completely chimes in with what we were doing in this study, trying to generate some kind of dialogue, some kind of culture of discussing and thinking about how it is that we teach research methods in the social sciences. Just to clarify what would happen if we didn't have any pedagogical research in this area. Early has written a paper on this and argued that without any kind of pedagogical research what people rely on is trial and error without any evidence base, without any theory base, they're just trying things out and doing what feels like works and not doing what doesn't feel like it works. The other thing that can happen without any kind of research of this kind is that people end up teaching as they were taught. Again, one of our participants in the study talked about this, a teacher of methods said, I'm constricting myself because that's what quantitative teaching has looked like for me, both as a student and as a teacher. So I haven't got a wider kind of frame of reference without research of this kind. The other thing that we know happens and we know this from doing the research is that without some kind of evidence to look at and mull over, people's kind of decision making and thinking tends to hone in on things that are mundane, instrumental decisions. They miss out on the deeper kind of reflections and people miss out on what it is that we can know together rather than individually about how we go about teaching research methods. So I said that pedagogy is never innocent and I would argue that research methods and research projects are never innocent either. They come about with their own values and a whole set of ideas underpinning them. In our pedagogy of methodological learning study, we definitely had values. We reflected on our values and one of the values that was really important was that we appreciate the knowledge that methods teachers have. We want to share in that knowledge but we also would say that no methods teacher hasn't got more that they can know, more that they can learn and so it's about valuing that knowledge but also sharing and bringing people together around that knowledge. In that spirit I think one of the things we did in the study is try to position ourselves as alongside us. I'm a methods teacher, I'm a researcher, I'm a methods learner, I'm all of those things and so in a way I wanted to position myself and Sarah too in the study we're alongside us working alongside methods teachers and learners in their crime. We certainly didn't want the study to be about casting judgment on other people and saying that your teaching is good or bad in some way. It's more about understanding together how it is that we go about methods teaching and this is where this concept of pedagogical content knowledge comes in and this was an important theoretical underpinning in the study. If we've used research methods, if we're a methodologist or a researcher in some way, then we have some content knowledge about our particular method. We know about action research or focus groups or multi-level modelling. If we've been a teacher or had some teacher training we've also got some pedagogical knowledge. We know about things that work in teaching and learning situations but pedagogical content knowledge is where they come together and we begin to really understand how you teach this particular content, the examples that work, the pace you need, the variety that might be involved and so our project was about getting a pedagogical content knowledge and that the last kind of value I think that I would want to stress is that we wanted our methods in the project to be methods that teach in a way. We wanted everybody involved in the study to be able to learn from and teach each other around the whole pedagogy that we were discussing. So let's move now to the methods that we used in the study and there were many methods because of the complexity of pedagogy. These methods were expert panels, so interviews and discussion with experts, focus groups, video stimulated recall reflection and dialogue. We had a methods learning diary circle and we used case studies so I'm just going to take each of those in turn for a moment and explain them just a little bit more for you. So the expert panel, this was very much the starting point for the whole study. It was about teasing out the pedagogical content knowledge of pedagogic leaders if you like. We've been teaching social science research methods for a long time, had built up some expertise, didn't necessarily understand their own expertise or not necessarily able to articulate it without the aid of probing questions or supportive interview context. Following those interviews with these pedagogic leaders, a forum for dialogue so that when we'd analysed some of the themes that were coming out from them, putting them back to those pedagogic leaders, our panel for them to discuss in an online forum together. So that expert panel, that happened in two stages, two phases. First of all we went to national experts and then we went internationally, so starting with people like John McInnes, Amanda Coffey, ending up with Yvona Lincoln, Andrew Gelman, 21 in all. And this expert panel really was just to give us some working concepts and some things to probe and take forward through the project. Very, very rich data and we've written about that. There's a paper about that in the British Journal of Educational Studies. The next method that we used to work with that data and in a way to deepen and broaden the conversation were a whole series of focus groups. There were seven in all over the two phases. We had naturally occurring focus groups of teachers of qualitative methods in one institution, quant methods in another institution, narrative methods. We brought together teachers who hadn't been in dialogue before from within universities, from social research organisations. We also have focus groups online of teachers who teach research methods online in digital spaces. And the whole purpose of these focus groups was to try to explore the kind of data that was coming out from the experts to see if that had resonance with people who teach research methods, stay in, day out, to explore with them the resources that they use, the people they talk to, how they went about their pedagogical craft. And those focus groups helped to give us a kind of understanding of people's approaches and strategies when they go around, go about teaching research methods. We then had a very particular kind of focus group and this was our video stimulated recall, reflection, discussion focus groups and I've done another NCRM video around that if you're interested in that method. And this was about getting closer up to the action. This was about instead of enabling people to talk about generally when I teach research methods, I try to go about it in this way to actually get close up and say when I did that piece of teaching, I did that discussion or that bit of lab work or I chose that resource because so it's much closer to the action. In the video recall, what we would do is spend a day with methods teachers and learners videoing their pedagogical encounter and then sitting down with them for an hour immediately afterwards showing back some extracts from the video and probing what was going on there. Probing not just the actions, but the beliefs, the values underneath the actions. Why did you choose that? Had you done that before? Was that risky? We very much in this method wanted to do these focus groups with teachers and learners together. We didn't want that to be separate at all. And that was about helping the learners helping the teachers to surface their rationale and vice versa in a way for the teachers to understand and to be helped to articulate what they were doing by having learners there probing it with them. Alongside this was this longitudinal element in our study. So we had a methods learning diary circle and this went on for a couple of years through the project where 10 methods learners kept a diary, an online diary of their methods learning and that might be courses, it might be things they were reading, supervision, discussions with other people, learning on the job, learning through piloting and our methods learners were people at the beginning of their doctoral journey, people at the end of their doctoral journey, people in the first stages of their careers working as researchers in the social sciences. And what we wanted from this method was to get a different perspective on learning, what goes on outside of the classroom and the structured pedagogical encounters. And as well as people kind of diarising that, we wanted those participants to learn from each other to be part of this whole kind of community, having this pedagogical conversation as well as this pedagogical data generation experience. So rather than keep their diary entries to themselves, they were blogging online with each other, their diary entries. So they ended up actually supporting each other, commenting on each other's learning, supporting that learning and they too came together in focus group type discussions at the beginning and middle and end of that method. So that was a very under explored kind of angle on this, that the methods learning perspective from the learners. All the way through these methods we've been generating data, analysing data, discussing the data and we've been doing that in a very kind of vertical way so taking one method or one methods teacher or learner and saying how is this journey, how is this thing for you, but also cutting across horizontally different methods, squants, quals, digital, mixed methods. So we had all of this different kind of analysis going on on different dimensions and this was beginning to generate for us a typology, a way of being able to incisively kind of understand the approaches, the strategies, the tactics and the tasks involved in methods learning and having kind of teased it all apart, the final method that we use was case study in a way to bring the whole thing back together to be holistic and we had two case studies. One was about teaching scientific, quantitative, computational research methods in an intensive summer school and the other one was about teaching ethnographic technique in a series of afternoon sessions over eight weeks but as researchers we immersed ourselves in that context and really kind of played around with and came to understand out our typology in a bit more depth. So when you look at these NCRM research methods, teaching research methods videos, what you're looking at is findings from a whole series of methods that we've brought together. So to recap, different methods, a substantial data set underpins the guidance that we're giving you. This we've engaged with I think 70 plus methods teachers over 80 methods learners across quants, quals, mixed methods, digital methods over four years in all. So when you engage with our resources around teaching research methods rather than relying on your own trial and error, you'll have this evidence base, you'll have this resource to inform your dialogue and your decision making.