 Welcome to St. Mary's University College this evening and to a lead lecture by somebody who needs very little introduction actually, Professor John West Burnham. I have a list of paper in my pocket that outlines the publications that John has to his name, particularly in the areas of leadership within education, but also around spiritual and moral discourse within education. And that's something that I'm quite passionate about. And what I think John does so beautifully, both in his writing and in the way that he inhabits leading in education, is that he does both with two things that perhaps are not particularly valued perhaps or lauded in leadership or in education. One which is grace and one which is humility. And I do believe that to lead with integrity one must have both grace and humility. And I do feel that John exudes that in the manner in which he writes, but he also inhabits that within his own disposition. And so this evening, after a very long day at school and perhaps meetings after school, it's fantastic to see so many of you here. And I think you do have a little gift this evening. So it's a very precious gift and it is the gift of John West Burnham. So do enjoy as you open it up. Thank you. That's a very kind and generous introduction to a topic which may not actually stir the soul for all of you, necessarily. In fact, I'm amazed there's nothing going on in Greater London tonight, which is more attractive than this. I mean, this really is the ultimate resort, isn't it? Equally, I am of course always impressed by the fact that you turn out for this kind of event. And to spend an hour or so talking about research methodology and a very significant number you, of course, have alcohol in your hand. And this may well be the new approach to research methodology is just how many glasses of wine does it take to complete a dissertation. You let me know in due course and I'm sure we can find a theory around it. What I want to do this evening is to really focus on not so much the content of your research but the process of your research. And I've been involved in supervising research for about 25 years and it is one of those areas where, shall we say, the excitement of some courses has been somewhat tempered by the realisation it's time for methods. And I've yet to find anybody who is, well, apart from a few very distinguished folks, who are very, you know, this is what I came to do. But it's one of those very necessary components, isn't it? And it really is about the integrity of your work, your scholarship and crucially the impact that you make with your studies that I'm primarily concerned this evening. And I'm going to work through a range of ideas and just offer you certain insights. And I will pause from time to time because at this time of day to sit passively and listen to somebody, especially with at least two glasses of wine behind you for some of you, is too much. And so there will be an opportunity to talk to your neighbour. Please decide if your neighbour is somebody that you wish to talk to. I don't think it will move if you don't like your neighbour, but the idea being there will be conversation. So let's begin by focusing on something really which is often missed out in my experience from the preamble to any dissertation, any thesis, which is what exactly am I, why am I doing this, apart from the obvious of completing my degree. And I think there's a very strong case, if you like, for going personal. And I do think, and it depends of course upon your supervisor's own predictions, but there is, I think, a real value in explaining and exploring your own motivation and the basis on which you're engaging in the research. And there's no one way forward on this, that all of those possibilities are valid. But of course we are looking for personal learning and growth. We are looking for the growth of personal and professional effectiveness. And I think that's an entirely valid and appropriate way forward. In order to be able to say, I am developing my professional understanding and in doing so I'm enhancing my confidence. And I think that one of the key factors, whether it's in classroom practice, whether it's in middle or school leadership, whether it's in any kind of professional practice, is that effective research should surely be a contributory factor to building confidence. And that really is, I think, when it begins to justify itself over and over again, and really good research seems to have that impact. Inevitably, and for some of you most importantly, your research will be part of a strategy which enhances the effectiveness of your school as a whole. And there's no contradiction between those first two, they're absolutely symbiotic. But nevertheless, what's the priority? Is this something that is part of an overarching school strategy? And certainly when you are working collaboratively within a school, and St Mary's has a very distinguished tradition of supporting that, then we know that's probably amongst the most effective research that can be done. It really does make a difference in schools, and it really does provide leverage for all sorts of powerful change. In many ways, shared research by colleagues in a school often provides the greatest methods in terms of bringing about significant change. How about this notion of the learning community? The learning community is very much an aspiration in all sorts of areas of professional life. And I think that one of the issues about creating a learning community is developing a shared language, isn't it? You can't be in a community if you can't communicate. And therefore your research may well be a significant component of actually helping to develop a common language, a shared vocabulary that really does enable detailed and very rich conversations to take place. And that I think is one of the great outcomes of building, developing confidence in vocabulary makes all the difference. In the current climate for most schools in this country, the fourth issue is perhaps a pivotal one. And there is no doubt at all that the focus on teaching and learning to close the gap has to be an absolute priority because of the accountability model that we're now working under, because of the moral imperative and simply because of the way in which we are now looking at the way schools work. And if you've had an opportunity to look at the new off-stead framework, you'll see that everything now is focused on this notion of closing the gap and that the emphasis across the off-stead framework is almost exclusively on teaching and learning. And that may be argued that if your research does not make a contribution, if you are working in the context of a school towards closing the gap, then it must be in some ways be marginal. Now, that doesn't diminish the enormous range of possibilities. But nevertheless, I believe that at the moment, given the pressures, we need to have a very clear focus on what is the central imperative. And there's finally a very strong case for arguing that if research in education does not contribute towards social justice, then it really is not fulfilling its true potential. And if you think about the situation in this country, it's certainly a great deal of contemporary research done in North America, focuses in on this challenge of equity. And as you know within our system, equity really is one of the fundamental issues that the whole issue of closing the gap is really best expressed as an issue of equity, isn't it? And therefore I think it's just one of those tests at the start of a research programme to begin to say of those five, or more importantly your version of those five, what exactly is it that I am seeking to achieve? Because that must have a significant impact upon your engagement, your motivation, upon your audience eventually perhaps, but also upon the way in which your research is designed and structured and worked through. And therefore clarity of purpose, as in so many human endeavours, strikes me as being the starting point. And then when you've achieved that, when you're confident about what your purpose is, then what is it actually you're trying to do with your research? And again I think sometimes people sort of drift into their dissertations without ever articulating in detail, the purpose of this research is to. And one of the most powerful areas of course, is the whole notion of evaluation within the school. And many years ago one of the leading writers on school effectiveness argued that evaluation is the Achilles heel of school improvement in this country. We're simply not that good at it. And the whole culture of monitoring, review and evaluation is an area where there can't be too much of it in terms of how did this work, what made the difference and so on. And evaluation as research of course is a fascinating study in its own right. And it has all sorts of issues and implications. But is it about evaluation? Is it actually saying, did this make a difference? For example in the current language, can we demonstrate impact? And if you look through all of the things that are going on in terms of educational reform at the moment, impact comes through over and over again as a key factor in determining whether this is an appropriate activity or not. Shared professional practice. And again that has to be a priority, doesn't it? Because one of the challenges and again it's embedded in the research, it's in the literature, one of our challenges as a country is the enormous variation that we have in practice. And therefore focusing in on the notion of how do we build shared practice of consistent quality has to be, I would think, a major outcome of any piece of research which is looking at the classroom practice, which is looking at the leadership of classroom practice, which is looking at the leadership of school improvement. All of those come down to how do we get consistency within the system. You may have come across yesterday, it was, wasn't it, that the Chief Inspector's, the Ofsted Chief Inspector's report was published. And in there she indicated that one of our major problems across the system now is the fact that about 40% of lessons in this country are satisfactory, which is really a challenging issue, isn't it, in the current climate. Are you testing alternative strategies? Are you beginning to say, does this work? Are you beginning, and here I think it's a huge opportunity, beginning to explore research-based classroom practice. Cos that does seem to me to be absolutely one of the pivotal factors in any model of research. One of the interesting things about very high-performing schools and high-performing education systems, particularly thinking here of Finland, is that by and large classroom practice and whole-school strategies are evidence-based. And that's where I think your research can be such a powerful resource in terms of in this context, at this time, these strategies do seem to work and these don't. And one of the great issues facing I think leadership in schools is the notion of how do we move into an evidence-based approach to education. Cos often we are an experience-based approach to education. And that basically I have been teaching longer than any of you in this classroom. Therefore, my experience counts for more of yours and I will not accept any challenge on that basis. Is that quite clear for everybody? I can go back to 1970, therefore do not broke any kind of interference. Whereas in fact, of course, my practice may be long-term self-indulgence. And I do stand by my professional right to be consistently incompetent. Unless we have the evidence, unless we begin to develop the data, unless we begin to be rigorous in testing. How do we actually bring about change in classroom practice? Well, the answer must be to some extent through research. The answer must be by focusing in on projects. And the history of education is littered with projects, isn't it? And often we do something, it seems to be good for everybody involved, and then we just drift out of it. It just disappears. And I was working today with a group of primary heads, and we were talking about philosophy for children. And one of them was saying how three or four years ago it was really well embedded in his school. And then there was a new head came in, there was a change of practice, and suddenly it was almost forgotten. Simply people moved on. And that's really sad because it made a difference. So the evidence around P4C is hugely positive. But nevertheless, the momentum for change was lost. And that's a tragedy. Your research can be an enormously powerful resource within the school. And then finally, just trying to make sense of the schools. Schools are incredibly complex places, aren't they? They really are very, very strange social organisations. And the more that we can understand how they work, the quality of social interactions and so on. And I came across recently a lovely example of this, which I can't prove, and I don't think I have time to test. The Professor of Anthropology at Oxford is a chap called Robin Dunbar. And he published a book, I think last year, maybe earlier this year, called How Many Friends Does One Person Need? Now if you get it for Christmas, it may be a very subtle message to you. And basically what Dunbar has done is to analyse the basis of human interaction from a historical, archaeological, anthropological, and also zoological perspective. And so he's come up with all sorts of interesting models of what it means to be a human in social interaction terms. And so he says basically, most people who are living effective lives in a circle of four or five people, usually close family or very close friends, who are absolutely fundamental to their lives. And then there's an outer circle of about 12 to 15, and the number 12 there is very significant, isn't it, who are absolutely central in the way that person lives. That, for example, he actually says, the death of one of those people would have a profound negative effect on the person at the centre. Then you're part of an outer circle of 30, 40 people. And then you come to what's been called Dunbar's number, which is 150. And he says that 150 keeps cropping up as one of the basic ways of understanding human society. So, for example, most of the villages in the Doomsday survey were 150. Most of the villages in the 18th century, when villages were being closed, were around 150. Apart from Kent, where they averaged 110. And we don't quite know yet what's different about Kent. There's no science that we can find. Apart from the people of Kent, of course. It's the average size of a company in the British Army. The basic unit of the Roman army 2,000 years before the company of the British Army was not the century, it was the manifold of 80. And two manifolds made up a century, 160 people. And it's just fascinating. And in terms of understanding a school, really interesting if you think about all those numbers and the way schools work. And it really, you know, there would be an opportunity to really say, can we really research the basis on which the social functions of a school might be more effective, rather than simply perpetuating the historical models? For example, is every child part of a small intensive group of three friends? And it might be then, you see, that we can move towards a strategy which is based on a very, very powerfully researched hypothesis, which can then be tested empirically within our own experience. That strikes me as being good research. That's when we see research really making a difference. So these first two areas I would suggest to you need to be very robustly argued for. I'm doing this research because and the reason underpins my thinking is, and I'm hoping to demonstrate that, and you make it personal. And you really do embed it in your own thinking. So I'll pause there for a moment or two just to let you have a reflection on the extent to which those criteria, those two sets of criteria, may well help you inform how you approach your research. As you don't have a good view of the screen, I will be very carefully talking through the points and you're very welcome to stop at any point and raise issues that are not clear. So let's say you've got the basics in place. You're clear about purpose, carefully focused on outcomes. You have explained to the reader the nature of this process that you're about to start. The reader is confident that you are in control of that and in essence this is clearly articulated as something significant and of value. So what have you got to do in order to make it work? And so the notion here of what are the criteria for successful research. And at the start of what has to be an ethical integrity, doesn't it? This, your research has to model values which we would expect within an academic community within an educational establishment and so on. And therefore the integrity of your research is more than just the quick, the formal bit in your methodology chapter saying, yes I have consulted the ethical framework. It is actually about working right the way through in a consistent way which is very clearly focused upon the respect for and the engagement of those who are involved in your research in whatever capacity. And as you know, the pivotal concept here is one of informed consent. And in education that raises a range of issues about the way in which we research. And it would be so much easier if we could just treat our colleagues as guinea pigs. So much simpler, but in fact that's not an option available to us. One university I worked with, they had a checklist of ethical guidelines which started with, used across the whole university, which started with the question, does this research involve experimentation on live animals? And somebody wrote in no just teachers. And you all know the issues, but I suppose that in marking dissertation upon dissertation over the years, the one area that often is compromised is this notion of explicit demonstration that you have followed clear ethical principles in terms of the design and the writing of your research. And that strikes me as being almost a pass-fail issue. If this is not clearly within an ethical framework, then it cannot be acceptable surely as a piece of research. The second issue is the issue of trustworthiness. And one of the challenges I think is to say, is this reliable? Is this simply a product of somebody's imagination? Is this the product of work which shall we say may not actually have integrity and veracity? And crucially, has the researcher demonstrated that she or he has triangulated their work? Because that must be part of this process of building confidence, mustn't it? And essentially think of the classic milking stool, the three-legged milking stool. And you need all three legs in place, don't you? You need your conceptual framework, which I'll come back to. You need your data. And then you need to be able to corroborate the relationship between your data and other appropriate data which reinforces, confirms, or indeed challenges your own data. And that source of that other data may be alternative methodologies. In other words, you've done the standard questionnaire, you've done the interview, that's great, good stuff. That's very monotechnic, isn't it? It's very narrow. How about alternative perspectives? How about allowing your research subjects to express different perspectives? It may be that you use different research. So, for example, you find parallel research to your own which allows you to corroborate and confirm or not your own findings. It may be that you are going to ask another person to look in an informed way at your research. But the crucial thing is you have to explicitly demonstrate that this is more than simply your construct. It has to be a construct which is very firmly anchored. Almost, if you don't like the notion of the milking stool, think of a tent. How many pegs have you put in which demonstrate that this is secure? And the more secure your research, the more you find alternative corroboration, the more you're able to demonstrate that this really is valid, that we can use it with confidence and we can replicate it with confidence, then I think you're doing your job. Otherwise, your research is incredibly fragile, isn't it? Is that coherent? Does that make any sense? And I think it's one of those areas where you can simply say, I've got the data on we go. You need to go back and say, can I trust that data? And can I demonstrate that that data is trustworthy? Does your research make a difference? And I've got no problem with people who write in their research conclusions, this piece of research didn't work. I didn't find out what I was expecting, but I've learnt a lot in the process, thank you. Hopefully you avoid that situation. But nevertheless, there must be, if we're working in education, we cannot surely have research which is totally abstracted, particularly at this level. It must make a difference. Has this actually impacted upon practice? Has it made a difference to the quality of teamwork? Has it allowed more effective conversations to take place about professional learning and development? All those issues need to be embedded, I believe, in a master's degree within an educational context. Is it professionally relevant? You decide that. But actually, there's such a massive range of activities going on. But I think, given the pressures of time, the scarcity of resources and so on, that your work does need to be addressing authentic and relevant issues. And then, as you all know, that the criteria for a successful piece of research does involve scholarship and the academic value. And that's expressed in all sorts of ways, isn't it? It comes down finally to careful proofreading, doesn't it? Something as basic as that. It comes down to the appropriate use of the technical language. It comes down to presentation. But it also comes down to the ability to articulate and express often quite complex concepts in a way that is meaningful and demonstrates the relationship with other aspects of your work. And that really is the scholarship, isn't it? That really is when you're in control of your subject matter, you're in control of your data, and you are able with fluency and confidence that's real scholarship. And the academic value of showing how your work integrates with challenges, supports other people's work, that, again, is very much part of the academic process. And again, and this may sound weary, and he's been around too long, but eventually, when he spent a few years marking master's dissertations and doctorates as well, then you do want to say, look, the best model for this type of research is the novel, not the short story. Because sometimes a dissertation reads like a series of short stories called chapter one, chapter two, chapter three, chapter four, chapter five, and if you're lucky, chapter six. And never the six shall meet in the sense that you can tell. Chapter one was written towards the end of the summer holidays. Chapter two was written during the autumn half term. Chapter three, which is the least satisfactory, was clearly started on Boxing Day. And they are quite separate events. And that's the worry. And that's why we need to see the research process as essentially iterative, as cumulative, as growing and developmental. And therefore, there needs to be very, very clear linkages between where you start and as you progress through the process. And so, for example, in the opening of your research, you explain the context of your research, the issues around your own particular school, if that's where you are researching. And then you begin to say, what are the issues emerging which you're addressing in your research? And that, I think, really does require revisiting over and over again. And again, it will depend upon your own supervisor's preferences. But I think there is a very strong case for the use of research questions. That basically you have one big research question, which becomes the title of your dissertation. And then you have a range of other research questions which are, what do I need to find out in order to do my research? And I'm amazed by how many dissertations do not actually pose the questions and then crucially revisit those questions. And so, for example, I would hope there would be a tentative set of questions early on in the introduction. But after the work that's been done on the literature review or the conceptual development work and so on, I would think that it should go back to the research questions and say, on the basis of this thinking, I've modified my questions. Otherwise it just becomes a series of compartmentalised activities, doesn't it? What we're looking for is flow. What we're looking for is interaction. And sometimes, in the worst cases, the review of the literature is basically books I have nearly read. Books in which I have used the index with great skill. Books in which I have found the facility on Amazon to skim some pages very helpful indeed. Or is that being too cynical, ladies and gentlemen? But the notion is that that second iteration is not about saying these are books I have read. It's saying this is the conceptual map I am using. And I think sometimes in many cases and in many publications there's a reference to the literature review rather than the conceptual model. And I think that's a really challenging issue. The notion is that what current knowledge exists in your subject area and then how does it interact with all the other factors? Because every topic you can think of there's a huge range of literature, isn't there? But what do you make of it? You, in that particular part of your research are sense-making, aren't you? You synthesise, you analyse, you demonstrate relationships. And that, I think, is when you really begin to demonstrate scholarship. That's when you begin to lay really firm foundations for your research. But again, there's so many dissertations where that section, chapter two, is in isolation and it's never referred to again. Whereas I would hope to see it, for example, once you have developed your conceptual framework, the range of criteria that you're going to use to inform your actual data collection, I would want to see that literature referred to again. You know, because it's the basis on which you've designed, isn't it? And too often we have chapter two, literature chapter three methodology. And I think that the notion of the iteration so that in many ways the last part of your research actually returns to your major question. You know, a bit like a symphony, a bit like an opera, whatever, that the opening bars of the overture are repeated in the very end of the piece of music as an echo that takes this full circle. And I think that is intellectually satisfying. That's intellectually coherent rather than this process bit, bit, bit, bit. So, what are the key questions? What are the issues? But I think you need to begin to use in terms of the design of your work. First and foremost, and I'm going to read these out for those of you who don't have a clear view of the screen, what are the generic and specific knowledge areas that you intend to work in? In other words, you've got a huge map called education. How are you going to narrow it down? So, are you looking, for example, at professional practice? Are you looking at school improvement? Are you looking at leadership? You need to be very robust and clear in terms of this is the contextual area. And then move into the notion of what's within that broad area that is your chosen context. What are the central issues, problems, topics, hypotheses that will be the focus of your research? So, for example, in the current climate in schools, part of the process of coming to terms with a massive amount of change might well be to say, just how effective are the teams at this school? And therefore, you're taking the notion of the very broadest issue of school effectiveness. You're then moving it down into a particular aspect of leadership, perhaps. You're then moving it down again into one particular component of leadership. But, of course, you've then got other issues which come in. For example, if you're looking at teams, you're also looking at interpersonal relationships. If you're looking at teams, you're looking at school organisation. And therefore, you begin to map out the interaction between the various elements. What are the key conceptual issues influencing your research? And what I mean there basically is the sorts of questions. For example, to use the teams example, one of the great myths is that teams actually work. Yeah? That there's a huge amount of rhetoric around effective teams. And yet, if you look at the reality, the vast majority are what Ken and Bob and Schmidt called, the terms go on, but they're basically sooner teams. That's what they call them. We are a team. We know we're a team because we meet regularly and we've called ourselves a team. And if anybody questions us, we will reinforce that we are a team. Now, the fact of the matter is, we don't work like a team at all. And therefore, the challenge to the mythologies and saying that here are the criteria for an effective team, is the issue of the conceptual framework because it's not enough simply to say we have called ourselves a team. It's to say, for example, we have extremely effective interpersonal relationships, which brings in another set of criteria. What's the conceptual map? What sorts of themes and issues are going to inform your research? In other words, if you're looking at a particular topic to do with classroom practice, and for an interesting example at the moment, it's the work done by Durham University, published by The Sutton Trust, on how to spend your pupil premium. And The Sutton Trust reports of really interesting and powerful resources, I think. I must say this piece of research on how to spend your pupil premiums has got, I think, some quite worrying issues around it. But it's really interesting. And one of the things they argue is that the centrality, for example, in terms of impact and effectiveness of ear support in the classroom. So that then gives you a handle to begin to look at all the different aspects of the notion of collaborative learning. And therefore you can map out from that one core theme and make the conceptual map of all the interacting elements that look at the effect on achievement of that particular approach. What sort of knowledge are you looking to create? Are you looking for hard, hard reliable data that's going to inform practice? Or are you looking for something that's more developmental? Are you looking at information which is prudently scientific? Are you looking to be something that's more philosophic, et cetera, et cetera? What sort of language are you using and, crucially, what are the methodological implications? In other words, it's back to the very, very simplistic model from business which is that form follows function. In other words, what are we trying to find out? Therefore we need to develop a methodological approach that actually allows us to find that out. And then you're into the methods, but only then into the methods. And again, being very crude, the issue around chapter three, the methods chapter, sometimes does not include justification as to why I'm using this approach. And that must be wrong, isn't it? And therefore there's a very real issue in saying why am I doing it and the process of continual justification. And that, I think, is when you really move into higher order writing, is this is why, this is why, this is why. And it's not descriptive narrative, it's analytic and explanatory. And I suppose over the years the phrase I've written more than anything else on drafts of dissertations is please move from description to analysis. Tell me why this is the case. Not simply assert. And that, I think, takes us into higher order. That, if you like, for me is master's level thinking. So, what does a successful dissertation do? In the words of a former colleague of mine, there are three things that make a dissertation work really well. The first is focus, the second is focus and the third is focus. He was not particularly articulate, I don't think, but he made his point very well. In other words, once you have decided what your research is about, once you've mapped out the conceptual terrain, then you focus in on it. And depth, depth, depth. And again, the analysis, the explanation and the depth is what makes for distinguished master's level writing. Your work is internally consistent. One of the joys of marking particularly doctorates, and it is a joy, is that for some reason, I suppose you just get used to it in time, is that you read through a dissertation a couple of times and then you spot the inconsistency. And the minute you spot the inconsistency, you think you've got him. And you sit there and you ask the gentle introductory questions, then you say, Mr X, on page 27 you assert ABC. And yet on page 280, you say it's actually CBA. Would you get a comment? And the poor man sits there like a rabbit in headlights. He's got me. Because he has not checked the internal consistency of the argument. And that must be scholarly, mustn't it? That must be what it's about. You make sure that you are using the same definitions throughout. You make sure that you're using the same descriptors. You make sure that your internal triangulation is constantly reinforcing and supporting assertions. You are looking to make sure that this is safe and internal consistency is the great way of doing that. As I've said, the process is iterative. It grows, it builds, it builds, it builds. Each stage of your research adds value to the previous one. They are not disjointed, they are integrated. And then, as I say, it's analytical. It really does need to explain why above everything else. We can get narrative anywhere, but explanation really does make the difference. Shall I pause there for a moment? She's getting quite warm in here. Is it just me? Take a few moments just to move around and rattle your cages. Let me move on so that we finish it on time. Let me try and demonstrate some aspects of those approaches in practice. Rawly speaking, I've read the Sutton Trust report on how to use your pupil premium, and I'm really taken with this notion that their findings, that feedback in the classroom is just about the most effective way of improving pupil achievement. I like that idea because it says it's high impact and low cost. That's appealing. Therefore I decide that I want to look at this notion of focusing on a particular pedagogic strategy but developing it consistently across the whole school. The key question, and again this may become the title of my work, is what team-based strategies help to support changes in classroom practice. I'm identifying at the outset the focus of my research, identifying the key elements in that research, and I'm beginning to say what's the problem, which is how do we change classroom practice? And then the specific questions, what changes in classroom practice are most likely to make an impact on pupil achievement. That becomes the key driving force. This is what I've really got to find out. An interesting one which we probably is too big for one dissertation, but how does professional knowledge relate to actual practice? Because there's a very, very complex link, isn't there, between knowledge and practice? I may know this but I may not do it. And there's an interesting one there, the notion of implicit and explicit knowledge professionally. If we had time then I'd ask you, the vast majority in this room I would guess can ride a bicycle. Yeah? You all know how to ride a bike. Just explain, come up here and explain how to ride a bike please. That would be exciting, wouldn't it? Because our implicit knowledge isn't often difficult to articulate as explicit knowledge. That's one of our key professional problems, isn't it? Our teams are significant resource to support professional learning and change practice. Because we've got so many teams in school, departmental teams, phased teams, et cetera, et cetera, pastoral teams, do they actually help in this central process? Because if they don't, why are we bothering with them? And then, if that's the area that we're researching, practice, perceptions of role, et cetera, et cetera, what are the methodological implications of that? And then, having identified the methodological implications, what are the methods that are appropriate? Does that work? Yeah, making sense? Thank you for that overwhelming positive response. So therefore, the map in terms of mapping out the conceptual framework is first of all this issue of changing classroom practice. And really to say if I want to really make an impact in terms of pupil achievement, then what are the implications of getting colleagues to shift their practice to follow the sorts of strategies advocated by the Durham University Sutton Trust report, am advocated, for example, by John Hattie in his book Visible Learning where he says feedback is the single most important powerful strategy and change of pupil achievement and then all the work that most of you are familiar with of Carol Dweckon feedback. So I've got a huge amount of data which says feedback is really good news. But then how do I make sure that my colleagues internalise the models of feedback as opposed to saying to the child that's good, well done. There you are, I've given you feedback. Yeah, because there's a huge gap there isn't there? And does working in teams make any difference? So there I've set out the conceptual relationships but then I've got to begin to identify the components in order to develop criteria which integrate those three elements. So first and foremost this is perhaps the basic stuff, apologies. What are the key text and sources? Well certainly on feedback in the classroom certainly John Hattie, certainly Carol Dweck and others. So I need to map out the conceptual territory and that's identifying the main texts. Which are most relevant and significant in this context. So there's a great deal of literature on teams. The majority of which is very limited in its usefulness in an educational context. Because for example much of literature in teams assumes that people can actually design teams. As you know in education we tend to inherit teams. And therefore teams in education will often be very different because of different contexts and so on. Beginning to look at the relationships between the various areas effective teams seem to spend a lot of time on process issues. They may not be aware of it but they're often talking about how they learn collaboratively. What's the link to that to professional knowledge. And then beginning to integrate and synthesise those ideas so that at the end of that review process developing the conceptual framework I have actually got my own map of how those three elements knowledge, practice and team membership actually might integrate. And I may well identify certain problem areas in doing that. But crucially I've got and it may be two reductionists to say this way I may have half a dozen key factors which help me to identify what it is that I need to focus on. And I then need to focus in on the particular methodological approach and I'm sorry for the fact that thing got corrupted in virtual translation. I'm not going to go through this in detail. But basically again form follows function in the sense that if you are looking at what is essentially the emotional engagement of colleagues in their own learning and asking people who've been in the classroom for 25 or 30 years to change their practice then the methodological implications point towards the right-hand side. This is going to be interpretive. This is going to be subjective. There's not much opportunity for objectivity in this area, is there? If you're talking about how does it feel to have a shared process which questions the orthodoxy of your classroom practice and therefore we need to have an appropriate methodological frame. Let's not pretend that this is going to be objective and scientific. Let's be much more aware of the fact that it is much more likely to be in terms of interpretation and subjectivity. And therefore, and again the literature is incredibly rich on this. But again if you have identified the key themes of your research as being subjective perception of being personal opinion and so on then make sure that your methodological approach absolutely fits that. And therefore the choice of methodology is going to be influenced by things like the scale of the study. Is this one team? And one of the issues always with scale, isn't it, is just to remember that you're working part-time and you have a full-time job and some of you may even have a life. And therefore this has to be real. It has to be possible and not to be too ambitious. How complex are these variables that you're working with? Because again, the greater the complexity the more, you know, the harder the work. And therefore, let's keep it simple let's keep it focused. It's not clever simply to go for elaborate strategies if a very simple and straightforward one will do. What resources do you need? And again what's... Who are you going to focus on? Are you going to look at one team? Are you going to compare teams? All of those factors are really challenging in terms of your capacity as researcher. Think about the nature of the sample group. How sophisticated are your colleagues? And that's a very rude question. But again, you have to think about the sample group. I mean, for example, in my experience over the past month or so, a very, very high proportion of teachers in this country have not heard of Carol Dweck. And therefore to start talking about Carol Dweck's model is some colleagues will actually take you down a real negative pathway, won't it? So you need to be aware of their level of sophistication. And then who's going to use this? Is this just a piece of academic research or might it actually make a difference somewhere? You select, you clarify your methodological approach. You are confident and robust that this doesn't have to be pseudo scientific, that you can use a whole range of strategies. And so, for example, depending on the topic you're looking at, it may be that some of the feminist perspectives in research are the most valid and most appropriate. And again, multiple interpretations of what that short phrase means, but essentially recognising the authenticity and integrity of the individual voice as being valid in its own right has got enormous potential in educational research. We don't always need to have the questionnaire. We need the interview. We sometimes just need to listen to the voice. I'm increasingly interested in biography and narrative, getting people to tell their stories. And that again, I think, is a hugely powerful area, which has been very much vindicated, I think, largely through the feminist perspective. But simply recognising and respecting that my account of my life is actually valid in its own right and is a powerful source of evidence if it is used carefully within a very clearly defined area and you look for the internal consistencies, you triangulate it internally. Of course there's participant observation. You are a member of the team and you are observing your team. Remember first and foremost the Hawthorne effect whereby workers actually improve their productivity in a research exercise, as you all know, not because of anything the management were doing because the researchers were there and they wanted to be, they were so pleased to be being investigated they all worked harder. So it may be that if your team know that you're researching the team they will all be very good. That's a real danger. Think about the big debate about Margaret Mead study and they actually went back and interviewed some of the women who are now very old women that Margaret Mead had interviewed as girls and there's all sorts of issues around academic wars between different cultures in ethnography but at least one group say that actually the women said well she was a very nice young foreigner and we wanted to be polite to her and she seemed very interested in sex so we told a lot of stories about sex and no, was this your life? Oh no, no, no, we wouldn't have dared behave like that. That's gross simplification but actually the issue is that sometimes we need to be really careful and that you need to be aware of your impact as researcher particularly in school situations where your status, your authority might well have impact that needs to be explained. It doesn't mean you can't do it and so on. The analysis of written sources is again a really interesting one and one of the things I'm doing at the moment working with schools on some of the issues about becoming focused on effective teaching and learning is the notion of what is the paper trail in school that demonstrates the focus on the quality of teaching and learning. It's a really interesting issue to say for example the agendas of leadership team meetings how much time is devoted to the quality of teaching and learning looking at the diaries of senior staff looking at the monitoring records for example and very quickly you get a very, very good view as to the extent to which this school is actually seriously committed to focusing in on teaching and learning. Interviews you know about and again the notion is that interviews are powerful but remember that at the moment there are 100 words a minute and have been for nearly an hour and can you really handle that much data and it's overwhelming and life is too short to transcribe tapes and you really need to be careful because sometimes the quality of the interview is wonderful and the data you get is so fabulous but it's just too much and again is this what we need in this context. The focus groups are an interesting one not often used but I think potentially quite powerful and simply looking at again using a different strategy and in a different methodological frame of how social interactions can produce some really powerful insights and the questionnaire, well you all know about the issues around questionnaires but then really significantly see if I were to actually do this piece of research on how have we changed class from practice then for me it could only be a piece of action research because that would provide the iterative structure to say we as a team are working collaboratively to change our practice and that an action research approach would be the only one really that would cover it so therefore that implies a very different methodological frame to some of the others all of those methods are valid but nevertheless we need to be used with great caution and again this notion of following through the internal logic of your study and finally colleagues what sorts of issues do we need to look at and again is that wonderful moment isn't that when you're writing an extended piece of work of any sort and you can see the ends in sight about say five or six hundred words to go and they're the slowest five or six hundred words in the world aren't they and you just want to scrub it off but actually again if your dissertation is a learning experience as it must be surely then it must conclude with some sort of review so is the conceptual framework reflected in the methods used and the structure used to analyse the data because the data analysis must refer to how the data is collected but also to the conceptual framework that actually led to the design of the data collection process that's the logic through have you answered the questions that you posed at the beginning and I think that is absolutely fundamental and so often neglected we started with questions we modified those questions three times in the conclusion I can demonstrate that the answer to these questions is bang bang bang if you have modified why did you do it how did you do it and again build your thinking build the process into the actual dissertation itself and then back to the original are you completing the circle is the internal logic of your dissertation coherent and explicit tell me how you're doing this what you're doing why you're doing it this way and finally are your findings valid will they make a difference and I would add it ought to be on that sheet and what have you learnt from this experience in this process I hope there's been some use to you ladies and gentlemen thank you very much for your tolerance of a crowded and rather full room but I hope there's been something there which has encouraged, supported and hopefully provided some ways forward for you thank you very much if we have time to go on the news I'm trying to be helpful to them and focus to myself you could focus in a triangulation a lot of the work because of the nature of the the study of more questionnaires and so no problem at all but could you be very pragmatic and say what about triangulation are more methods might be really useful that you are considering I mean it's a great challenge isn't it and I mean I would guess that 75-85% of all dissertations done in education involved questionnaires and interviews and the notion is, as I mentioned briefly earlier is to find either a parallel piece of research to your own and if you know do the journal search I mean I hate to do this to you ladies and gentlemen but the chances are that your research will not be unique I know that's a shock and I'm really sorry to raise the matter and even if it's not identical to your own if you can find a journal article which has similar focus to your own piece of work that I think provides really powerful corroboration then you can for example use the critical friend approach get somebody who's not been involved in your research to look at it and act as a sort of mediating agent and I find that a very useful model because I then think that you get very robust and often very critical support but it is actually saying this sounds this looks right from my perspective and then what you might consider is if your work is essentially quantitative then are there qualitative techniques that might help to reinforce and vice versa but crucially I think for me the most important thing is to actually consider using an alternative strategy not replicating your research all over again but actually doing a microcosm and taking one particular part of your findings and using a test piece of alternative data collection in order to explore how that would does it hold up if you use it against other absolutely thank you absolutely thank you for that anything else can we start writing them for you but they'll be helpful another question a lot of students move towards impact stars and impact stars and they're two very interesting concepts could you open them up and say what dimensions might be useful to explore because they are very often engaged I think the thing with impact studies for me is the extent to which intentions are translated into concrete action that really if we're making impact then we're saying we are going to embark upon this particular strategy for example in order to boost literacy in years 5 and 6 we are going to build up a really powerful buddy scheme now the impact is very simple in that case has this made a difference to literacy levels across and so on the one hand literacy sorry impact studies can be very very sharply focused and I think that that's where they are the most useful where you can actually demonstrate that something has changed I think that's why we call them impact studies effectiveness for me is rather more generic and therefore it's the extent to which we are moving towards or have achieved a range of criteria which are more elusive that perhaps the one would imply in an impact study and so as you know the whole school effective movement is posited on very broad criteria and the notion is how can we demonstrate that we are beginning to make progress in that direction and so in essence impact studies focused clearly delineated very much to do with measuring change over a short period of time effectiveness rather longer term rather more generic and crucially may be not as amenable to simple quantitative outcomes thank you again unless you have a third of course just to say thank you very much before I say that more formally at least on behalf of St Mary's University College in the MA and Leading Innovation and Change programme tonight's powerpoint will be available it will be sent to you electronically the video recording will be available on the website on the Leading Innovation and Change website and I'll make sure that you each get a link sent to you and if you do have any comments about tonight's evening then Celine will be asking you to send them back to us but for now thank you all very much for coming I know you've all had very long days you've got a very long day again tomorrow but I think it brings us back Professor Burnham to a question which you raised about data right at the start of this evening's lecture and it's the extent to which we can trust the data which we generate through our own research and I would suggest to you all that testimony to Professor West Burnham's trustworthiness is the number of people that we have here this evening and the sheer joy I think of has been listening to you so I think it's enhanced my understanding is to be a professor a bit more by listening to you this evening but I'd like to thank you thank you very much indeed and thank you all very much for coming tonight thank you all, goodnight, go home