 Thank you very much indeed. Good evening. Thank you very much indeed. Good to know you're there. Anybody else there at the moment? Or are you still recovering from the day? Yn y mwynhwng, fydd gweithio i'r cwylau ydy, yn tynnu i'r bwysigol, i'r ffeiriadau i'r tymon, eto swath. Also you'll be spending quite a bit of time in the course of this session of talking to each other. Just make sure you're sitting next to your intellectual equal. Yn y bwysig, dyna'r ffordd. Rhaid i'n ffordd. Rwy'n ffordd, mae'n dweud hynnyd i'r ffordd, oherwydd y ffordd yn y ddechrau ymlaen, yn y gallu'r iawn. Mae'n rhaid i'r ffordd ymlaen yw'r ffordd yn y gallu ardal o'r cyffredin. Trofodd y ddechrau, oherwydd y rheswch ar y Cymru, Mae'r mwynhau hyn yn diogelio, a oes o hynny yw'r 12,000 rhaid. Mae'r bwrdd wedi'u lluniau a rhaglenジodau gennym i'g ddim yn ysgrif iawn, ac mae'n rhoi arwain yn y cydwg ac mae'n ysgrif iawn, oherwydd mae'r cyflasio o'r cyffredinaniaeth. Ac mae hefyd yn rhoi, mae'n callu i'n mynd i gael i'w ffwrdd mewn eich ffordd. Mae'r gwyllt arall wedi amser, ond mae'n mynd i'ch gael i'ch gael i'ch gael i'ch gael i'ch gael i llawer o ddeibwyd, i'r cyfnod yn gallu am meddwl i'r dyfodol, i'n meddwl i gefnod hwnnw yn ddigwyddol, ond byddaeth fel y myfydli nad ydym ysgol yma yma'n ei wneud y dyfalio, fel yей fod yna ga'r myfydli yn ni weld y ddweud. Mae'n myfydli wedi bod'i gweithio'r ddwybyn iawn o gyfa. Maen nhw, mae'n gweithio â'r cyfnod, byddwyd ychydig papur ac mae'n da i'r cyflym, os rydw i'n ddechrau'r cyfrwdd ddwybyn iawn i'r cydyllidau. Felly, rydyn ni'n gwybod yw'r bwysig ffagmatig. Rydyn ni'n gwybod y ffordd yw'r ystod, sy'n ddigon i'r ffordd yw'r prifyddiad yn ymddangos o'r coleg sy'n ddigon i'r ffordd yw'r pwysig P.H.D. A ydych chi'n rhaid i'r pwysig yw 99,500 rhaid. Ac mae'n meddwl i'r gwybod, mae'n meddwl i'r prifyddiadau. A'r ffordd yw'r ysgrifennu yw'r ysgrifennu o'r ffordd yw'r ysgrifennu. Yes it's her life's work basically. And so if you think about writing 99,000 quick', which is longer than most novels, then you've really got to be focussed and believed that this is something important and interesting for you. So the first issue I put to you because under pressure from work and the pressure from home and under the fact that you've got to do this in all sorts of different times and spaces and so on you need a topic that is of significant significance to you. You need a topic that's got legs. You need a topic that you can perceive some value in completing and I can't stress that strongly enough because otherwise it can become a chore, and that's a rid tragedy. And I think that's really sad if you're just plowering through the words to get there, and you're not saying this is useful, this is significant or even enjoyable. Llyr eich gwaith yn eu cael ei ddweud diwrnwys sy'n gallu gwirio y dyfodol, sydd wedi gweithio'n middol am y bywyd. A gyd yn dweud y byddai yn ei gael chi amlaenwyr. Rydym yn gyfio'r cychwyn i ddwy'r bydd yn allan o'r byw, ac yn defnyddio'r peth yw'r�n dweud arnyn i'r penderl Pleaseb. Mae hwyl drwy hyn wedi'u hwn i ddyddio'r gweithio'r byw, y byddai'r fyrdi'n cyfryd o'r gwneud y byddai'r bwaith mewn. Dieb yw yn cyfle alarmol ac mae'n gwneud i jechyd yn ddod o'r swydd ymlaen. Mae'r gwiswr llyfrion gyda ddwyngannau'n amlwgol, sefyllfa y mae'r yngrifennu gyda'r ysgol adeiladau yma yn gyflu'r Ysgol ysgol. Bydd yma'n ei wneud i ddylu iawn o'r ysgol i'r ysgol yn gweinydd, ac mae'n sgwrn i'r cychwyn fydd yn gweithio'i'r llyfr. I gwaith y bydd rhesiwle o'r mater, a gwneud i dda gwaith, a hynny'n cael y cydwyllwch ar ddweud, ac fewn i gael y dibwysig, yn ei weith o hwn yma, a gwneud i fel, ac i fi'n glis yw sydd wedi chi gweithio effaith ymlaen i chi fel y dyfodol. Ac rydyn ni'n rhan o'r adael y chyfledd ychydig i chi wedi gwneud. A gyda'r wyxfodus a yn ei ffibwysgwr oherwydd y cyd-dynnu'r holl mwr fydd ddigon o'r dydd. Mae'n rhan o'ch hunain. Mae'n fawr o'r ffibwysgwr oherwydd ymweld fydd yw'r bach o'r cyfasol yn dod, ac mae'n rhan o'ch yn fawr o'r fawr o'r fawr oherwydd ymweld yw'r ddysgu. Mae'n rhan o'ch yn fawr o'r ddysgu'r ddysgu'r fawr o'r ddysgu'r ddysgu'r ddysgu'r ddysgu'r... maen nhw'n ddweud y lluniau, a hwn yn ymdweud y cwrs. Rydyn ni ddim yn ystyried o'r modd o'n ddweud, ac rydyn ni'n ddweud, rydyn ni'n ddweud, rydyn ni'n ddweud y bwysig mewn gwir, rydyn ni'n ddweud, ac rydyn ni'n ddweud y bwysig ar y lleol yw'r rheinyddiant. Felly, yma'r wych yn ysgriffyniad yn ddweud mewn gweithio yw'r rhagion ar y gweithio mewn gwir, Can I really get the mileage out of it? And pivotally, is it possible? Is it doable? Simple as that. Because you may have very, very real aspirations, but actually is it going to make a difference? Is it going to work for you? So that's the first issue I put to you. The second issue is question one, which is what are you actually looking to do? And it may be that you are saying, I'm going to contribute to the sum of human knowledge. And that's wonderful. But that's not maybe the most useful use of this kind of activity. You may be that you find that if you don't get it right, you're simply contributing to the sum of human misery. So you do need to be consciously aware that the purpose of doing this research is to, and then I think usually it's something to do with school improvement, isn't it? It is about looking at, in the final analysis, a changing practice. It is about moving your team, your colleagues, your department, your understanding in school forward on a certain topic. And that really is the most powerful form of research in education, I think. That research which is absolutely focused on how do we make this better? How do we make this work? Crucially, how do we really build confidence that my time, my energy is going to make an impact? And what I would suggest to you is that an acid test all the way through is, what will happen as a result of me doing this research? And therefore, it's got to be personally engaging, it's got to be professionally appropriate, it's got to be morally appropriate. OK, lots of nodding going on around there. Is that all felico hearing for everybody? So, you've already started thinking, you've already started the process. With your neighbour please two things. Firstly, hello, how are you if you haven't already said so. And then secondly, what looks to be at the moment the key topic that is going to dominate your thinking this evening and in terms of your decision. Have you identified it? What does it look like? Where are you going? What at the moment is your understanding of the area that you want to work in? Just a two or three minute introductory buzz please. Any thoughts at the stage or is it too early in the session for conversation? OK, I'll move us on. Now, part of the process of really producing a compelling piece of work that actually does make a difference is to be, as I just hinted earlier, very very sensitive to context. And therefore, you need to be very rigorous I think in terms of really defining the context because in many respects all research is context specific isn't it? You cannot have context free research essentially. And one of the ways I think of thinking about context is to think in terms of the three Ss, self, school and system. And so, the context as far as the systems concerned at the moment is probably one of the periods of greatest of evil in education in two generations in this country. We're going through more change and I'm just listening to Mr Gove on Thursday morning at 10.30. He's going to announce even more changes. And basically we have a fundamental shift in the way in which our education system works. And it's very difficult to exaggerate or engage in hyperbole about just how much has changed. So your research is being done at one of the most dynamic and turbulent periods in school life, in living memory basically. So that must have a very very real issue isn't it? So the system is about the policies that are impacting upon your school. And it would be naive and very limiting to have a piece of research which ignored the fact that the Ofsted framework has changed out of all recognition. That the policies from central government have focused almost entirely upon raising achievement, however defined. And that the legal status of schools is changing at a hitherto unprecedented rate. All of that points to a very very different working environment which may explain why many of you are so tired. Even though you've just had half term you're tired aren't you? Never mind only eight weeks to go is it? So your research needs to be contextualized in terms of recognizing that the demands upon the profession in terms of centrally generated policies are very very fundamental. The school issues again are a pivotal area in terms of your research being a success. Recognizing just where your school is and also the politics of the school in terms of issues which are or are not appropriate for you to raise and so on. I suppose that for virtually every school in the country at the moment the driving force is this imperative in terms of closing the gap isn't it? That's the one that makes the difference. To what extent is your school demonstrating progress? To what extent are you moving towards a situation where the majority of lessons i.e. more than 80% of lessons are good or good without standing features or outstanding? Because that's the bottom line at the moment isn't it? And I was working in a school yesterday that's in a fairly challenging part of Midlands Town sorry and they've just got good secondary academy. And the reason they didn't get outstanding even though they made remarkable progress was simply because there were not enough outstanding lessons. So if we're being pragmatic then we say look if I've got any energy and if I'm focusing in on my workplace then the focus on the quality of teaching and learning has to be one of the driving forces doesn't it? And I would guess that most schools now are addressing those two in a way that's never been the case before. Certainly the new off-stead framework provides you with a very different culture of working doesn't it? So it's that kind of issue that you say if we look at the big picture and then we look at the school picture what are the really significant issues that are going to influence whatever topic I choose on to do. But they will have an impact on it. And then thirdly don't forget yourself in all of this. Because one of the issues I think that again I regard it as fundamental to any piece of research is that you are not an anonymous writer. You are an actively engaged professional and therefore where are you in all of this? What's your perspective? Crucially where are you coming from? What's your role in school? What do you see as the key responsibilities in school and so on? And of course this will change your own position, your school's position and the policy position will all change. And so the news from what a fortnight ago now that all of the proposals for changing off-stead are going to be implemented apart from the no notice inspections. And so you can't be satisfactory anymore, you're requiring improvement and you can't be requiring improvement twice. And so if you're working in a school that I'm working with at the moment in Southampton they had inspection in March they were satisfactory. But that means they're going to be good next time doesn't it? No option or they're going to be taken over. That's the context issue isn't it? Let's be real, let's be pragmatic and say there are real live issues here that need to be addressed and that at the heart of them is teaching and learning. Now that's being as I say very pragmatic but nevertheless if you're going to do any kind of research which has an impact upon the quality of the way in which your school functions then I think all of those have to be taken into account. Is that fair? Is that reasonable? Just for my sake as much as for students linked with the context of the school. So along the students here you might have independent schools, personal schools, cwmrems, whatever. And some people are in traditional schools where they have a mix of the autocratic and full spectrum. And all I'm really asking here is how cautious or how much nuanced language should a student use because it can't just say that the head's broken so this should go away. No that is probably not a good idea. So there they are, they're MA students right now in Cadellian, and in the contextualised in the school it has to be very precise with language or fields if they're attacking Cedeleusia to you or the head of faculties. It's a really interesting point isn't it because you can't anonymise yourself out of existence basically. I mean there are procedures whereby you can anonymise but what you can't do is deny the fact that you're writing about your school. And what you can't do is put in a dissertation which has no name on it basically. And therefore you're very subtle, you've got to be very cautious in saying there may be no go areas here. That if the school is extremely dysfunctional because of the personal habits of the headteacher then I really don't think you can actually put those in print. Well it depends what they are of course but if they are pleasant personal habits then we really don't want to know. And you can't actually, the laws of slander are really quite significant. So that gives you an immediate caution doesn't it that if it comes down to the quality of human relationships in school then I really am not going to be able to go there unless I've got incredibly robust evidence and I'm prepared to accept the consequences of going public. And that's a bit like the guy I mentioned earlier who did the marketing stuff. He ignored the internal structures and relationships and so on. So you've got to be politically savvy and you've got to work in an area where you can be confidently publicly robust. Would you agree? It's no good having a wonderful topic which is about the total dysfunctionality of the senior leadership team. Can I research this please? Oh yes please do come in and sit on our meetings. We know you're fighting all the time and we know that your meetings are a shambles and close to anarchy. And I'd like to write that up please. Well why not? It's nonsense isn't it? So therefore you've got to be very clear that you are working within an area where basically A you have access and that's really important and B it can be public because technically the university publishes your dissertations. I know it doesn't come out in the book for that but actually they become public documents and therefore thank you it's a very helpful point you made. You've got to be very clear in mapping out your project that this is something that I can go public on and this is something where I won't find the doors being closed in my face because of the sensitivity of the issues involved. Why is it that the Underwater Basket Weaving Department in this school is so bad? Well the head of Underwater Basket Weaving may have a problem with you doing that. Yep and there's the challenge. Any other examples around that? Would that be then any way of anonymity? Yes but I mean that is the off-state report and again don't take this as being literally true. The off-state report is objective data of a type. And therefore it's in the public domain and you can use it and in fact if you're looking at issues around certain factors then the off-state report may be a very very powerful piece of evidence in terms of explaining the school's current context. But again you've got to be cautious because for example in the school I was referring to earlier they have a sentence in their report which says something like this school is good but not yet outstanding because not enough lessons are of outstanding quality. Well that's quite challenging. So you've got to think about the internment politics of the school. How do you actually engage with colleagues around the fact that a significant proportion of the lessons are not of that standard? And people are very sensitive about that but that's the sort of issue that may be necessary to engage with if you're really going to make an impact in terms of improvement and change. So we avoid the hard topics but they're the important topics often. So thank you again for that. It's very helpful. Okay so let's then move into the detail and this really is almost sort of playing with people like playing with fridge magnets. How do I arrange all these ideas? And let me give you an example of the sorts of topics there is that you might need to engage with. So for example closing the gap is a very significant component I think of any focus on teaching and learning and leadership in schools at the moment simply because it is the driving force. And so therefore at some point you need to address the issue of your understanding of what it means in terms of this gap. Sometimes it's referred to as the long tail of underachives now. And what does it look like in your context? So nationally we have a problem where more young people fail to reach a great standard in this system than in most of the European system. Some schools have no gap in this country, others have a very long time indeed. So one of the key components of your conceptual framework, the contextual issues, is going to be something to do with closing the gap. And although I have no problems at all with totally ignoring Ofsted it's going to be very difficult isn't it? Because for example if you're doing work around leadership then the Ofsted framework has totally changed our understanding of what school leadership looks like, hasn't it? If you haven't a chance to read the framework if you read the section on leadership then basically what it's saying is the function of school leaders, whether senior leaders or middle leaders is to improve the quality of teaching learning. That is a fundamental shift from the 101 things that school leaders could do. If you think about the previous Ofsted framework of 27 categories and the current Ofsted framework with four categories then there is a real shift in terms of the conceptual framework that we're working on. In other words the language has changed, the priorities have changed and therefore you may need to be very much focused in on the issues around the prevailing orthodoxy in terms of accountability and the criteria for what constitutes good practice and so on. And that's the issue around classroom practice. And again one has to say if you have got a certain amount of time, a certain amount of energy available to you, what's the most significant thing to look at? It is almost certainly classroom practice isn't it? That's the one that's going to make or break schools, that's the one that is most significant, that's the one that really I think is at the heart of most of what's going on. Now there are 101 different ways of approaching that. But one of the challenges is, and this can be worked at many different levels, is simply how confident and robust is your school with regard to the quality of teaching and practice and basically how do we understand that process in terms of how do we improve, how do we support? For example, there is a great deal of evidence and again you've got to debate it which points out that from a number of perspectives the single most powerful change that will improve the performance of many young people is to do with the quality of teacher feedback in the classroom and I think that's fairly compelling. So for example you've got John Hattie's book Visible Learning which is the most definitive synthesis of research that's ever been around classroom practice. And just to give you an idea of how academics live their lives, for the past 25 years or so Professor Hattie has looked at every single piece of research ever done on effective classroom practice. He and his team have looked at over 500,000 pieces of classroom based research. You really think what sort of life is involved in that don't you? And the book Visible Learning which of course every classroom in your school has a copy of doesn't it? Really is the synthesis of all that research and what it says in impeccable and very very robust language is basically the feedback that teachers give students is the single most powerful vehicle for bringing about improved achievement. And you say that's really compelling. So if we're looking at a context in which we're looking at closing the gap, the off-stead frameworks focus on teaching and learning, the centrality of classroom practice and everything that goes on in the current climate, then hey we've got wonderful news. John Hattie says and then we can go and look at the work of Carol Dweck. Yeah, if you haven't come across Carol Dweck, her website is just caraldweck.org and she spells her name D-W-E-C-K. And her research points to the fact that guess what, the single most powerful factor in improving pupil achievement, student achievement, is the quality of feedback. But Dweck says the sort of feedback that many teachers give, i.e. what a good piece of work you are a good lad, is totally the wrong sort. What we should be saying, that's good but can't you improve it? This is on the interaction in the classroom now. And so that was a very useful question but could you speak up please? So I'm giving you positive affirmations but I'm actually challenging you and that's what Carol Dweck says makes the difference. And so in John Hattie's work, the formative assessment based on written work, that comes about fourth or fifth down the list and it's important. But according to Dweck and according to Hattie it's the way that you and I interact and the way that I'm saying basically this would be good if you had put in more effort. That this would be good if you thought about doing it in a different way. The school on Merseyside, their mission statement if you accept the value of such things, is be kind, work hard. And that's interesting isn't it because that's the Carol Dweck notion that actually we don't work hard enough. I've done some work here, you say something positive to me about it, well done John, you've tried really hard there. Now that's good John but it could be better. So for some colleagues that's really challenging isn't it? So you've got Hattie, you've got Dweck and then you've got the Sutton Trust report published by the report by the University of Durham published by the Sutton Trust in May of last year on how to spend your pupil premium. And are you all familiar with this? Great, because what it does of course is to say if you really want to improve academic achievement in schools then do not even think about reducing class size. Because as you know reducing class size will lead to a fall in standards won't it? All agreed on that? Great, no argument there then, that's good isn't it? Because if we were to reduce, mathematicians help me here, if we were to reduce the average class size in primary schools say from 28 to 30 down to 20 how many more teachers would we need? Round about 150,000. Now even with the army being closed down we're not going to get enough are we? And according to some very distinguished academics actually reducing class size from 30 down to 20 would make no difference, in fact it would make things worse. It's only when we get down to 12 to 15 it begins to bite. And equally the presence in your classrooms of teaching assistants has made no impact over the past 10, 12 years on standards has it? No, thank you very much indeed. We now have 270% more people in schools than we had in 1999 but they've made virtually no discernible impact upon achievement. They've made an impact in all sorts of wonderful ways, they're great people but what they've not done is improve performance. What according to the Sutton Trust report makes the difference with regard to performance in classrooms is first and foremost, guess what, feedback. Can you see where I'm taking you colleagues? And the notion is that what you're doing is saying in this context what seems to making the difference. And so you go through all of those processes and say the evidence is overwhelming. And again it depends how you regard evidence that this is what we should be doing. But then we have the problem of identifying what it actually looks like and that's available. But then we move on to the final component which is that if we're really going to secure sustained achievement across the school then it might just be that some of our colleagues have got to engage in personal change. And that's where a topic like this gets interesting. And there's the challenge isn't it? Because this is, I mean hopefully you've seen there's a fairly neat logic in going from closing the gap, the off-state framework, classroom practice, focusing on feedback but then we come up to the big challenge don't we? If we're really going to make a difference you've got to take somebody like Mr West Burnham who's been in teaching for 42 years and basically prefers to work in his own rather idiosyncratic way. And John Hattie's got a wonderful phrase, most teachers do little harm. Isn't that great? And unfortunately this teacher does. But I've decided to stand up for a few years to help you out. Because I will work in my usual idiosyncratic way where I work according to my terms, according to my way of comfort in the classroom. Come on, make me change. Any volunteers to make me change please? Because that's when you move into the leadership issues, that's when you move into the pedagogic issues isn't it? That's when suddenly this project which was going so well moves into a totally different category. Because in the final analysis what you cannot do is say, hello John I've come to research how you change your practice in the classroom. Because I will say there are certain key misapprehensions, firstly that you're coming into my classroom and secondly that I'm going to change. And there's the challenge isn't it? That's why the notion that I'm going to research the issues around changing practice in classrooms in school may be slightly naive. Does that all make sense colleagues? Yep, so you go down from the big picture to the school picture but when you come down to my practice in my classroom no thank you door shut. And that is one of the great challenges. We're doing research which by most criteria is perfectly important but which takes us into very very sensitive areas. And I'm sure you can all think and in fact play with it now please. The notion is that your research is based upon good will isn't it? Your research is based upon volunteers and therefore your topic has to have the capacity to actually allow people to engage with it. So I've just talked you through one model of a potential person until we come to the real sticking point. So it may be that what you can't do is research how people actually change their practice. You may have to research people's perceptions about changing their practice. That's valid. You may have to talk to colleagues who are middle leaders and say how do you perceive your role in changing practice? Because it's that sort of issue which is the really challenging one. So let's look at the nice neat logic that it's all available and I mean the issue of the topic of feedback has got some fabulous research associated with it. Incredibly robust but it comes to the point eventually where it is simply too difficult to make it work in practice and maybe outside the scope of your research. So if you would please just for five minutes or so and those of you who are still beginning to come round there's more cake, there's more tea, help yourselves but basically what are the boundaries of your research please? Any issues? Any thoughts? Several of you just raised them personally but did that make sense, that process? Yes. Good. So therefore let's be pragmatic, let's be realistic, let's be very clear that there are boundaries in which we are going to have to work within if we're going to deliver a coherent and useful piece of research. And sometimes it's not so much how are we going to change classroom practice, it's rather what's the people's attitudes towards changing classroom practice, that in itself is valuable, isn't it? So don't expect that the world will change because you're doing a piece of small scale research but at the same time don't lose ambition, don't lose aspiration in terms of the potential, it's where you feel your research is going to make a difference. That's the important thing. You answer meaning, you've spoken about perhaps his work and perhaps his work as being based upon robust data and you've just spoken about clear and clear and research inquiry. Could you define and differentiate perhaps as we go through the course of this evening to make clear what you understand as a robust research inquiry? Thank you for that. Think of any television programme or film drama based around a court case. And if it's based in this country then it's all about baristers, isn't it? And it's all about baristers putting forward evidence. Now very crudely, as you are aware, there's a continuum of evidence which in American movies is basically over here with DNA, which is usually available within half an hour of the murder taking place. Yep. And DNA, as far as we know, although there's some question marked over it now, but as far as we know DNA is basically absolute certainty. So DNA, hard scientific based evidence is virtually, if you like, 100% reliable. It's not, but it's as close as. At the other end of the spectrum is basically circumstantial evidence. And the laws of evidence in British courts are very clear as to where juries should place their greatest trust. So the issue I would put to you ladies and gentlemen is basically what sources of evidence do you trust? Yep, and that's very personal because the anecdotal evidence sometimes is immensely powerful. And never forget your own experience. As I was just saying, I introduced the hatty work on TAs to a group of primary deputies and said basically your TAs are a waste of money you'd be far better off getting rid of them and employing two, three, four full-time qualified teachers instead. And one of them said that's rubbish because my TA is magical. And my response was your TA may be magical but the evidence is that she is part of a group who have not made a contribution towards raising achievement. And there's the tension. What I know in my experience, which is robust to me but may not be, and here's the key word, generalisable. In other words, can we make this a universal proposition? Is it true in all cases? And I think that is where the big debate comes in. Hatty has spent most of his adult life scouring different sources of research evidence which in themselves are, here come the key words, valid and reliable in order to synthesise and make a macro study, is how he describes it, which is based upon components which are highly reliable of themselves and the argument there is that because this is made up of reliable studies, in turn the macro study becomes totally reliable and robust and trustworthy and therefore you can use it. The Sutton Trust report and they are very clear about just how cautious they are about certain aspects because their evidence base is not as strong as Hatty's but they're still very clear that they are confident in their outcomes and then the Carol Dweck work is based upon years and years of research based in cognitive psychology. So again you see that's where we can be robust. This is not somebody saying here's a good idea but this takes us into the problem we were just discussing that education policy at national level and at school level may not take evidence into account. And one of the great challenges and there's a huge amount of work based on this area is the extent to which leadership and practice in schools in this country is evidence based because if it was then we would be, if you take the Sutton Trust report, their top three areas in terms of impact on achievement and cost effectiveness are firstly feedback, secondly cognitive development, i.e. philosophy for children as an example and thirdly peer support. And they're saying those three have the greatest impact and therefore are most worthy of your attention. I haven't come across many schools where those three have been put into place in the past 12 months since that report was published. Why? Because a, local practice, b, uncertainty, ambiguity about, can we trust this research? And remember that in this country the word academic in the education profession is a term of abuse. We can't do that, it's far too academic. Isn't that true? So we have a different culture at work but what I would suggest to you is that sometimes it's entirely right and proper to be over here on the circumstantial side, sometimes you have to be over here on the forensic side. For example, if you are exploring teachers' emotional response to change in their classroom practice and see how I put in a caveat word immediately their emotional response to change, then I think it's entirely proper to collect anecdotal evidence. Because it's not anecdotal evidence, it's personal life stories, isn't it? And there's a very, very strong and very powerful methodological argument in favour of using that kind of data. But then you have to be very careful about how much credence you place on your conclusions because what you are hard dealing with is evidence based upon people's emotional responses which are not subject to any kind of formal cross-referencing. And therefore you're talking about people's emotional outbursts essentially. But the notion of narrative of biography I think is hugely powerful but we have to remember not to base policy on it in the first instance but corroborate and extend. Does that go some way? So see yourselves essentially as researchers in the position of baristers in your job, in your dissertation is to amass evidence partly through your literature review that says here's the corroborate of evidence and then through your own investigation which says here's the specific evidence. And then you have increased confidence and therefore I can trust what you say. Is that right? And I think that I'm always anxious at master's level about going into massive amounts of statistical work but basically you need to have the same disciplines with regard to your evidence as the statistician does with regard to the data that they generate. It's got to be valid, it's got to be reliable. All accountability and reliability have you spent some time on that? You're comfortable? That's fine, we'll move on then. Sorry. Never apologise. We're glad you're here. I'm just thinking about triangulation of my data because I'm looking at guess and parental responses to a particular scenario. Is it all right to triangulate within the same paradigm? Yes. Because I'm not using specific data but I'm using maybe into parents, students and teachers. Absolutely right. Did you hear that wonderful phrase then ladies and gentlemen? Triangulation within the same paradigm. This is a man who's going places, right? The language is there. So that's exactly right because you have created a paradigm which is the contextual issue for your research. And then within that paradigm you are making sure there is consistency in the relationships between the different elements. And that's great because the more you triangulate the more you find internal consistency in your paradigm. The higher is the trustworthiness of your research. Thank you for that. I'm a scientist which I think is a bit of a hindrance. No-one commented when you said about that in the reliable. I've gone alone. I'll speak to you at another point. Could you just briefly say something about Felicity in the library? Yes, sure. Let me try and discuss it this way. If I get this wrong then shout me down. Some people, apparently, are concerned about their health and well-being and their major measure of checking this is by weighing themselves, checking on their weight. And so they go to the bathroom once a week and they weigh themselves, let's say, on week one they're 75 kilos, week two they're 76 kilos, week three they're 80 kilos, that was a good party, and so on. And that what you're finding is that if you actually record the scores and you know this scientifically and you find there's a pattern emerging where they're clustering, then you've got reliability because basically there aren't extremes. Yep. Now then, that's reliability. You can basically say that each time you test the machine it comes up with broadly consistent results. However, when you go to another machine you find that in fact you weigh 60 kilos. So although your machine is reliable it's not actually valid. And many of you may have come across the experience that you know how much you weigh until you go to the clinic. When the medical scales give you a totally false reading several kilos ahead of your own. And does that make sense, that example? And so basically I'm going to investigate pupil perceptions of. And so I need to make sure that it's reliable in the sense that the questionnaire I'm using produces the same results. Yep. But then it's also got to be valid in the sense that it's actually describing what I wanted to describe. And that means for example the ambiguity that you find with some questions and questionnaires. And therefore if you, that's why you must that's why you must trial a questionnaire especially if you're going to have a quantitative element within it in order to make sure that it is reliable i.e. you get a broadly consistent feedback but also it's valid because it's actually telling you what you need to know. It's obviously very subjective. Can it still be valid and reliable if I'm using anecdote or a group interviews and observations from the type line that helps? When we move from the allegedly objective into the subjective then I have no problem with that at all as long as you are very precise in saying this is the status of this data. Yep. And we'll come on to this in more detail in just a moment but basically I think the crucial thing is that virtually any form of evidence is acceptable as long as you acknowledge the fact that there are different types of evidence and you say and my evidence is this type of evidence and it's trustworthy because I have triangulated it within the paradigm to use a well-known phrasal saying that's been invented this evening. And that process of triangulation so there's a big row at the moment, well not so much at the moment it's blown over now with regard to field studies in anthropology, ethnography and this was prompted by an Australian anthropologist who basically challenged Margaret Mead's work growing up in Samoa and growing up in Samoa was one of the pivotal pieces of anthropological study and it gave a real insight into a society which was totally outside most western peoples understanding and what does an Australian anthropologist went to Samoa and said to the old ladies does anybody here remembering talking to an American lady about 60 years ago and several of them were still alive and so this guy had conversations with them about their conversations with Margaret Mead and they said well she was a lovely girl and she was very interested in our sex lives and we told her some stories to keep her happy and the issue is that according to this Australian anthropologist Mead was simply beguiled by the exotic and didn't test, didn't corroborate and therefore she wrote a book about a society which was based upon fiction because it was so powerful at the time everybody accepted it but now even within a study of simple societies you've got a triangulate you've got a test you've got to say is this really the case or is this just people being nice to the visitor and that as you know is often referred to in another context as the Hawthorne effect whereby the very act of talking to somebody changes their perception and their engagement and if you haven't heard about the Hawthorne effect it was studied then in the 1930s and I think it was an electrical components factory and they had some sociologists in to say what makes people work harder and they looked at the figures in terms of productivity and so on and they discovered that the most important factor in productivity increasing were visits by the sociologists in other words it's being part of a process which made people behave in a different way so you need to be very very punctilious for example if you are researching with children the ethical issues on that are fundamental informed consent is absolutely vital then one of the challenges is how can you be sure that the pupils that you are questioning are replying to you as researcher or you as their favourite teacher that they want to please you or they give you the answers that you think and that's one of the challenges and that's why some form of triangulation is absolutely essential and so for example you use different techniques so it may be that the most powerful sorts of ideas and I did some work a few years ago on children's spirituality and the stuff they spoke about was just wonderful but we had to be very careful in terms of saying this is spirituality because what we were doing I think was actually imposing our values in a way that might not have been appropriate yep so it's really important points that you've raised the notion is how can I increase confidence in other words do I convict or equip on the basis of the evidence that I presented it's also really important that when you're doing the trial of your research tool that you take that you just test the extent to which you're asking leading questions as well because for all you want to count the Hawthorne effect if you ask a leading question you'll get the answer that you want so I think you also need to be very careful when you're not asking a tagged type of question so try triangulation through different techniques and give an example of that so is that just like using a survey so that's not wise yep absolutely so you might for example teacher appraisal now they are guidelines they're not mandatory but they put forward their preferred model of teacher appraisal in schools so you're working in a school of 100 teaching staff your research is to look at teacher attitudes towards the appraisal process now you can't interview 100 teachers that would be the amount of data would simply be unmanagable that's another issue by the way make sure you can manage your data but basically what you might decide to do is firstly to survey in a fairly simple way all the staff to get a broad quantifiable response in terms of a range of particular areas which you've identified has been significant on the basis of that you could then go and say I'm going to interview and here comes the next big challenge what sample are you going to look at and so with 100 staff you might say I'm going to have a small sample of say 10, 12 and they're going to be a vertical section of the school in terms of hierarchy management structure and so on so I can differentiate between different attitudes so then you've got more specific data and then thirdly what you may do is simply go back to another sample and say here's what this sample said how does that coincide with what you say does that work yes and so each time you take the same basic data and investigate it in a different way you are building confidence I would argue yep thank you I'm going to do it for time heaven's above are you all comfortable are people alright so where are we we're only on four I'm afraid so it may be midnight but meals and blankets will be brought in now then I think that one of the one of the causes of most anxiety overdoing research projects and one of the areas where people do get into difficulty is because they don't do what a former colleague of mine said was the best and simple most important rule in terms of successful research which is a three part rule part number one focus part number two focus part number three focus focus focus focus and so for example we're looking at the issue of moving practice across the school into certain areas which we know are more successful than others and therefore it may be that the focus of your research is as it says in question teacher attitudes to ensuring consistent practice in classrooms yep so that one of the arguments is that the way that you get rid of this tale of underachievement the way that you increase confidence the way that you get outstanding from ofsted is by high levels of consistency so you are not going to investigate how to make teaching in your school more consistent because that's beyond your remit but what would be interesting and again it's laying foundations for future activities to say let's look at people's attitudes to the notion that if you like there are certain non-negotiables in terms of classroom practice in this school we are going to say that there are things that we expect everybody to do and so your focus is how the teachers respond to the notion that irrespective of ofsted or secretary of state or whatever that we are expecting you to adopt certain strategies irrespective of your personal preferences that's interesting isn't it because in some schools the culture is essentially laissez faire when it comes to teaching and learning in other schools things are far more directed but if we are really going to close the gap and here comes a real leadership challenge then we need to be confident that more teachers are using effective strategies for most of the time and in some aspects of our educational system that is a very very different way of approaching things now then you're not going to research how it works in practice but what you can do with appropriate support from senior colleagues is to simply say let's test out the waters how do people feel about this how do you feel for example that you should begin to use a different range of strategies according to for example in the ofsted framework it's very clear I haven't got the words exactly right but that every student including vulnerable students makes progress every lesson now that strikes me as a basic human right so what do we have to do in order to achieve that where the answer is we have to have certain strategies in place which are non-negotiable and absolutely built into requirements the role of middle leaders is to secure that so for example you might have a project which is saying to middle leader colleagues how do you feel about having to and the word here is probably wrong not in force but secure consistent practice in certain aspects of people's classroom work that would be quite an interesting topic wouldn't it because in some context we are not used to that at all but the issue is if this is robust high confidence then shouldn't everybody be using it irrespective of their personal beliefs or comfort zones or whatever one of the interesting things about medical research is that if they find for example that they're researching into a particular cancer drug and there's a case about this quite recently wasn't there if they find it actually works then the trial is stopped immediately and it becomes available that one of the really interesting things about for example surgical practice is that if somebody finds a better way of dealing with a particular issue then that goes on to the internet almost immediately the medical profession are very very good at sharing the best practice and there are certain things as you know if you've been anywhere near hospitals is that there are certain negotiables in terms of good practice aren't there you do not do these things I had a brief medical experience recently and it was very interesting because none of the male staff wore ties or jackets it was all open neck shirts and short sleeve shirts because that's one of the best ways ties apparently are really disgusting in the hospital environment because they pick up everything so don't wear ties how do you know who the consultants are because they used to wear bow ties didn't they that was better but the notion is that it's based on good hygiene practice but it actually requires a cultural shift to say you don't go to work if you're a senior doctor wearing a jacket and a tie you go to work wearing a short sleeve shirt and an open neck shirt that's totally different in terms of perception in terms of behaviours and so on that's difficult in human beings isn't it anything else on this one so get the focus right now point number five I'm sure most of you have already done work on this and this again is personal I'm not saying there's one right answer my own view is that you need to think of your research as essentially a problem solving activity and the best way to set that up is to have a research question which probably comes at the end of your first chapter the context chapter if you like if that's the way you decide to do it and saying on the basis of the issues that I'm exploring I'm going to come down to one research key research issue which is my big research question and again this is purely my own personal way of working so don't take it too seriously but I think that eventually that research question becomes your title because then you've got the focus clear focus focus focus all the way through and that there may be subordinate research questions so there may be I'm going to research teacher perceptions of consistency so one subordinate question will be what does the literature tell me about effective practice another research question will be what are the methodological approaches that are most appropriate for finding this kind of evidence out and so on and so on and so on and again and this may be too too oversimplified but basically your research questions help you structure your final thesis they give you the key themes it's a bit like in a piece of music there are three, four, five main themes and they come in sequence and then at the end you know exactly where you are because all the various different themes have been played does that make any sense it's nod or smile so there's an issue about literature and one of the the key things and again this is personal anecdotal therefore we do it with caution but sometimes you read dissertations and I think the structure approach here is very different but you almost read four or five different chapters in the sense of here's my introduction here's the literature, here's the methodology here's the results here's the conclusions and what I would suggest to you ladies and gentlemen is that you need to have some kind of continual thread running through and that's your conceptual framework so for example given the context of my school given the issues that we face in terms of our intake given our culture, given our history then I think that the most appropriate way of conceptualising, improving teaching learning is and you take the evidence and you synthesise it and that gives you the foundations for your questionnaires etc etc for example the issue which is going to be very challenging in some secondary schools that everybody teaches literacy first that your role as a subject teacher is subordinate to your role as a facilitator of literacy now then what does that mean in practice well we need a conceptual map of the role of the teacher with regard to engaging with literacy and using history, geography maths, science as a vehicle for securing literacy because it's the literacy that really makes the difference isn't it if you ain't got the literacy you're going nowhere but you need to explain exactly what you mean by literacy yes because it is a contested topic isn't it and therefore you need to be aware of the pressure that the department will be putting on primary schools in terms of all the things that Mr Gove was talking about he's talking about them on Thursday but they were leaked on Sunday but basically learn a poem by the age of five etc etc now I think the Iliad may be too much for certain infants but I think in reception they could do with Wordsworth couldn't they they'd handle that but the notion is that you need to say in the context of this assignment I am taking literacy to mean that and I'm aware that there are alternative perceptions John, can I ask you about literature do you know if your research is into something and it might be very much British literature on the top so it might be North America and it might be 1980s 1980s and 90s do you know we're talking about sort of a liberty city you could do literature if you could I don't know about Europe they might be very much British research within the last decade so when I'm really asking this how much emphasis do you give to say Australian and North American research which might be 20 or 30 years old if it's that old it's probably passed itself by day because the context has changed even in the 1980s I would worry because I mean there's so much now thank you for raising the point because it does move us on nicely and I'll build on it now if I may but basically my view is that you take whatever evidence you can find in terms of literature and therefore you do the searching and that depending upon your focus you begin to track down appropriate sources and you've got the library here as a resource and you do it in two ways that you look for the big names and then the other is you go into the journals now then there's no rule of thumb on this because you're quite right some topics will have very very limited amounts written about them so for example if you decide to look at teams in schools it's overwhelming massive amount of stuff but if you look at middle leaders as performance managers virtually nothing and so therefore this again part of the pragmatic approach saying it has my topic got legs well the answer is how I got access to appropriate research one of the best ways of starting your research in terms of mapping out what's available is to go onto the national college website because that is the best free resource in the history of the world ever and you are all signed up members of the national college aren't you good I'm not going to ask for a hands up on this one but can I just say to you it's already lost its autonomy it's now a department of the department and it's very fragile at the moment and if you don't belong to it then it will the government will eventually say well nobody's using it the more use you make of it and it's the most fabulous free resource it's the envy of just about every other system in the world and if you use their search procedures you will find an amazing amount of resource on most topics that are likely to crop up in terms of your dissertations would you say if something's very rich in research is that slightly off because so many people don't research on the topic no it means that you're dealing with something that's very important don't it so it's silver but if you're on electricity it's a good response absolutely but we still haven't cracked it have we it's still a problem but there again you see no piece of research can ever be truly original that's impossible in this day and age but doing the research in your context and making it real within your situation that's the great thing that's the thing that makes the difference it's new knowledge for your school that's perfect thank you and therefore if your school is having an issue and struggling with the notion of particularly for example boys literacy then if there hasn't been a systematic study of what research might be helpful to our school then you are doing something very powerful and valuable so you need to have this conceptual framework I'll come back to that in a second you need to be very clear about your methodology and you need to be clear about your methods more on that in just a second the key conceptual frames are things like for example the notion of arguing that the reason why we should close the gap the reason why we should get rid of this tale of underachievement is to do with equity in education and there is a moral imperative that how can you be an educator if you are not personally morally committed to the notion that every child is entitled to exactly the same educational opportunities but it's quite an important frame isn't it because if you believe in equity then you have to say how does that figure with our work with children with special needs for example are we really making sure that we are guaranteeing their entitlement in the curriculum because that's a challenging one I may have used this example on a previous occasion but a year and a half ago the parents of Down syndrome children their association looked at schools that said on their website we are inclusive and they took along their Down syndrome teenage children and said you say you're an inclusive school here's our child and we would love her to be educated at your school and in 90% of the cases the school said we're not quite that inclusive that's scandalous isn't it they should actually not be allowed to say we're an inclusive school they'll say we are an inclusive school up to the point where it becomes inconvenient for us to accommodate your child something like that we're an inclusive school apart from children with the following behavioural problems because that's the reality isn't it but if you believe that equity is the basis of justice in a society if you believe that your role as a leader is to secure the entitlement of every child to effective learning then shouldn't that be underpinning your research in a 30 fundamental way how do we secure effective learning for every child I will put my hand up now and say when I was timetabling a secondary school of 105 staff not all of those staff were equally effective and I engaged in a strategy known as defensive timetabling which fortunately doesn't happen anymore I believe is that true whereby teachers are placed with classes with whom they will do the least damage and you may have heard about this you may have heard of situations for example the shocking thing that happened years ago when by the most effective teachers went with the most able children fortunately that doesn't happen anymore either does it no of course not good good good good sorry to waste your time on those issues there's an issue around what you believe to be the moral basis of what you do there's an issue around what you understand by effective pedagogy there's an issue around how change occurs in schools there's no right answers on any of those but in your presentation you need to be fairly robust and confident that you have thought through your take on that and then we move on to the methodological issues in other words I've identified what I want to find out now the big question is how am I going to find it out and that takes us back to the barrister that takes us back to the law courts and I suppose the first issue I would raise with you on this is the notion of be very circumspect in interviews questionnaires and so on the amount of data that can be in the amount but basically a human being talking at the rate I'm talking at the moment I'm probably talking at the rate of two to three words a second and if you were to take me for half an hour yeah the most enormous amount of stuff and how are you going to manage that life is too short to sit and listening to me for half an hour you can't transcribe it either can you you haven't got time or skill to do that haven't got the resource to do that and therefore you have to be very very clear and say I'm going to use semi-structured interviews which are going to be very short very focused and blah blah blah John, so it's interesting that just a weekend I was cooking this very very helpful employee of Apple directs to me towards an app called Dragon Diction really great so voice record your interviews and it will transcribe it for you so it's Dragon Diction it's free as an app thank you that's great but again you can have an overwhelming amount of data that's my anxiety so be modest in how big your questionnaires are be cautious in how many people you interview make it appropriate to you managing it with a limited time frame with limited resources but the crucial thing is what sort of methodological approach are you going to adopt in other words and I suppose the key phrase here is form follows function if you are investigating teacher attitudes then you may want to get their own personal language in place and therefore your methodology will move towards the qualitative if you're doing a survey of a different type of issue then it may be that you move more into the quantitative but you need to be very clear that going into the qualitative and alternatively into the quantitative has implications in terms of the way in which you present the knowledge you're creating and you need to talk about that and that's the sort of issue on page 2 which I'm not going to go into detail now but basically in various continuum you need to say am I going to be postivistic in other words work on a scientific frame or am I going to be working on a phenomenological model whereby it's essentially the difference between objective and subjective and that sort of debate will determine the validity the reliability and the trustworthiness you need to tell us what sort of strategy you're using in order to justify the approach and you have very proved it at the bottom there on the extreme left you have science at its very best which is based on experimentation which is herbal blind and so on and absolutely rigorous and systematic and then you go through the social sciences which aspires to be objective but there's a big debate about that then you're into the politics and then you're into personal narratives and then you're into personal beliefs and you need to know where you're situating your research on that continuum because that's going to make all the difference to the credibility acknowledging the fact that sometimes your evidence may be challenged, may be questioned and then having identified what's the most appropriate methodological approach you then need to say what methods are most appropriate and I suppose that of all the thousands of master's dissertations in education that are done over the years everybody comes down to the questionnaire with a few interviews and again be open for example one of the areas that I find fascinating is the way that people engage with personal change in a very demanding and challenging environment and therefore I've done some work in the past around the use of diaries just keeping a very simple log of how do I feel about the work today and when people do that over time you build up an incredibly powerful resource but it's way out on the qualitative side of things isn't it but actually it's an authentic voice and there's a wonderful amount of work done in terms of research within a feminist context of recognising the authenticity of people's voices and some very interesting work around student voice as well and so don't just write it off because it's subjective because it may be it gives you the authentic engagement with what people are really feeling but that depends upon what it is you're trying to find out doesn't it it's for purpose what do I want to find out what's the best way of finding out I will get you away on time I promise very briefly the ethical issues essentially it's all to do with your your sample isn't it and again as we mentioned earlier it's the informed consent people know that you're researching them and I think it's it's very very unprofessional and it has happened whereby people send out surveys and they say it's a school survey and in fact it's their personal research that's not acceptable is it and that's the sort of issue that we need to be very careful about on the issue of the ethical form that we filled in one university I worked in the first question on the form was does this research involve experimentation on animals and one of my students wrote no it's okay to hear nine students you need to be very punctilious indeed in terms of making sure that you meet the criteria because the current thinking around researching a practitioner-led research based in the workplace is that you've got to be absolutely crystal clear in terms of the clarity number ten I think we've mentioned number eleven is interesting isn't it because you've really got to say who's my audience, what's this for how does it look and if I may say so with a degree of pain from over the years is it readable you know and is it structured is it logical crucially is it consistently presented and then finally and again I don't know what you'll how this works here but I would expect in any piece of research in the conclusion you to say I have changed I have learnt, I have grown I have made a difference because of this research and you need to move into some kind of meta reflection on the process of being a researcher and how it's informed your own professional practice yeah I'll just take a pause because I've scrambled through that we've got about just over five minutes left just very quickly please with your neighbour things which are positive things which are challenging things which are worrying you still and then we'll have five minutes quick buzz on issues emerging and then we'll call it a day so thank you very much for your stamina hopefully that's given you some framework and are the gaps are the things which are not I mean that was very fast overview I appreciate but is the only thing that really is bugging you came here this evening to have sorted out and it's not been sorted out it's not a burning issue mate it's something that you're thinking about it's a biasing process of research how addressing that because obviously we're all from our own point of views yes would you be able to I think that goes back to paradigms and triangulation basically and that's the theme of the evening I want you all to sing that as you go out but I was going to say because we're only trialling at all at this stage we're not really absolutely and therefore you've put on very very careful and very precise caveats and I think that I think the mistake that people make is not to say this is very early stuff that we are really cautious about this data and we are anxious about transferability it's a putability and so on and as long as you are very robust in saying we have got to put very clear boundaries on this this works in this context but it works because of these variables and one of the best ways of handing a lot of these issues is to say let's identify the variables that are functioning here and say it's working because of this particular constellation of variables but if you take one out then it might well be compromised and then you begin to analyse and explain rather than simply describe and that's good research that's really good writing thank you a general looking around the room saying who's going to dare to speak at this point of the evening ok all done