 Thank you. It's better not to clap yet because you haven't heard what I've got to say. What I want to do is just spend about 25 minutes talking about what it is that makes projects go well and what it is that makes projects go pretty badly. Most pilots don't go anywhere. They're mostly a waste of money. The country doesn't really scale up the things that we do as pilots terribly well. And I'd like to try and take some of the lessons we've learned from industry and apply them here, hoping that we'll have some more pilot projects that will do well for their audiences. Just a note of caution that the images in this are not copyright cleared for anything other than looking at here. Best to think, first of all, about the barriers. I'm going to give you my top 10 things to remember when you're running pilots. Understand what the barriers are for your audiences. Now, the end audiences are likely to be teachers or students, and the barriers for those people are likely to be some combination of cost, relevance and ease of use. That's well known, and in terms of digital inclusion for the country, those are still the main barriers, with the main one now being relevance for the people who are excluded. But for the people who are going to produce this pilot, there's a whole lot of other barriers. So you've got some pilot projects, some great scheme that you want to devise. The people who are going to do it, are they actually talented enough to do this thing? Because quite a lot of times you meet people who are just not quite up to it, and it's worth taking a step back and thinking, this is just never going to work. Do they have experience of some kinds? They might be very talented, but have they got some experience to be able to bring to bear? Do they have the resources? And most of all, are they a coherent group, or can they be made into a coherent group? Because you can have lots of talented people who are all pulling in different directions doing their own thing. So generally, these are the reasons why things fail, and lack of talent is the one that nobody really likes to talk about. Next thing to remember, number nine on the list of priorities, is who is it actually that this pilot is trying to impress? It may be some combination of a teacher, students, head teacher, maybe a local authority, maybe the parents, maybe a broadcaster or the PR people who are going to sort of tell the public about this wonderful thing. It might be the EU or it might be sort of politicos. But generally, people don't face up to who it is they're actually trying to impress. And when academic institutions are involved in a pilot, the people they're mostly trying to impress, in my view, are their academic peers. And what you need in an academic paper and what you need to impress your academic peers is probably not what you need in order to scale up a pilot service for the general public. So things which involve universities are very difficult in terms of industrialising the product. I can see I'm going to be popular here. So number eight on the list of things to worry about are there are certain things when you're doing a pilot, an educational pilot, that you need to integrate well. By integrating it means you have to sort of bring everyone together at the same time as if you were making a cake right from the start. You don't sort of think at the end of it, oh I might need some flour. You get it all together at the beginning. And these are the things and each person in the audience here will have some sort of knowledge of these areas and probably more than one. But they need to be brought together. And one of the things that stops projects like this doing really well quite often is that there's a confusion between what is project management and what is editorial leadership. This happens a lot in government projects that a project manager is put in place to ensure that the project runs as it should and according to specification. But there often isn't a person that says do you know that's running just right but it's just a little bit crap isn't it and why don't we do something else. You need to know the difference between is your project a very well defined, very clear, a ferris wheel sitting on the south bank of the Thames which takes people on a tour around it or is it a millennium dome which as Siwak said very nice but what is it for. Now these projects both largely went to time and budget. Some of the contents of the dome didn't but the actual building itself did. It did all the things that were in the specification and yet one was a little bit crap and the other is really good. Now in broadcasting the industry that I've come from there is an executive producer who on behalf of the audience says do you know that's great this is fantastic that's a bit crap let's do more of this right. Who is it in the pilot who's going to take that role and if you don't have a person who can take that role God forbid it's a committee then the thing will turn out a bit crap. You have to make sure that everyone actually understands what the pilot is and what it's for and that means particularly the technologists and designers who are going to be building the thing. Don't wait till the end to describe to everyone what this thing is. Have what's called a sunrise meeting where everyone gets to not only understand the pilot but the background to it as much knowledge as possible should be imparted so that people can really add to it and add their intelligence. Seem so obvious doesn't it? That rarely happens. So you need to understand where your pilot fits in. Now if you think of the administration side of where pilots might happen in schools making schools run better or educational institutions that's one thing and there'll be a whole list of attributes that a pilot needs to have. If you're thinking about a project that might have something to do with the educational chain and how people learn then if you think of that chain and I'm not a sort of theoretician I'll just sort of look at it in the way that broadcasters or advertisers would hear that at one end of the chain you want to stimulate interest and at the other end of the chain you want people to create or do or have a change in their brains in some way and in between you need to go through some steps which are to engage people, to guide people to resources and brains to communicate. That's part of the process of learning. So you need to think about where it is on this chain or if you have a different chain that you use because of your theoretical backgrounds where is it on the chain that your pilot sits what is it that you're enabling people to do what's the thing that you're underpinning. Now if you use new technology cleverly you might be able to enable those stages better you might be able to lower the barriers between stages so instead of someone having to go physically from A to B in order to do something you might be able to do just here when they want to. You might be able to help to create virtuous circles and by virtuous circles let me explain there was a project we did for as part of a programme in government called Culture Online where it was designed for people who are users of the mental health services and in their own terminology mad people this is a project with sane and mind and everyone else and we did a project called Mad for Arts this is like the word queer it's being retaken by the people in that group. Now the sort of idea of the project was to enable people who class themselves as being mad to give comments on art, architecture and music to become critics of art, architecture and music and that would be of interest both to so-called mad people and all the rest of us who are also so-called mad at least some of the time and that by doing that they would be able to A have a voice, B learn new techniques and also give something to everyone else that might be of interest. Now the galleries and museums around the country opened their doors to people and enabled people to contribute to a big website the best material from the website appeared on the community channel and the best material from that was transmitted before and after the main evening news on channel 5 with a strong call to action for more people to take part Now the important point there was that that virtuous circle of content you give people the tools they use the tools that enables more people to see what it is that they've done and that itself encourages more people to take part means that broadcasters and media companies aren't necessarily in the role of PR anymore they're actually integrated into the project and that's very important so think about one of the things people don't think about enough is where the virtuous circle of content is going to come How is it that users are going to encourage other users to take part and that's the classic way of doing that I almost lost it there because I just saw it as an aside I came up on that sort of weird pendolino train that makes you feel sick as it turns around corners and just about the second bullet point there I started thinking I was going to collapse It's always better to tell people after the event than before One of the things you need to think about is whether your project is going to help people to learn or grow or develop and how it is that each person can add value for others Choosing good measures of success is top thing to do If you were measuring the success of a football team we're going to measure the number of players we're going to measure the cost per goal we're going to measure the time taken to change shirts in between halves These are stupid things but all around the world of pilot projects for education you see really stupid measures When I was at the BBC for a while I was in charge of BBC online and my salary I managed to get linked ever so slightly to the growth in internet traffic of the BBC over the growth of the internet as a whole So if I could make the BBC site grow faster than the internet as a whole in terms of traffic I would get some sort of small bonus So the first thing I did because the traffic was measured in page views I made all the pages smaller So the audience had to click around all over the place Boy did I do well out of that for about a week So the lesson here is really think about what the if you're going to have targets and that's the kind of world we live in pick the right ones for goodness sake Page views are not a good target generally because they're not necessarily linked to what people are learning that you want to impart then learning is somehow what you need to measure or at least attitudes to it So number four, partnerships When you're entering into a partnership as many pilots are as many pilots are partnerships you need to think about what each partner says they want So if you do a partnership with the BBC they will say I think we agreed this wasn't being recorded they will say that they want public service, doing good all the things you know reach and audience share and all that kind of thing And if you want to do a partnership with a big museum or gallery they'll say roughly the same thing What they actually want is to do all those things but they'll also want some combination of political influence, power, limelight or money Now if you're the BBC what you most want in the world is credit for what you do with the public strummed into you if you work at the BBC they strap you into a chair, shine lights in your eyes and every day they tell you all credit comes back to the BBC Now if you're an organisation trying to do a deal with the BBC that's a useful thing for you to know If you're an organisation doing a deal with an academic department of the university there's some combination of these things that the university will want largely also credit and limelight though probably credit and limelight and a different sphere from the BBC so that might be still a good partnership but it does tell you something when you start thinking about what people actually want how it is that you're going to drive success and most people want personal aggrandizement of some kind so if you can organise that personal aggrandizement up front and it might not even cost anything that's a good way of making pilots work Now everything you can about your audiences just a quick story about General Motors actually you might not think that General Motors has much to teach this audience If you're designing cars there are some things that you know that people want the public want and you meet those needs there'll be things that you can't meet their needs but you know that they want all the same and General Motors found itself in the position of wanting to differentiate itself in the marketplace so they knew that they could ask people the public would say well we want different speed of the car and we want different colours they knew that at the time they wanted cup holders and CD players and faster speed this is a while ago and at the time weren't meeting those needs but they knew they could and then there were things that the audience doesn't know about but the car manufacturer has put in anyway like crumpled zones and stuff which most people don't care about but what do you do about the things on the top right there where you're not meeting people's needs and if you go and ask them what their needs are they don't know and that's a lot of new services fall into that category what GM did was they sent researchers to to live in people's cars for about three weeks all over the United States so you come out your door in the morning someone in your car in the back and they just observed for three weeks and they all went back to base after three weeks and they compared their notes and one of the things they discovered was that people were forever ordering flowers from in and around the car interesting thing you know that they were worried about breaking down in a bad area that they were worried about losing the keys to their car locking themselves out of it and they were worried about not being able to find their car in a car park which is probably the size of Europe so those are the sorts of things that people were worried about and on the basis of that GM developed Onstar which solves all those problems with one bound they were free and that was a very very successful service for GM and differentiated them in the market all based on observation at the BBC when I was running educational campaigns we had a campaign for basic literacy and it was aimed at getting people who couldn't read or write to go and do courses fantastically economically useful to the public all the agencies said the big thing that we should go on would be embarrassment embarrassment of not being able to read to your kids and so on and embarrassment of having to go into shops and sort of say I've left my glasses behind could you read this for me that seemed kind of obvious to us but we could all read and write and when we went as the advertising agency suggested and lived for two or three days with families where people were functionally illiterate we all came back and compared notes and we found something quite different which actually it wasn't to do with embarrassment it wasn't to do with inconvenience it was to do with guilt and we did all the things that advertisers would do and put spots all across the schedule which are absolutely focused tearjerkingly, emotionally, Hollywood style guilt, guilt, guilt, guilt, guilt to the point that people were writing into complain not the target group they didn't complain 450,000 people responded to it now what I don't know is how many people would have responded if we'd have done the usual thing but I know that it was effective so don't take people's word for what the audience is like go and see them yourselves a word on new formats and services the word is storyboard what I mean by that is I don't expect you to be able to read the writing on this in fact I sort of prefer you didn't but the if you sketch out not using a computer program of any kind but just using pencil and paper a storyboard, a cartoon for how you think your new service will be used so in this case we've got a couple of people watching telly and then a teacher shows something in a classroom and then this happens they get some gear from somewhere and then some stars turn up there's a story here a lay person should be able to look at the story with someone explaining it to them saying this happens and then this happens and be able to understand it well enough to be able to say do you know, I don't think they do that or what if it's raining there or where do they get that from right it's a way of having people cluster around something and be able to comment on it without it being all sort of finely produced like a computer program would be just sketches doesn't cost really very much at all and if I was a consultant or a cleverer consultant I would have some sort of very expensive tool here but all you need is pencil and paper and a group of people who can cluster around and say would people do that at this time so whatever pilot you do just storyboard it now finally number one is something missing from this you've got the content, you've got the technology you've got the user experience these are the things that we normally pilot okay and the reason that the pilots tend not to go anywhere is because they tend to test the wrong things and the things that are not tested sufficiently are the business processes right, I can see you're all slipping into a coma so let me just explain what I mean you know, a pilot is fine and for academic research it's brilliant because all academic research is about kind of doing something new publishing it, getting your name on it but actually what the industry and we are in an industry needs are things that are scalable sustainable, replicable all over the country, scale it up manage to fund themselves these things are rarely tested right and maybe we should do every pilot should have an academic partner of the London Business School or LSE or Manchester Business School or something right because these are the things that cause the pilots not to go anywhere and since most of the time you just know that the pilots aren't going to go anywhere then sometimes I wonder what's the point of doing in the first place right, but these are the things that you really need to think about now commercial companies spend very little time on the invention of new things and loads of time on these right, public service does it all the other way around and government in particular when a new minister comes in wants to be associated with something new not very exciting if it's someone else's idea not many votes in it and they don't want to spend the time and effort looking at these things so if you really want your pilot to go somewhere and you want to make a mark on society that's the thing to think about so just I'll finish with just a few sort of questions to ask a cultural resonance that some of you might spot here why is this project different from all other projects is the thing that you need to to ask yourself that's the thing and of course how many questions will there be there will be four questions so will more people learn better that's one of the things that we're in business for will this save money will teachers or students be happier and can this actually be done better without new technology it's worth asking and the reason I have these this is my litmus test but the reason I put those pictures up is because I went to a school recently where they were very proud of an online system to allow people to do litmus tests virtually I just felt like slapping the teacher that's probably not allowed is it so my final plea is please is whoever you think they are fiddle with things because it's by fiddling with things by actually taking them up and playing with them that you learn what's possible when I became a television director for a while it was only as a result of someone when I was an engineer putting a camera in my hands and saying go out for half a day and make a little film and it's at that moment that I thought oh I hadn't thought about what might be possible here so you need to sort of be familiar enough with the new technology whatever it is just to be able to see what's possible that doesn't mean you have to be proficient and just be curious thank you what chimed with me was the communication part of the process I tend to spend a lot of my time actually trying to convince people about a project and its aims and these sorts of things without being commissioned if you see what I mean because I think where the BBC process is probably kind of like you're looking for new ideas whereas perhaps in our position the academic kind of side of things are not necessarily looking for new ideas absolutely I mean the BBC is looking for new ideas I spent three and a half years in the GCMS which I think they were definitely not looking for new ideas obviously sometimes you just wish you could claw back those words but the and there are plenty of organisations that sort of want new ideas in certain sorts of new ideas but not these ones and the trick is to first of all any group of people will need different ways of coming at things you can't have a one size fits all communication strategy about what it is that you want to do you can't persuade everyone in the same way we know that from our family life we want to take a trip to Paris and different people need a little bit of kind of cajoling in different ways this will be the advantage for you and eventually you get agreement so that would be one thing second thing is think back to those drivers in a partnership and even if it's a partnership within an organisation there are people who will have different reasons for wanting to do things or not wanting to do things so if you can get to those it won't be what they say they want because what they say they want people are clever they'll say they want motherhood apple pie better cudos for the university whatever but what they actually want may be some of those things plus a whole load of other things so you need to really think about what it is that's going to drive people I don't know whether I've answered your question I know the questions it was great for me the the area you didn't cover was conflicting requirements and within projects sort of the wildness of the art of the possible being turned into the practical delivery on time and to budget I'd just be interested in your comments about sort of change control and environmental influences and how you tame them yeah you see there are two answers to this and the professional answer is the one that's all about all that change control stuff and Prince II and processes for changing things and so on and that's all good and there are manuals on it and it's fine and yet even with all those manuals it kind of all goes a bit pear shaped sometimes right and what you need is talented leadership right and if you haven't got talented leadership do a different project find a different project to do it really is and that is one of the reasons why people sometimes don't want to do things just to go back to your point I like to think of myself as a bright-eyed bushy tail person who's keen on things but sometimes someone comes along with a project and you think that's a fantastic idea that's really good it would really help the public it would really help my career I'm up for this look at the shit who's leading it it's just going to be awful isn't it and I'd rather put my time into something else and forget about that because life's too short I mean life is like that unfortunately and you know if you're talented enough then go to another organisation if you're mobile enough I was sort of forced to do that because I only had short contracts around the place so I had to go but you know sometimes you just take a step back and you just think this ain't going to work and you know who might be your greatest friend the chief financial officer you know they're the people she or he has to look after the money for the organisation and if you say look I'm really not sure this is going to be good value then they'll probably take notice I don't know it's a sort of bit in public service organisations in universities and governments and government departments there are layers of politics that just sort of gets sort of slapped on everything which the politics is there in commercial organisations it's just sort of more transparent and I don't know you just sort of there's a sort of pragmatism isn't it you just have to sort of take a step back and think that ain't going to work there was a lady here Jackie one last question Thank you, yeah my question is about the I the impact word and you talked about success factors I think there's a lot being driven by research councils and other bodies at the moment in terms of how do we measure impact I wanted to ask you not about the quantitative stuff because we can do that but about the qualitative stuff do you have experience of how you can sort of get that qualitative beneficial impact So right the sort of two questions within that one is if you could have a checklist of things that would be it make for compelling experiences that people would want to tell their friends about and so on what would be on that checklist and there is a checklist which I don't have time for unfortunately but there's six factors and you can go through them and first is it well defined is it accessible is it transformative significant there are these things that whether you're a game show designer theme park designer or anything else you should know about and email me and I'll help you the other factor is how do you know that the thing you've done has been impactful and one of the things that you can do is ask people you can say has this changed your life how much has it changed your life has it changed your life and better or worse and you need to frame the questions in such a way that it's easy and more easy and possible for them to tell you negative things than positive things so if you're in a bank or a restaurant and you're giving service to people and you you say how was everything for you when you're checking out of a hotel or what have you the purpose of that is not to find out what it was like that is no way to ask someone they don't just want to get the hell out and they don't want to say bad things to you and they'll just say fine we've all been there if you wanted to really find out you would say can you tell me the worst two things about this evening can you tell me what was the if there was one thing about this restaurant meal that you didn't like tell me what it is and that way you get a different kind of feedback so it's partly your question is so deep and happily or very unhappily for the rest of the audience talk for an hour about it but there's a whole lot of things that you can do just email we have to end there because we've sort of overshuttered times only slightly thanks very much for releasing to me ladies and gentlemen thank you