 Professor Colin Latcham is an Australian researcher and consultant who's been with us at the Commonwealth of Learning in Vancouver, Canada for the last one month, working on a very interesting project. Many of you will know that the Commonwealth of Learning, or COL, has developed several quality assurance toolkits in open-end distance learning. These are specifically for open schooling for teacher education and higher education, but we didn't have anything for non-formal education and there's a lot of work that we do in that area at the community level. So Professor Colin Latcham, or Colin as I'll call him, very kindly agreed to take this on and he's been here working on that. So we'll talk about that today so that you know what it is that he exactly did. So my first question, Colin, is that why a quality assurance toolkit for non-formal education? Well, I became very interested in this. I'm very interested in quality assurance and just co-authored a book on this subject and we found that non-formal education was an area that really was not being very well covered in the literature. I think the last work was by Tony Dodds about 1996 in this area, so that's quite some time, and I'd seen a number of reports on non-formal education and I was quite concerned because it seemed that very often the quality was not assured in these projects, which of course is a matter of concern because it's not in itself a very high status area and to try to get funding can be quite difficult. There are quite a lot of pilot projects rather than long-term sustained programs. So it's very important I think to assure all of the stakeholders that the money is being well spent and the outcomes that are required for non-formal education are being realized. So what would you say is the difference in the quality assurance for non-formal education and quality assurance you know for in an institutional context? Well, I think this is what's made the project particularly interesting for me because it's really turned into quite a major research study. You know basically when you're looking at assuring quality in a university's situation, which is basically in my background, I mean you're really looking to see whether the graduates have achieved the kind of outcomes that that are expected both in terms of the disciplines they studied and the kind of generic attributes that employers and society at large are requiring. Now when you come to non-formal education, these outcomes may not only be concerned with individuals but with whole communities and so because I was adopting an outcomes approach or rather to use the development terminology, the outputs, outcomes and impacts approach, what I had to do was to go and research what was actually happening in non-formal education. I found a really fascinating range of programs going on from basic literacy right through to encouraging peace and reconciliation in war-torn communities, helping people with work skills or to set up small to medium enterprises through to the use of organizations like telecentres. So I had to work through all of these and look at the evaluations to work out what kind of outcomes were we actually talking about and then having drawn those up I then translated these into what we would either call key performance indicators or critical success factors, very simple statements that can be based upon direct observation of documents or the learners or the community's performance during and after the event. So are these generic enough because non-formal education would cover so many different contexts? Well there's quite an extensive list of suggested performance indicators and I'm not suggesting that everybody should work with all of them. They're really indicating the kind of performance indicators that one would be looking for and I've broken them down into three areas. First of all there is the the outputs or the outcomes or the impacts on the learners. Then there's the what effect is how would you measure the quality of the actual learning provision you know the actual program and then finally how would you judge the outcomes from the community level and the outputs are short-term, the outcomes are mid-term and the impacts are long-term. So I'm really advocating a pick-and-mix approach and you even add your own and then what I've done is provide a very simple matrix within which you can take note and score how these performance indicators have actually been achieved because a lot of the people this is aimed for are not full-time educators. They won't have large budgets, they will have very limited time so my aim has been to give them a very simple toolkit that they can use in the field and yet give the necessary results. Because I imagine some of this will have to be translated into local languages because we work with grassroots communities. But I think involving the partners as you seem to say is very important in this whole enterprise. Who are the people who will be using this toolkit? Well when you look at some of these quality assurance reports and evaluations and audits one can see quite clearly that things went wrong right from the very beginning. So the first thing I'd say is that the quality assurance must be built into the very initial planning of the project and then it should be monitored throughout the project and then summatively applied at the end of the project. Now what this means then is that the toolkit is really for policy makers and planners but equally important it's for the actual practitioners. So they must be trained in and familiarized with the quality order system because they will be the people who are doing some of the monitoring and providing some of the reports and without that you have a really weak link in the chain. So what are some of the other ways that the planners and the practitioners are going to use this toolkit? Right from the beginning as you say that it's a good research instrument, it's a good resource because it has case studies of what's happening in different projects so it might trigger off good ideas in people so it becomes a good useful planning. Yes well I think it gives them a very very clear step-by-step guide to the steps to go through. I mean for example I've looked at the types of data that you can gather up in the quality assurance process and the critical questions you need to ask yourself about the reliability and the effectiveness of gathering up data in those particular ways. I think the other thing that probably going back to your earlier question about how this differs from from higher education, higher education institutions and schools on the whole tend to operate on their own. Yes okay you know the parents and the employers and the communities have a role but when it comes to non-formal education very often there are all kinds of partnerships for example with the lifelong learning for farmers critically important to have a partnership with the local banks who are going to provide the loans to the farmers to implement what they learned on the training and also with the mobile phone companies because the learners had said that this was the best way they could learn so they had to be cheap to use mobile phones. So in other words part of the planning actually involves looking at the partners and of course the more partners you get in the more important is to make sure that the quality assurance system involves all of those partners as well as the sponsor or the provider. We're really looking for models that are shown to be working that are then not only sustainable but transferable so that other people can learn from successful projects and sadly at the moment that research those findings those quality assurances just aren't there. I think the data is very important you know if we have to sort of convince other people that this model works they want evidence first. They must do they must do I mean money is very constrained and you know we're competing against you know all kinds of other demands for for valuable resources and yet when you look at non-formal education as I see it when we look globally throughout the whole Commonwealth you know higher education schools is really at the tip of Niceburg and below that we have a mass of people who need to be helped to become literate to be more aware of how to conduct health care who need to be helped to build their communities to be self-sustaining and and so on and my feeling is that you know we really need to focus much more on that but if we're going to do it we must do it really well. I think that's very important especially getting the partners on the same page yes absolutely because ultimately it's the partners who generate the data in the field yes and they are the ones who should be convinced because they are the ones who deliver and it's through them that you know organizations like GOL and other development agencies work so I think this toolkit is meant primarily for the partners. Well I think it's for the I think it's for the proposals of projects you know if if somebody is applying for funding I think they should be capable of signing up to an agreement this is how we're going to do it and these are the results we're going to we're going to provide then I think the actual partners yes as you say must be all on the same page and then the communities themselves need to know because a lot of these non-formal education programs I mean take telecentres they depend enormously upon community support and community input I mean I suppose this is another big difference going back to your early question between non-formal education and formal education this is really starting with learners needs or community needs you don't start with the set curriculum or an agreement on what it is that's going to be so there has to be a great deal of involvement of the community in actually planning the programs and as we saw with lifelong learning for farmers which I thought was wonderful it's now reached a stage where the farmers themselves are running their own training website and producing their own training videos for use on mobile phones well that's where you've got evidence that something is really taking off that's an impact right that that's that really is an impact you know I mean 2000 women have generated over 2.5 million canadian in the last four years so that is a very major impact yes and we hope that we'll be able to run similar projects much better more effectively with more value for money by using this toolkit so thank you very much Colin well thank you very much it's been a wonderful time I mean it's a very valuable piece of work so thank you thank you thanks a lot