 Our talk today is by two wonderful researchers. Dr. Michelle Milade is professor of communication studies in the Wilkinson College of Humanities and Social Sciences at Tappan University. Before she moved to Southern California, although she was previously a U.S.C. graduate, she served as the founding director of the Penn State Qualitative Research Interest Group, which is an interdisciplinary community of researchers involved in supporting qualitative inquiry. Our research addresses human communication and health, including areas such as substance use prevention, suicide, and family mental health. Our community-based participatory research has involved numerous creative projects to translate research findings into social change. For the past 20 years, she has served as the principal qualitative methodologies for a National Institute on Drug Abuse Minor Research. This work has developed one of the most successful evidence-based substance use prevention programs in the United States and reaches youth in 43 countries worldwide. She has lectured, conducted research, and served as a consultant at universities across the U.S. and internationally. Please join me to welcome Dr. Michelle Milade to our campus. She will be joined by Dr. Michael Heck, who is a distinguished professor of communication arts and sciences and crime, law, and justice at Penn State University. The National Institutes of Health has funded Dr. Heck's long-term research on drug-resistant strategies continuously since 1989. That is a remarkable feat for anyone who is ready to reinvigorate in this day. He was among the four to study the social processes of adolescent drug owners, including an examination of the role of ethnicity and co-op operation in these processes. The project has produced a number of theoretical advances, as well as the development and efficacious multicultural school-based interventions for middle school communities that, since it's adopted by TARE, you all know that TARE America believed to be the most widely disseminated middle school drug prevention program in the world. A new elementary version of the curriculum based on social emotional life is being implemented worldwide as we speak. His book, Adolescent Relationships and Drug Use, summarizes some of this work. I'm pretty sure I have bookstore currency, and the video test produced for the project had won numerous international and national awards, including a series of 2000 regional Emmy awards for student productions. Dr. Nex has received numerous awards, including the National Communication Association of Darrell Power-Phillips Award for Distinguished Applying Communication Scholarship and the Society for Who's Mentioned Researches Mentoring Award. Please turn me to Dr. Nex to how it happens. They will both talk to us today about why it's supposed to work and why it's not supposed to work. Or not. Yes, baby. Thank you so very much for that lovely introduction. I'm actually going to start with a story, may seem off-topic here, but my mom is a wonderful cook. She spent a couple of years trying to perfect the cheesecake recipe. The cheesecake recipe. And she had all the ingredients, all the measurements, the timer was supposed to be in the oven, everything. So I, when I was first learning how to cook, I'm like, okay, I can do this. For a family that's going to make her cheesecake. But I didn't have sugar. I had sweet and low. So, did it make no sweet and low? How it's concentrated? It's like a ratio of one to three. Okay, so you need one sweet and low for like three tablespoons or two tablespoons of sugar. Well, I had as much as a sweet and low. So what did I do? Three cups of sugar. I poured them all in. What do you think happened? Yeah, exactly. It was totally inedible. I thought I was being created. I thought I was being created. There was this, this cheesecake recipe that was, was developed down to a science. But then I wanted to do it. I changed an ingredient for convenience sake. And it totally didn't work. Should have worked in theory. It was a recipe. It won't do that. I thought it was sugar or something low with sugar. But it's not. Okay, so I'm going to go back here. And that's a way of introducing here the best laid plans of mice and men. We develop health interventions. We develop them with care. We spend lots of time developing them. And then they go out into the field. And somebody put sweet and low instead of sugar that we've developed there. Now somebody put chocolate sauce on the cheesecake and it could be better than what we planned. So what we're going to talk about today are ways that we adapt in the field. We develop health interventions. We put them into the field. They get adapted. Historically in the field, they say we should have fidelity, right? We should have fidelity. And do our interventions the way that they're planned without any adaptions. Adaptions, our adaptations are bad. So we're going to challenge that a little bit and say, you know, adaptations are. They're going to be. So let's live with it. Let's roll with it. There's my little cheesecake. Although prevention programs are developed carefully in implementers such as teachers. In our case, we have school-based interventions. Our trained to implement programs of fidelity. Real world events sometimes conspire to influence program delivery and ways program developers do not imagine. Again, we cannot anticipate all the way that people implementing, whether they're medical assistants, medical educators, teachers, or in the dare, in the case of dare, officers going to implement our programs. So what we're going to talk about today is creating, adapting, and disseminating effective health messages in particular interventions. We're going to talk about developing messages, adapting evidence-based messages, intentionally and studying unintentional adaptations, and then a little bit about dissemination here. So, you know, you ask yourself, do we still have to worry about drug use? The war on drugs in my era, we don't hear that rhetoric so much about the war on drugs anymore, but it is still clearly a problem. Again, reducing substance use is one of the objectives of Health and People 2020. It's still an issue. And how many of you know someone who has had a problem with drug addiction or substance use problems here? It's still, we're talking across cultures, urban, rural, we're going to talk a little bit about urban and rural kind of things, across ethnic groupings. So it's still a problem. We still need to build health interventions for this. And I'm going to give it over to Michael now to talk a little bit about developing our program. We're going to try to confuse you by switching back and forth. The program we created was called Keeping Real. It's a school-based multicultural intervention that grew out of Dr. Miller Day's master's work, actually. So those of you who are students here, long-term, big, great things can grow out of the work that you do right now. He was my advisor. He's not old. So for over 25 years, we've been trying to do two things. One is understand the social processes by which drugs are offered. Some of you know the, just say no movement that emerged in the 1980s. Nancy Reagan was actually pushing this idea. It was based on real science which was that we needed to understand what was going on when kids were offered drugs. And what did they say? What went on? But it turned out that not many people were studying what went on in this situation. And that was the basis for the intervention that we created called Keeping It Real, which proved effective in reducing alcohol tobacco marijuana use in a 14-month period. So what we thought we were doing was developing an ethic for creating interventions. So we thought the methods that we used, we went out and studied what happened and then took those narratives, created performances, and created a curriculum out of those performances were something that would be replicated as an ethic. Well, we did this and suddenly the National Registry of Evidence-Based Programs and Practices labeled us an evidence-based program and people wanted to buy this program. You know, we're professors. We didn't even have money to make copies of the videotapes to sell them, to just mail them out. So we suddenly became a commodity and people did not want to go through the effort of creating their own program using a method. They just wanted to use the program we created because it was the only multicultural evidence-based program. In particular, it was the only one that included information and cultural references to Latinos. And so it became widely used and it was disseminated in a number of places. So the dissemination process, as Dr. Melendezia said, traditionally tries to seek the depth. So you create this program and then you try to replicate exactly in various different places. And any change from what you design is seen as an error or bad or something to be eliminated. And some of the programs have been more successful. Gary Holtz has a nursing program where he trains home nurses and he's been pretty good at getting fidelity to that program. Most of the school-based programs may be not so much so. What actually happens most of the time, a doctorate evidence-based practice, if they do, they use some of it, they mix them together, they add their own things. I just met with our local school district in Skate College and they said that they don't really want to use my program. What they wanted to do is work with them and integrate three programs they have into something just for them. And I said, well you lose the evidence-based claim from that and they said they didn't care. They just wanted to do what they wanted to do. So rarely did programs actually get used the way they were designed. So what we've argued is that because the actual practice is described as this adaptation, that in reality what we need is a practice-based science of prevention. We need to base what we do and what we think about when we design prevention programs based on how they're going to be used. When we come up with a different definition there which is fairly controversial in the prevention field and at this point some people throw things at us and I try to talk. Some people like it, some people don't. Usually people are out in the field. How many of you have been in the field trying to do health work or some work with kids or something like that? Well then you know what I'm talking about because whatever I tell you to do you're going to do your own thing anyway. So based on that designers should reinvent new populations implement those will adapt and participants are not passive recipients. Now we're not going to spend a lot of time on the last one we could do during the Q&A you can bring this up. But often in prevention theory they assume that if I'm doing an prevention message for you that you're just sitting there and sucking in and pounding into your brain or something but we know that people aren't like that. That they have cognitive processes and interpretations and social processes and that's probably all we'll say at this point unless you have some questions. So overall model, we've come up with to talk about the process of adaptation. Going in very specific directions and looking at the three forms we're going to emphasize the top two section segments. Back to that brilliant idea. Okay, designer adaptation. So the thing, again the hope is about fidelity keep it the way it is don't change it. Really negates the need to adapt to different populations and to different audiences. You need to work at the field of communication. You need to do audience analysis. You need to know who's going to get the intervention and the messages that they will be getting. So the principle of cultural grounding for health message design starts with culture, obviously. It's identifying from the ground up so instead of from the top down so this is what we know what works and this is the information we know that they need let's create this intervention, send it out it's instead going from the bottom up what is this culture? What are the values? What are the norms? What are the standard ways of interacting around whatever this health topic it might be? So starting with the culture and addressing its complexity multiple identities Again we have the strongest influence messages influence us very strongly when we see ourselves in our group memberships represented and reflected and acknowledged not ignored. Our very first curriculum that we're going to talk about in a minute is multicultural. The most effective one was multicultural. We did test again specifically at a Latino version, a black and white version and a multicultural version and which one do you think was the most affected across black, white and Latino populations in that in the particular area we were testing it is was multicultural because everyone saw themselves reflected in those messages and in those images active participation of cultural group members in message construction this is, I'm going to pause because this is really important instead of all of us academics up here just saying this is what the message should be really talking to the people who are going to be our recipients who are going to be the people who are targeting the target population members what are the messages how do we get them to help us construct messages so that the voice is represented as well in the intervention. So from us our programs are culturally grounded prevention messages and we point the from kids through kids to kids creating concentric health messages because we start by talking to the kids whatever messages where they're coming from and if I talk with rural kids their experiences of drug offers in particular our program is not about don't do drugs our program, our intervention is not about communication confidence it is about being socially competent if we choose to resist substances and it's about decision making and making healthy choices but getting youth narratives and rural cultures may be different than you say Orange County versus San Francisco youth are involved in the narrative message development and youth are involved in actually pilot testing intervention itself so why narrative, why kids stories why do we want kids to tell us their stories narrative evidence so again we have statistical evidence say 80% of kids will X well kids don't often see themselves in 8% what that 8% is they often see themselves in the stories of others so when I got offered drugs or when I was at this party this happened, I didn't know what to do and so X narratives give voice to student experience or youth experience it paints a picture to illustrate some of the statistical evidence that we might provide it's more engaging sometimes than statistical evidence it often reaches low knowledge audiences much more than statistical evidence might and often times it's just telling a story that renders complex information more understandable so we want to create new narratives for kids who really haven't really thought that so we have 5th grade 7th grade interventions we want to get a kid that passed before that really started using so large levels of use so we want to create new narratives of resistance new narratives of new norms, new narratives of how they can interact when they get offers of substances, alcohol, tobacco other drugs we want to reinforce healthy narratives if they already have certain narratives that are replete in this particular cultural group we want to reinforce those in the intervention that we have and in any unhealthy narrative what we deem as unhealthy in terms of substance use we want to be able to challenge and provide alternative narratives for them hopefully in that situation so the narrative engagement theory that we have several publications that deal with this so we have narrative messages that include characters, setting, actions problems and solutions we have a video we have a 10 lesson curriculum and there are videos in 5 of them but there's a story line across all 10 lessons these help build novel mental models alternative ways of thinking and understanding and not just don't do drugs but alternative there's a grain there alternative ways of thinking about substances and substance use giving them behavioral models of cool kids who don't engage in alcohol and drug use and improving narrative knowledge which isn't just knowledge of facts and understandings and thinking cognitive narrative knowledge is experiential I totally understand there I've been in that situation and I didn't know what to do or maybe now I do know what to do here's some alternatives for me leaning to changes in skills shaking norms, attitudes and intents more cognitive and then social proliferation we want kids to talk about the videos we want kids to talk about the messages we want them to talk about their other kids the other their peers the other members so we want this proliferation of the message to happen across the not just in the classroom but across the school and maybe into the household and we predict that this will impact health behaviors how do we do this so our method that you talked about that we thought we were developing this method is we start by interviewing to a lot of interviews with our target population getting stories getting stories of when a drug offers but it's also stories of when you used and stories of when families and friends have been impacted so what are the messages here that you've gotten from the situations that you've been in doing focus groups not only with kids but with people who work with this so focus groups with say teachers counselors individuals who work with the target population I was a teen advisory group so at every step of the way they didn't give us feedback saying yeah that's really that's really corny having them develop our messages like the logos and having them really part of message development video development from kids through kids to kids so from kids the interviews the narratives we always have kids who've never had younger than high school aged kids creating videos based on the narratives that we collected from the younger kids collect the narrative develop these video interventions with guidance by professionals the high school kids creating these videos to go into the curriculum and then thank those two kids so from kids through kids developing those messages those media messages to kids the audience and then we adapt the curriculum we talk about it a little bit we adapt it in many ways for different audiences and then we go right back to the beginning what you know what's working what's not working and keeping on adapting in that way so keeping it real is the name of our curriculum keeping it real AIR okay Nancy Reagan just saying no drugs from a communication perspective we've learned actually cross cultures cross regions there are pretty much four ways resistant strategies that seem to be effective refuse, explain, avoid and leave that was our original keeping it real logo that the kids designed and that was back in the 80s so refuse, explain, avoid and leave and we keep expecting new kinds of resistance strategies to emerge with all the people that we've talked to but most resistance strategies can fall into those categories keeping it real R-E-A-L stands for refuse, explain, avoid and leave and Mike is going to talk more about the curriculum okay so as Dr. Miladay mentioned it's a narrative intervention so the idea is that we have these stories that we want to enforce healthy models or reshape the stories that they have the approach is not to be moralistic and we train the teachers and the other implementers not to do that not to use fear messages fear messages are pretty effective on adolescent kids they don't work very well in those situations and so basically our intervention is involved with the stories that kids tell fed back to them and discuss with the classroom and what we're trying to do is to talk to them about the choices and the options and the way they make decisions about risky situations in their lives and give them a new way to think about those things you can see the five of them are video based we'll play some of the videos first video sets the stage here's an introduction attempts to get the kids I'll stop there and then one video for each of the usual strategies this is very much about how to be along with their friends how to talk to their friends how to form healthy relationships these are the narrative elements that we have in the curriculum the videos, discussions, role plays all created out of the kids own stories that Dr. Nolday referred to create all of these different narrative elements so this is the first video this is the interactive video for the multicultural intervention and when we show you the rural video let's see if you can pick out some contrast think about what you see as a similarity difference so this is the video that there it is that the South Mountain High School was the former arts high school graded for the schools in South Phoenix and in Mexico difficulties those of you here early weren't loud enough before right any questions while we're playing we'll try again if you want to work we write the script we go out and shoot it in this program we get to work every position learn everything and choose what we want to get into grade schools and we ask them what's the proper grade what then who would you expect to make a story out of it we script write it during the summer the two weeks from there when we came two minutes I said everybody can relate so I said I got here I know these people I can put myself in their place and this is how I go I did a situation on that so it's more of this to surrounding the people that are around what we're doing I didn't expect to use videos that I haven't seen in other videos is we're using realistic looking fake substances like vitamin B for crack or ready help and most of the things they don't really show okay so this was intended to be a multi-cultural urban videos or anything that would lend you to believe that we accomplished or didn't accomplish our wills the urban data the multi-cultural so there is American culture when we are talking international but the students that I saw in that video I wouldn't consider the diversity that's represented in the American population it looks like a small excitement or multi-cultural a person that was developed through the schools in south Phoenix 65% of the kids in those schools are Latino mostly Mexican art now 20% of the time we were doing this we were white and about 10% were African American so it reflects the cultures largely in those three groups we need to get more of the videos because as the videos develop they are based on narratives of members of each of the groups narratives from all over the groups I have to tell you that this is kind of a grand thing we started doing this as ethnicity and we got labeled the ethnic group in NIH the differences by gender were much bigger than they were by race youth culture overrides a lot of the ethnic differences in these schools because they were behind the integration that are true and that's one of the reasons that we were surprised for example when schools in Georgia and Florida and New York started to adopt these videos because they were both Latinos and it just seemed like it was not totally appropriate but then they would say to us we don't have anything that's any element of Latino culture so please let us use it what did the video attempt to do? anything you got out of that? what was the purpose of the video? why did the kids talk to the video? why would the performers talk to it? yeah kind of what you were referring to getting credibility and authenticity I was talking to resist the message given it's something you relate to we did, we did the whole thing ourselves right and then they're involved and how they were involved eventually they talk about why they were involved in it so there's that whole big process beginning there's the overview of the backwards narrative it starts off with that shot of the freeway you really won't see that in Central Pencil can I say it? Central Pencil Tucky Philadelphia Pittsburgh, Kentucky in between so anyway that's the opening video this is a very great story when we worked in South Angus huge neighborhood in South Angus it's just really really big that was the best people in there when we got there we discovered there were no shopping centers and no movie theaters in Carnet all of the kids from this neighborhood 45 schools are involved in our study on the same bus and went to Alituki which is a white middle class neighborhood just outside of South Phoenix to the same shopping center to go hang out and these kids that we see became huge stars middle school kids would be running up to them and say oh I saw you in the videos in our school you're really good actor you're going to be a star the woman that you saw talk about at Harvard by the way pretty cool and actually one of the people in our cast high school kid was a very serious alcohol and the little kids were running up to him and he said the sisters and the brothers need me and he went cultured we were trapped for four years we were sober for four years I believe we saved that kid's life totally unmeasured, totally unreplicated probably the biggest success they've ever had in 27 years of prevention research but yeah and that's one of the things that we lose when the program goes national because there's not that feeling that I know that kid he's like my friend's father brother and cousin and things like that which you get in the world of community so this study as I said there were 45 schools there were four different conditions the videos, the Latino condition were made up only of the stories that we got from Latino kids that were actors from different racial groups but mostly the stories were all Latino I could talk about why it became white black later multicultural mix up we gave pre-test, pre-post-test the last post-test 14 months after the intervention ended in which we find using growth model, general estimating equations we find statistically significant effects on tobacco, alcohol and marijuana use this is kind of graphically related what's interesting in this that you don't always find in this population is the fact that marijuana use is higher than tobacco use but that actually reflects University of Phoenix and Latino culture and their choices so did you explain what the control was the control group did not they continued with what they wanted and one of the problems with school based prevention is you can't tell a school well they don't use anything now so some of these were using evidence based programs of time kind of waters down the effect but yeah they continued to use whatever they were using up to that point so that they in fact many of the schools to this day are still using it but we can talk about that too so currently finishing up a project in rural Pennsylvania where we adapted to keeping real, re-grounded keeping real to a rural community in the research that attempted doing that by open source the key factors, the biggest difference in REAL is that for the explained factor rural kids say this is not who I am and they all know the only other adaptation beyond that were among the lives of a group at the University of Central Florida that adapted to teaching Latino sexual prevention resistance and the other strategy that came out of that discussion was I would punch you in the face so I don't know if they actually do that but they said new materials, new role plays the eighth grade for booster lessons where they create their own messengers and this is the the new logo that was created by the kids for their own and here is the video let's go to the tape hurry everybody, quiet on the stage get things started, okay? and action we're going to start with the blind shot and we're going to repeat it I want like, you know, with the kids to see what I schools really like these children are all made by kids they're written, produced filmed, edited I like how they're having fun before the grads come our goal is to make these scenes as much like real life as possible and the movies use techniques from keeping it real how the kids see that, drugs and alcohol do not benefit you anyway does this look different from the other one? okay, the rural kids said please just don't show cows they're a major reverse or Amish people now, no cows, no Amish people don't portray us that way but you've got the same idea with the kids trying to say oh, we name this, it's us it's from kids through kids to kids to have some identification with it and then said they're really immediately on rural environments what looks like them there were no other programs they created for rural kids and all dated on the use of rural kids is the first program that will have to fill that purpose it's surprising that 20% of the country put into the kids' intervention in rural environments they're really quickly so they came to us and said how many of you have been through there? how many of you like there? you like the new there so I've been telling them they're starting to let it and they always created their own interventions and after a number of years and a number of programs that shelly said did not show the effects they would like they decided to adopt an evidence-based program they looked at a bunch of them they decided we'd get them the best and hence they didn't charge them very much and so in the middle school version they were fighting with them 250,000 kids, 23 countries then they decided that they should redo their elementary program where they get terrible results in Philadelphia and that is just going to fill out or go to about 1.25 million kids and 44 countries around the world making the largest prevention program in the world it's thought they could have something more than that again but this is the cover let's get through this the brief intro it's a national deal so the challenge is it's national how do you culturally ground it so that's a question to consider these are the representatives there are rural, urban, and sub-urban versions of these videos but they don't have the same intro which is national now the fifth grade curriculum we chose a little different because fifth grade kids there are a few kids are actually using substances at this point so we're getting them at the decision-making stage the time when they are really pulling out the way of becoming independent thinkers we started out with live-action with these kids but we are using peers so we're using these 10 to 12 year old kids where a psychoanalyst is one of the talented kids trying to get real full performances out of them is a challenge so what we chose to do again, for kids is that we we are using live-action and animation so the intro of the videos for the fifth graders for Der is live-actions, live-action characters but then they work and transition into animated characters by the end of the very first lesson and then there's live-action and animated characters throughout the entire curriculum but I'll show you just a little bit of a clip from one of these and then I tried it I'm really glad I did I really love it I was like, what? I just looked at it and said, no way I'm just awake right now and everything I'm not going to cover for you What do you think a lot of your mom does find out? Oh well, I'm also my karate class when friends talk to me to take me a chance and try karate, nice to know for a few months and then I tried it I'm really glad I did I really love it If we go into positive risks not just drugs or evil and bad what are some positive risks that we take in our lives and we're trying karate we like it if you're doing it but some of how we're customizing the curriculum to fit different audiences Now let's move on because we only have a few minutes here and I'm going to like about three or four minutes because I want to take questions that's designer attitude that's us as designers adapting for different audiences because it's four kids three kids, two kids trying to adapt but we hand it off, right? we create these, we hand them off to teachers to implement or to officers to implement we're really interested in how are they changing the recipe, right? how are they altering the recipe what's happening in the classroom so school adoption of evidence-based programs are very popular now because for example, SAMHSA has a registry of effective evidence-based programs so if I'm a school principal I can just go to this list like data did and choose what are the effective evidence-based effective programs out there for my topic here obesity prevention, drug prevention what have you in terms of that so they're becoming much more prevalent so we did a study here we did 31 teachers in 25 schools 70 students that we studied we wanted to know the degree of implementer change, types of changes reasons for changes we got teacher self-reports as well as we got observer ratings we randomly selected 276 of the 730 videos the teachers put we had video tapes at the back of the classroom and the entire lesson was video tapes all these lessons shipped over to us and they're all digital so we have all this digital data if anybody's interested in doing video tape analysis come talk to me but we randomly selected 276 videos to look at the adaptations the teachers were making teachers reported that 32% of the time they delivered in fidelity they did it as we asked them to do it in the curriculum however the observers when they rated it 2.6% of the lessons were developed with fidelity I'm not going to go over all these but suffices to say partially a many lesson component was most popular because it was coded as that if they left out something but we don't know from just this if it was important something what I think is consequential is completely omitting there was clearly a component of that lesson that we put in there for a reason so completely eliminating it adding their own content their own two cents and revising content doesn't mean that what they added was inherently bad so no it wasn't controlled and the message wasn't in line perhaps with what we had developed revising the content changing saying something that was supposed to be done as an individual into an entire class so what if it's about privacy what if it's asking kids to disclose some information you make it a full class activity or if it's something that we wanted to establish norms and we wanted it to be a whole class activity instead you do it as an individual or a dyad or something so again these kinds of changes there's all sorts of give the example of the one guy who so we train them among other things we teach them set up a camera every lesson ends with roll points he walks in and he goes like this he opens the book and goes okay we need to do some roll points today break through the end of the thing and he's taking it himself he's taking all the pain to do this lesson any of you were teachers we do that we adapt the curriculum the information to suit our needs and that's exactly what happens is teachers give reasons for doing this mostly practical considerations that didn't have enough time I had other things going on we had an assembly whatever so I needed to truncate it to the most fun stuff and I'm adapting it to adapt to our rural students or because half of these students we had the classic curriculum half had the rural and then there were controls so the ones who got the more urban curriculum would say we're adapting it to make it more rural or make it more appropriate for our 7th grade students or to my actually my own teaching style we found that teaching style is huge so we're trying to really identify what are the styles that teachers are using to implement curriculum maybe customize messages specifically for them or lessons that cater to those particular styles so implementation quality again for us we're looking at the delivery quality the kinds of styles that the teacher had the fidelity of the implementation and outcomes it did have a significant effect so implementation quality quality quality so we don't know right we want to measure outcomes we want to measure drug use if we don't find any effect is it because we have a bad curriculum and we didn't adapt it well on our end or is it because it's not being implemented in the field in the way that it should be so implementation quality and adherence to it did have an effect on our particular outcomes in the onOS test delivery quality was significantly related to norms and substance use but not related to efficacy child's efficacy adherence was significantly related to norms marginally related to substance use and not related to efficacy do you want to step in there with any clarification well I just think it's interesting that if you deliver it well you're animated in the way you teach it and the students are engaged in it's not important it's more important if you actually teach which is inconsistent with that view of adaptation and pure fidelity can you explain maybe the efficacy part I mean it's not clear to me why you wouldn't correlate with efficacy if it's not delivered appropriately you go back in the last two bullets yeah efficacy means itself efficacy is their belief that they can perform the behaviors and so the I think one of the problems we've always had with these particular set of RAAM strategies is because they're so indigenous to them students recognize them you give them a self court measure even if you haven't got a court certificate they say they do them all and I think part of the problem is that when you're asking about that they say they can do it because it's the way they naturally do things so I think there's an epilogical problem in traditional design where you give people pre-test and post-test I would say a bunch of things that would you know for the future that we want to do and things that are curious, we only have a few minutes so I would like to build questions questions yes so I guess most of you are in the test of NILA it's kind of interesting to look back um this is I think it's a long time ago so when you're looking at these populations it obviously makes sense to address different demographics you know in different regions but within a high school cohort even between saying these demographics there are even different subgroups some people are definitely more susceptible than others and so I can imagine you know there's people who are kind of on the fringe who maybe associate with groups that do drugs and groups that don't do drugs there are those who don't associate and there are those who are simplifying it, there are those who are very strong or engaging in it and so I can imagine that in each of these three groups they're going to respond very differently to these videos and some may just look at it and be like what's my name and some may be like I don't know I'm kind of hesitant towards it and others may be like wow this is great so you know within this are you targeting a certain one of those groups or are you in fact somehow trying to put in a specific facet that that will try to attract each of those yeah the targeting this is a prevention, intervention we're not dealing at all with really high risk groups or treatment in terms of that so it's not that we're eliminating that but we don't really address that it's much more prevention inhibiting initiation as well as decreasing use once it does happen in terms of really high risk so we don't really fully address that in this program yeah I think it's really a good question the answer is part of the thing we can go on for an hour about this question but remember that there are multiple narratives in every lesson and our goal is to select narratives that cut across a group of people to basically re-brand or re-story what means to be a drug user or a non-user and basically create this image of non-use that is popular and exciting and mature because the indigenous narrative about drug users are that they are mature adventurous, popular it's a very positive image of people who use drugs in middle school and early high school and frankly when we measure the personalities of those kids it's inaccurate portrayal of them secondly one of the things that we are looking at down the road that comes into that participant thing is we have a number of people on our team and what we hope to be able to do is to track the message into networks narrative networks and track where it's infused and not there's a woman named Linda Collins at our university methodology center who's talking about a chapter in convention and the idea being that once you get the message out you see how it's working and then stage two is to adopt to its values so if you look at who the message reached and who it didn't reach you can send the second message out and see who it's not reaching decide what message it could be and how to get it into the audience that's the future your work the dare program see dare started as a program and then came to the realization that they were a dissemination vehicle that you have these officers who by the way deliver curriculum with greater fidelity to the kids in class so they had this training marvelous training and dissemination vehicle but they didn't have the right curriculum so they adopted ours but we have not been in the field to evaluate it taught in that framework yet looking for a donor seventh grade's been in for two years and the elementary discipline in this year so they're using keeping real now the officers got retrained to use keeping real so we did that so we had our curriculum and then we adapted it specifically for officers to go into the classroom and I'll tell you this talk about constituencies we have officers, we have sheriff's departments we have education people, we have counseling people you have the kids so a lot of people left the table redesigning now and officers are not teachers so the expressions are very explicit you know all we know is that there has been very successful in marketing our program and the number of schools and the number of countries has increased we don't know yet starting next year we're going to be evaluating we're doing a pilot study right now yeah we're doing a pilot review well first of all this was an incredible reputation and a really excellent example of how development interaction and then suddenly in the past it was just very exciting to see I have a few questions the first is the intervention rate on that so there are a different components do you ever break it down to what you can see exactly like behind the block box which piece of the activity will be in and which part and then the second question is obviously this is expensive I mean your videos and production it's just incredible those of us who are beginning our careers and how do you move from sort of your basic program to something that's finally wonderful I think for you I have a great mentor and I know people want to get going so quickly and I can talk to people at lunch about this but yes we have plans actually what's the secret sauce what are the four dimensions of the different curricula so then if I have 20 minutes which one is the white kid and we are studying that behind the black box what exactly you know how does it affect how do you do the two second explanation how do you do that the same person I mentioned before in one of the columns there's an article about something called the most system it goes back to now some variants factorial designs you just give some of the program to some of the people some of the program to other people and you do it that way I have to tell you that schools tell us but the reality is there are kids in every school that can make these videos and maybe better than what we have here I actually don't think it costs very much for every school in the country to make their own videos their own versions of the names because you know there's little cameras right here they're fine they cost $100 they're $150 and so I think you could disseminate I mean you know what I don't think it would be so terrible if they used iPhones and created YouTube videos and that version the original video the original keeping a real video costs $48,000 so the Metro version is statistically at first one of those components the things that were dropped the things were kept in kind of cursing those out and identifying those and then going out and testing it using that strategy we do know that if they see just the video well it's not a real experimental design if they see just the five videos that you're stabbing then they're done it's actually four or more thank you thank you