 This is one of our big issue big topic webinars and this week. It's talk data to me I'm delighted to introduce Erica marks and Carolyn Silphon to take us through over to you Okay. Hi everybody. I'm going to I'm doing the tech and doing a little introduction sharing control Well, not sharing the tech controls with Carolyn because she's like hundreds of miles away but sharing this Presentation there's going to be three of us. There's myself. There's Carolyn and then Nancy. What is coming on at the top of the hour? we have 90 minutes and That and the order of things is going to be so this this webinar is a recreation to some extent of a session that the three of us did at the AI and conference in 2019 in Stony Brook and Luckily our session was like a Friday morning or something when we all were fully alert because by the end of the conference we were like kind of going to activities that didn't involve words, but at the very beginning we're back and The way it's what we're going to do is Carolyn's going to first introduce About just science and how to approach how to connect science to the type of work We do and things to be keeping in mind Then I'm going to come in for a little bit about thinking about how to work with your client and how What kind of questions to be asking to get the bigger context of why are we doing this work? And how do we measure success in the large picture? Then we're back to Carolyn and she's gonna we're gonna about 20 minutes on Creating learning objectives for your sessions that you can get good data from that you can measure how to tighten that up We're going to send you into breakout rooms after she orient us so everyone will have five minutes or so in a breakout room To workshop your session a little bit based on what she said then I am part two Which is collecting real-time data in the room So interactive ways to collect data and have it be meaningful to your clients while you're running your session And then again We'll send you into breakout rooms to talk about how maybe you could incorporate those tools or something else and come back and then Nancy Watt should be coming on the call at the top of the hour and She's going to do a section on appreciating bias like understanding How does bias come in and how does that? How do we work with these messy humans that we are what to keep in mind? And then she's going to send you into breakout I'm going to send you into breakout rooms again, and then we'll wrap up with a group discussion That is the plan So Yes, go for your interact us throughout yeah, yes, we're trying to build in this and of course we have all this plan for you so See what else do I want to say this is being recorded we hit that button and We have a feedback survey at the end So we'll have a new link at the end and many of you completed the survey at the beginning Which is a recreation of the part of the worksheet that we use in that session to help you get thinking about your About your session. Okay, Carolyn. I am turning it over. Do we have anyone who was at our session or are we all new? Okay Are we gonna do this with slides or with that? Well, I guess while we're while we're getting that you got right now, Carolyn So I'm relatively new to the AIN community. I'm a physics professor That's my background. They're very passionate about education and My hope as we enter into the next decade is that we and by we I mean as seven billion people Can learn to make better decisions grounded in both compassion and I've been really impressed with what I've seen with applied improv in terms of developing compassion and then the other The other piece and we can yeah, we can go to the to the Feynman slide Is how we can ground what we do in evidence as well? And the idea is our brains were naturally selected to survive not to be correct not to be accurate and It's very easy for us to fool ourselves. And so we want to constantly We'd be checking and testing and basing our decisions in in as close to reality as we can and we can never Exactly perceive reality perfectly But we can get closer and closer to perceiving react reality more and more accurately And yeah, this is a quote by the late by the by a late great physicist Richard Feynman the test of all knowledge is experiment. So how do we do this? Typically it's some sort of iterative cycle You've all done this in your workshops. I'm sure in a typical workshop You observe something you analyze something you test your assumptions and you test what you see You test the model that you create based on your observation And so this is also the scientific process. And so how can we? delve into this and make this more precise and and and use this to inform our work and Improve what we do Since we are in this case, we're dealing with Education with learning with people with emotions. So we're gathering data about people whenever we're gathering data about people We want to consider The ethical considerations behind it This community I know is extremely compassionate already. So it's more just how do we be mindful of things? We might not even be aware of So be completely transparent if we are collecting data how we might use that how the data might be stored Ensure that participation is voluntary and that participant may withdraw at any time So in the interest of that here, we are collecting some data, right? We you most of you have filled out a form already And we will be using that data only to inform our own The to improve future workshops, so we will not be sharing that data for from this webinar And it will be stored on Google Drive and you may use a pseudonym as you have likely seen in the form So please feel free to that and you may leave the webinar at any time. I See a poll. Okay, great. Let's see what happens with the poll says do you use tools to measure the results of your section? So here are here are the non-existent results that you could probably see now. Oh I see results Really, they all say zero to me No, let's look at 38% no 38% other 25% All right. Well the leader the emperor the leader here has no idea what's going on. So I see zero for everything. I'm going to close this Stop sharing there we go. So the second question is That you can answer in the chat is what tools do you use so if you answered? Yes The use tools go ahead and just type in the chat. What tools and I know you did this on the survey Where we just got back. Yes. So this way we can share with everybody So it looks like the answers that came in on the poll According to Doug who reported them here were 38% yes 38% no and 25 percent other right And I'm reading these because I believe in the recording you won't see the chat So what I'm seeing coming in is Single post workshop survey form sometimes smile sheets, which means like at the end everyone's like did you like it? Yes, I did. Yes It was great a few questions for qualitative info some scales sometimes I use google forms with an a to 10 scale Maybe 110 evaluation from the participants and what the objectives of the sessions were so from one to 10 scale So google forms afterwards Some people are using things in the room. Some people are doing online A lot of post survey things post post session evaluations three qualitative questions three qualitative questions All right, do you have any any comments about that carolin before we Move into mostly what people are using thus far our surveys to assess the result Yes so and um And again in the pre survey we've asked everyone to come this is a working workshop, so we've asked everyone to come with a specific uh With a specific thing in mind a specific workshop in mind Yeah So here's what to think here's my little piece about thinking about your specific workshop. So This is thinking about When you meet with your client like you're doing a workshop and it's very easy to get focused on like my awesome workshop And is my workshop great and do they like it and just be in this narrow box And that's that's important and we're going to spend time You know being able to answer that because we as facilitators and trainers obviously want to know Are we immediately successful with the goals we set out? With our individual workshop or our little training whatever training program we're doing and it is important to whoever Is loud people please meet themselves. Um Yeah, Melissa could you move yourself? Um And it's easy and that's important right it's important to define your objectives for just success in that single Three hour period or one day period or six week period And your client has hired you Not to just be successful in your period, but they have a goal right so when you meet with your client I like to map out three these three places Like where are they now? So what's going on in their business or their organization? What's going on that has them having a conversation with you? Where do they want to be and how will they know they get there? So maybe they want to increase sales or maybe they want to have their team of Like have some organizational goals or some results they want to achieve As a team So what does that look like that may be a year out or two years out or five years out? Where do they want to be what's the bigger context? And then an interesting question to also ask them is if things just keep going the way they're going And just what is your most? Probable future that you're living into so not Um, not good or bad just like the most likely default future Now the gap between their default future and where they want to be The bigger that gap is the more likely that they're looking for solutions And if your program is whatever it's going to fit in that gap so That gap People are going to hire you if that gap is perceived to be big So either the pain of their default future or the greatness of where they want to be depends a little bit on The type of clients you work with whether you're working in the space where people are trying to avoid a painful future Or whether they're aspiring to a positive future So you can lean into each of those questions to see which one is going to be more motivating to your client So this is and and the answer is to how you're how they are going to know i'm going to stop the share you get the idea of the Like my my cursor doesn't work on the screen This is great. I'm love being this like tech show All right, I think i'm back am I back Can you guys see me? All right good um so Well, as I say so, yeah, so those the answers to like how your client defined success May not be measurements that you can measure, right? There are things that are internal to their own processes there are things like retention or sales or You know, there are things that are about them. So having that conversation so that you have those Numbers or those qualitative measures to go back to with your client is an important part of the process So I am asking in the chat go ahead and type in If you know like how is your client when they look back and they say wow, I'm so glad I brought in that consultant What are the numbers or what are the measures that they're using to define success? So if anyone wants to type those in We'll read them to you the bridge goes over the back bridge the gap. There's lots of bridge Uh, okay. Well, there's some there's some feedback here about the language Um, yes your program is the bridge of the gap or maybe it's the ladder or maybe it's maybe the gap is uh I don't know. Yeah, you can use whatever language works for you So let's see did anyone type in I would say delta, you know like the difference Use the client use the language that works for your client is really what you do You listen to them and then maybe flip it to be a positive frame because that's part of our job To be professionals at looking for opportunity and creating opportunity So this is a really I you know, whether people type in or not This is a useful thing to take a note of to make sure you're having this conversation with your client And that can be a big service to them to really dig in and ask them over and over again, right? Like how are you going to know it's going to be successful and then and maybe to build in an evaluation Make sure that they're also evaluating how they're doing. So maybe they already have an annual survey Maybe they already have things that they do make sure that you know Just help them tighten up their own thinking about are they Are they paying attention to what really counts in their business? And you're kind of helping them think that through and then it really gives you great information for designing your session Yeah, and we do have Paying avoidance is one thing that has come up on the chat as something that paying avoidance on things like communications So reducing pain on communication um And I would dig in more right like how do you know pain communication pain has been reduced, you know sometimes Like a client I just work with they're like I want less she was the HR person. She's like I want less people in my office I want less conversations like okay. Are you tracking those conversations? But can we turn that into Something that that you can keep track of so we can really know for sure in what way is it have we succeeded or not or made a difference Okay, there's also how it's so impact when the when the clients views are Retentions costs revenues and these are very able to keep in mind as part of the bigger picture So we can help bridge that A connection as well Yeah, why do you want to have less conversations? Yeah, like what keep going like You know, why is that important? Why is that important? Why is that important? How will you know what difference will it make? Like, you know, you're really helping them See and create the future they want In asking these questions. I'm curious What success people have had in Tying to more measurable business results I did have a client who said after we did a team alignment meeting a two-day meeting that there were fewer HR issues And Exactly Has anyone else had Yeah, I had a hospital that I was working with that I was proposing with their measure of success Was that their nurses would be sedating their patients less They wanted me to come do like comp help their nurses be better at dealing with empathy And better at dealing with conflict and their measure of success is there would be less incidents of their nurses The current way of dealing with patients was to sedate them All right. So that was a very concrete intense measure of success in the big picture So, all right, I'm going to turn it back over to carolin for um creating learning objective So now we're going to go back to this so so it's good to keep that big picture in mind Now we're going to go back to the smaller scale of our specific workshops And yeah, I see there is another academic in the house as well So I do think of this in terms of learning goals, but these are training goals. However you want to think of them We want to be able to Be very specific about what we want to do and then we will be better able to assess whether we achieve those goals now again not there are Goals on larger scales that we might not be able to get measurements for we don't want to say those aren't valuable But we there is a lot that we can measure and we want to be As scientific as we can about what we can be scientific about not that there's many valuable things that are beyond science um And so the goals and if you did the pre survey you thought a little bit about what are the goals for your set We want to try to be very specific about these and think about And both direct measures of these goals and indirect measures People thus far have used surveys which tend to be an indirect measure. We're trying to get at you know, what did people actually learn? It can be really powerful to think about Can we actually measure what they learned? Can we actually measure the behavior that we're hoping to see? So i'm giving an example from my experience. So we want to we we Shortly we will put you into breakout rooms and ask you to get very specific in terms of Participants will be able to do what after your workshop that they they perhaps struggled with doing before um So when I first think about this for my course, I think oh, I want students to make meaning out of physics and mathematics And most most of you are probably well, what what does that mean? Right? That's that's very very big Or we often think oh, I want them to understand. I want them to communicate better And it's these are all glorious goals and big And so how can we make this more specific? So um The a more specific goal like what do I really mean by that? oops, I mean to interpret generate and translate Among multiple representations. So this is you know, like wraps the equations words of Phenomena in the physical world. So being able to you know, watch a Ball rolling down a ramp and translate among multiple different representations And that's what I mean by making meaning out of this be able to describe it Using multiple different ways and analyze it using multiple different tools Um, and this is something that I could actually Test I could observe right like I could give students a problem and be like, hey Can you actually represent the same physical phenomenon? In multiple ways using diagrams graphs tables words equations So I could get a direct measure of that goal Sometimes it's hard to collect that data. So we often do use indirect measures as well But the hope is to have something very specific That we could in theory at least collect direct data and see a change in behavior Um Ideally, yeah, so the question came in on chat before and after That would be ideal If we can do that and so presurveys are great. That's you might have noticed we kind of did that, right? Like what are your goals? How do you state your goals before this webinar? How do you state your goals after this webinar? So that's great because that's one of the goals of our webinar meta at the meta level Yes, it's great if you can do before and after generally at least at the end at least To be able to see hey by the end are they are they able to do this? But the best would be to to see is is there any gain? Is there any growth in this ability? So to help you uh with this I'm going to give you a Link to some so this link that I'm sending in is a bunch of verbs. So it's basically this Um, what we're showing you here. I find this is a very useful tool to try to be specific For the educators in the house. It's loosely based on blooms taxonomy um To think in terms of how what specifically do I want people to be able to do as a result? And to try to use very specific verbs and active verbs. So again the prompt is I'll put it into the chat participants will be able to And you're going to want to fill that in And these are just some example verbs to help Get the brainstorming process started Um, and we're going to put you into breakout rooms because this is often easier to do with a partner You're going to have five minutes To discuss your workshop. So you won't you'll obviously introduce yourselves Share the workshop you're working on um, your initial goal that you had In mind and then try to help each other In the meantime, I'm going to send you all into breakout rooms, which is another technological miracle that you're all going to appreciate So I'm going to send you into breakout rooms so you can work on that Like participants will be able to what and get that into a nice concrete form. Yeah, and now we'll try to Yeah, so copy that link at least because I'm not the link might go away when you go into breakout rooms So copy that link you will get access to it while you are in your breakout room. We promise and uh, and the goal will be in five minutes be able to tell To share and chat your partners Both if you can all right, you can send us a message in the breakout rooms by typing one in sending it to all Oh, okay. That does work. Yep five minutes They're enjoying their conversation Yes, that's the sign. Yeah, they have seven seconds and they're going to get all booted in here. Well, that's good to know Yes They love the full five minutes. Yep Welcome back. So if you are willing to share your partner's goals in uh In the chat and we can read them out or if you want to share them verbally, whichever you prefer I start. Um, so I was with david and uh And you're welcome So um david said that he's working with um, a client experience team with uh, with um, um The charity it's like a charity over several states And they work with patient experience to improve the empathy from the medical staff and um And the objective was that to to develop the empathy And actually we didn't get the time to to get to to specifics, but we actually came up with the idea that Co-creating that with the client and starting with what you want to see improved and then turning this into observable concrete measurable um All behavioral kind of countable things would would help great and we have a mark and feud uh So we want to know what are the what are the goals so the the to do So the participants will be able to some examples of specific things that that came out and we realized that might not have been long enough for everyone That's it's it's it's we're trying to squeeze a lot into here You know some of the goals that I'm I'm Looks like I'll have the opportunity to work with a team over three months and kick off with the team alignment meeting and then help them take Their uh create a playbook and then actually bring into fruition about how they work together So, you know, one of the things they want to be able to do is openly give and receive feedback um respond emotionally intelligently to triggers and You know create psychological safety Um, we will do some type of pre work assessment Some of those things are going to be interesting I'm not sure how to measure how well people can openly give and receive feedback I mean we can give them With more time we could dim we could brainstorm that but we could I think we can all individually imagine like Improv games that involve giving and receiving feedback where we could actually see how well they do Hard to measure but certainly test certainly in theory testable directly and certainly indirectly By people self assessing and more people reporting before and after lots of ways you could ask them or ask someone else Yeah, that's a yeah, so it'll be a mixture of observable skill Because ideally I'll be able to work with them in their meetings and of course assessments along the way Yeah, cool Well, I think it's a good segue. Um, because I see David in here is about Scale of one to ten how psychologically safe do you feel on your team? so like you can ask scaling questions And that's a good segue to something I do often different variations I'm going to show you two variations of using scaling questions in the room So scaling questions meaning you ask a question and you make a statement and people respond to Like how strongly they agree or disagree so you could think of that as a scale like zero means I have I don't I completely disagree or I have no connection to this and ten being You know, I completely agree or I strongly feel aligned with this and if you ask people in surveys, you know They'll give you numbers One thing I like to do is do something to get it in the room and in people's bodies. So if you if you designate Oh, um, let me see if I have Slides maybe the first the first thing I'm going the first method of doing this. I'm going to show you Is perhaps the most complicated So and it has the benefit of allowing people to answer in a way that They get to physically feel what's the right answer for them and Their data is anonymized in the in the room so that If you're asking a sensitive question like how psychologically safe you feel on your team By definition if you're asking that question and someone does not feel like they can Be truly honest on that team and you're trying to ask it live in the room You're not going to get honest answers, right? And you're also going to be putting your participants on the spot In a way that is violating the trust that they're giving you as their facilitator So this is a way to ask these more sensitive questions To allow people to use their bodies and to protect the anonymity and it becomes an empathy exercise Let me see If I have slides that are going to be specifically helpful for this so this I'm going to give you an example of A way I ran this exercise for Lewis wows for who is at the new jersey institute of technology And he just started recently launched the new jersey Institute of technology center for applied improv in theater at their at their campus And they had me come in and he wanted me to do a talk on confidence But also about they have a very diverse community also about inclusion and people feeling like they belong so it was like a That's why I created this talk or this workshop and my Way of framing it was that your sense of confidence Is a co-creation between what you're giving out in the world and then the reply you're getting back from the system that you're in So the first work being to the first context to set the frame for how do you improve your confidence? or how do you have more confidence is to reveal the system that you're operating in and I use these quadrants that cat coppett presented at the AIN conference in Irvine a couple years ago And I'm going to put them up on the screen so you see them So there's two axes in this in this slide There's two concepts in here. There's a concept of Having a low sense of belonging to having a high sense of belonging And a concept of having a low value on uniqueness and a high value on uniqueness I'm actually going to take the slide away so that everyone's not just reading it Because I want you. Whoops. Sorry guys There we go. All right back to me. All right So the two axes and what I did is I I didn't show them that slide I'm showing you that slide so you can see like what this is going to map into what I did is I gave everybody a card They look like this so people got a paper card that looked like this And then I had signs on the wall And I had a sign on the back of the wall that said low sense of belonging And I had a sign at the front of the room that said high sense of belonging And I had I defined what that was so a low sense of belonging feeling like I don't Belong here right other people belong here, but I don't There is an in-group. I'm not part of it high sense of belonging I feel like I belong here and I am part of the in-group and I had people just physically walk back and forth in the room And not to stop where they felt was true for them But just as like just to pass through where felt was true for them And then when they had that like barometer in their gut, then they would mark on their paper where that was Then I did the same thing for this axis of a sense of my uniqueness is valued So what makes me special what makes me different my culture my heritage my perspectives my opinions my My race my gender all everything that makes me unique as a human Is that do I feel like that is valued within this group? Or do I feel like that is actually a point that makes me excluded from this group or is not valued is actively rejected So they walked that axis So they walked they feel it they feel it they figure out where it is then they mark that on their card Then we do a little like how do you cross these axes and what you end up is okay? Where is their current state at the cross so they'd end up in one of those four quadrants? And if you saw if you memorize that slide the four quadrants were inclusion, which is both of those measures We are high exclusion both of those measures are low Assimilation meaning high sense of belonging but you have to sacrifice or hide some of your uniqueness or Differentiation, which is like tokenism high sense of your uniqueness is valued, but you're not you don't belong so So they they got to see where they ended up and then We also I did the same thing as that gap slide or there was it was renamed here in the chat But the gap slide where I had them say okay If this plays out the way it seems to be going For six months Where are you going to end up where are you and they walk both the axes? Where's your default so they could define where is their default future? And where and then also ask them Where do you want to be in six months? Right and then that's also kind of a test of people are people understanding this because presumably they'll be high on Both axes and they'll be both up in the top corner. So they don't write their names on this They just have this in their hand It just has a star across and a dot and then I have them put the card by their side And then just swap swap swap swap cards with like five other people until now you're holding a card. That's not yours And then I have them hold that card and then they map the whole group maps themselves in the room So now you're standing for somebody else who's in this room And then it becomes an empathy exercise And I made it and I definitely like Make that real for people like you're standing for someone. What would have someone stand here? Where are they going? We you know, where's their Default future what might have someone in this group be answering the way they answered here and it be it's uh Yes, carolin. Yeah, it's confidential. It's anonymous and it's the people who are in that room There are people in all four quadrants. It was very powerful that we did the conversations there They had them walk to the default future of their person They talked with the people in there to be like, okay, what would have where'd your person come from? You know, what is the pattern that's happening here? And then sometime in the in the desired future And then at the end you collect the cards Because do never leave your data in the room Even if it's anonymized because people will say they'll give you the cards back But they won't you don't if you at all can get your stuff back take photos of anything So you collect the cards and then you can plot that stuff and give that data back to your client If there is a big gap between I'll show you for a different quadrant exercise An example, how long does that how long does this typically take? It depends on how much time you spend having those meaningful conversations while people are standing in the quadrants, right? so But if you just want to like get that data maybe 15 minutes 10 minutes You have to really make sure people understand how to do the quadrants and how to mark them Let me see if I can show you But the point is that it's a meaningful activity to be doing And you're getting the data I'm going to show you so here's two other quadrants. I think you can see this Someone mentioned psychological safety. So those two axes in Amy evansson's work are about accountability How how strongly do you feel like you're accountable for the the success of this team as a team and how safe does this feel like a team where everyone can share without fear of Of like pushback at something exact words Same thing it generates for quadrants It's another kind of measure that you can't just ask people because you're gonna You know, you're gonna put people in the spot and you're not going to get real answers And so I did that same one and I'll show you an example from a A client. This is where they were current This is where they wanted to be desired And then this is where their default future was and you could see how it like it lines up on kind of those extremes Right, they're either going to app it in this case. They're going to advocacy or they're going to performance And in some one when I was really ambitious about this I took a client and I made like little movies that showed all the dots like moving around So you can really get into that This was the quadrants. We did at stony brook We put the importance of implementing all the stuff we just taught to you and your confidence and your ability to implement We did the same thing asked the same questions people plotted I made a little key here of these blobs See these things that look like a disease all over the slide. Those are like the answers of different Amounts and so you see a lot of people In high important slow confidence and high importance high confidence And a few people who are like, why am I in this session? Maybe one person? This is not important So that's that Let me check the time I and then I'll tell you one I'm going to present one other way to do this if you're dealing with a system And I just did this at a retreat where You want to get a snapshot of where everybody is you want everybody to see this snapshot And you can ask questions that are and you're mindful of the questions are not too Are People are going to be able to answer honestly and it's going to be meaningful for them to answer You can do this activity called constellations And one way to get good questions is to work with the team and make sure that They're the questions they want that everyone answered and the way this works You put something in the center of the room That represents complete agreement or alignment with whatever you're asking them about I think I can pin my video and then you'll see Ah, there we go. I got the controls back So you put something in the center of the room and that represents just a physical thing and you're like this You know this this coffee cup represents or candle or whatever represents complete agreement And then you have everyone just walk around and then you read a neutrally worded statement like Um I have I am clear what my role is in this project and people walk around and then they stand However in proximity to that central object To the degree they feel like that is true for them. So you kind of establish the boundaries It's really the same as like zero to ten scale But it's physically in the body It's called consolation So it seems cool and like you get in is more organic looking because people are just radiating out from that center point If you wanted to collect data You could have people put a dot like a sticker dot on the floor And later you could take a photo or measure or whatever you want or you just had people physically there and what I do with this one is Go and stand next to someone and they've answered and the next two not like this like why do you answer this? But next to them and you just say why are you standing where you're standing? And they'll answer they'll give a sentence and then the second question I ask is What does it feel like to be here? And they might say they might be way on the outside and they might say fine And so it really shows the teams how much They don't know about how people are really feeling Like it really is great for revealing how much people make up stories And it gives you kind of a snapshot of like of those kind of questions. How aligned are we about this? I have what the resources I need to complete this whatever question that You can ask that It's okay to ask the whole group because there's no place not to stand right unlike the cards like there's You have to stand somewhere All right, so those are my two Those are the two things I wanted to share with you I'm gonna um any questions before I send you in a breakout room to see to just talk about what you How you might use this or other ideas that might come to you about how to collect data in the room All right breakout room time you may or may not be with the same person you were with before Probably not And uh, you'll have a great time So oh look you're gonna be in with the same people you are gonna be with the same people. There you go You should be going now Not a single so we're uh, we're getting Nancy in she's gonna do the next section. They see her eyeballs here um, but Nancy i'm gonna just take some Sharing out of the last breakout rooms while you get oriented So anyone have any things to share out of your conversations or just questions I had something to share that this is melissa. I suddenly have to share I was typing it into the chat and then I got booted into the breakout room. I had a something When you're Standing next to somebody in a constellation or the quadrant mapping or I guess it's the constellations um, I would suggest rather than asking Why are you standing here? A slight modification to the language to say To start with a what statement rather than a why statement, you know, what made you decide to stand here Something like that because why can As as jim and my group said it can Have a sort of a finger pointing kind of a reaction from people Yes, and it's funny almost always I do not have why questions for that exact reason and I have found That maybe it's something about physically. Yes, you can change the question to be what has you standing here And it will it will it will do that. It will neutralize that and I've also found that Some maybe it's something about the fact you're physically standing and they're sort of like a maybe it's my tone of the curiosity that why actually Seems to work for this particular exercise that you're right for best practices You know, what has you standing here? What has You being here Yeah, someone to tell me about your choice Yeah, it's not even a choice, right? It's just like a what what it's truth, you know What what has you been here? um I was excited I use pull everywhere. Oh doug is saying you can use pull everywhere for quadrants. That's awesome Yeah, so you could do this electronically So you could have people and that might be even more efficient, right? And have them physically walk it and then they put the numbers in and then they end up Other than they wouldn't necessarily have the cards to swap around anyway, you can combining these technologies is very exciting Something I've done with groups is what I call a spectrum Which is like what you were talking about, um And I start with just something easy like how long have you been with the organization? So they see that dynamic and which is interesting and can promote conversation and Where are you now with communication if that's the topic and where do you want to be? And what I so what I'm thinking now is doing that and then continuing with something that might be a little more difficult to talk about With the anonymous way and I really like what you're saying erica about them walking the spectrum So that's embodied and then they mark up on their cards. Um, yeah Yeah, I guess this is all a step up from There's just to sort of get to know you type of things, right where you assign to the court You this is another sort of a thing I learned from ain like oh cat versus dog people and you define the You know people line up across a corner across the corners of the room And then the next question you ask like the opposite corners of the room So everyone has to move so if you're asking them like You know, whatever questions you're asking to just warm them up to physically answering questions Yeah, and and to your point about the constellations or that kind of spectrum thing I told them ahead of time I'm only going to ask you these two questions and I'm not going to ask you any more questions And we did it just to like I put a lot of framing around That activity before we did it so people knew that they weren't It wasn't going to go further than that, you know, they knew they were going to be asked to honesty to this point and then Yeah, anyway any other comments Quadrant design might be I love this as a it's a great exercise It's a visceral as you said you get data that you can use To show your stakeholders who are spending money What you're working on and if you put up that quadrant where there's high importance and low confidence What a great driver to do more work Right, yeah, and if you collect these cards and then you graph them you go back to your client with those graphs and then if You know you work with them for six months And then you can redo this And then see and then show like, okay. What are those results? You know what's happened and all of these things are just questions They're all of these are inquiries like okay. What might have changed what didn't change, you know all in the service of of Trying to tease out as carolin could probably give us a carolin or nancy This is actually the segue to Nancy about that curiosity of like what is Let's be curious. What is what's here? What are the biases? What's affecting us? What's affecting the human and how we can use data to To open up conversations of patterns that previously were not visible Nancy your segment. Yeah, okay. Sorry. Yes I think sometimes when people are struggling in any relationship It's hard just to really define what a great future looks like And I imagine probably part of this Is if they're in very low confidence to talk about what does it mean to be in high confidence? What does a team look like so maybe in helping them to find that future might Make it easier for them to think about where they are and where they aren't Yes, I just did a Retreat two days ago and I used images So I have all these images out there and have them Just give them a little talk about like letting go of your thoughts about it Letting go of your left brain and just We we did a visualization exercise about what the future looks like and then having them just go Instinctively pick a photo that matches the sense of how the team would be operating in this future And then they pick a photo of like what's this in silence And then pick a photo of the sense of what this team would be operating is operating about now And then we started putting words to those for a future then there and then I'll see if I can I'll pull up a photo of that was a really powerful exercise another way to like get people sort of into the sense of it Without just being like well, what's it going to be like and what's the number and you know Like how do we get people? I think it's part of what we can do right is have all these activities to have people access a part of their wisdom that's Squirreled away in their bodies and in their images and their intuition And ridges paul wants to talk about ridges. All right, Nancy. I'm gonna stop talking because you're brilliant and we all want to hear from you No, um, hello. Well, it's nice to see some familiar all of your names are familiar And uh, and it's really nice to be here. My apologies for being a little late What better word what better sentence to segue into a discussion with fellow ainors than access a different part of our wisdom And you know part of why data collection is important to me as I've shared with erica and car and carolin and others is to Not only elevate the status of our of our profession and give us more validity as well as more business and Help us greatly, but it gives us a an opportunity to reflect not just who's in the room But who how we facilitate how we understand that data So I've been tasked with um, a couple things here in the next few minutes I'm going to talk to you about two different studies that uh, I've been privileged to be a part of One is the social anxiety Lowering study with one of canada's largest research-based hospitals and universities and that's been going on since 2016 and then a recent one from 2019 With a local workforce employment board and we have used direct applied improvisation to get people back to work So both qualitative and quantitative uh measures with those two projects The first one Is the youth wellness center at st. Joseph's health care and uh, as I say since 2016 We have been uh doing a 12 week improv bootcamp to Uh, it's a bit of exposure therapy and we have uh with our group inner city youth that are tasked with The inclusion and the communication and the collaboration of building ensemble. That is what we do, which is code for resiliency and by making by yes ending by employing the the science of neuroplasticity and changing their mindset and Forcing them to focus on the other make your partner look good and being deeply and playfully mindful in the moment in an improv exercise These individuals who are afflicted with anxiety disorder, and I don't know if you're familiar with it But it it can be truly debilitating the uh, it gives them the opportunity to By focusing on the other not having the time energy or inclination to worry about their Self-centered selves and I say that with love and kindness, you know deeply the anxiety is um As you know, we talk about building not just a safe place But a brave place and that is and as keith johnstone said, you know with yes We are rewarded with the adventure that we will take with no we are rewarded with the security we retain And it is in that uh retaining the security that solidifies and tells these individuals who are afflicted with anxiety Disorder that they're safe for if they don't say for don't Improv is a wonderful tool to counteract those cognitive Distortions and look at how the stories that they're telling themselves in their brain may or may not be true I am happy to share our research protocol and the literacy literature search and the academic research that's been done Through this study and i'm happy to uh send it out and share it with you In this study, we had both qualitative and quantitative measures to it The quantitative measures is a validated score called gadd generalized anxiety disorder as well as the self-esteem score those were the quantitative measures pre and post the qualitative measures that we took were a lot of personal interviews and and reflections and uh and shareings that the kids provided us for that Right now. It's a conference paper. It's not published yet, but you know knock on wood and hopefully, you know, we get to present at some health care health care conferences around it and it is um and so What a pleasure Okay next uh Next study that we uh that I did started in sort of started back in the in the summer and uh in a nutshell it is Using applied improvisation to get people back to work um a local city uh about an hour and a half away from where I live is london ontario it is uh a city of around uh 500 000 people and they have identified uh I believe 14 000 people who are who are out of work, but Identify themselves as wanting to work and people are unemployed for a lot of different reasons Whether they are waiting for recall for the factory or mental illness or family or whatever reason, you know downsize So they are not in the canadian welfare system. They are identified as wanting to work and not working for a host of reasons the um The rfp came across my desk through another consulting company and they were asking for a completely new and innovative way to help people leverage their uh their journey back into employability because they have been Uh in the workforce world in that social agency world They have been uh tasked with resume writing and interview skills workshops to death and they do not work So we put together um a a six-week applied improv boot camp and uh and it was based on the research that we did around uh Two things twofold what the employers are asking for in employees today, which as many of you know are things like uh communication skills collaboration skills empathy skills, you know that type of uh adaptability all of that and um I don't have to tell you how well improv is to Cultivate those skills So that's so we gave them the skills that the employers are saying they wanted at the same time That we are building in them some of the uh some of the qualities that they are lacking that are making them discouraged to finding work and uh and those are resiliency and and uh risk taking and uh facing those cognitive distortions that again are Leaving them defeated in this in this process of looking for work. And uh, so it was a pilot project We had um and we had hard metrics for after six weeks of putting them through this Those metrics were one of three things at the end of six weeks We needed 85 of them to have met one of the three criteria The first one was that they are gainfully employed that they are back in the workforce Or two that they are back in school college some sort of retraining they have uh through this experience of Creativity and improv perhaps they have identified something else that they wanted to do and and uh, you know Go back to college for chef or whatever or three interestingly they have um through this process identified and issue or uh a challenge where they need a professional counseling or help for that and um, I can tell you that those metrics were met 60 of them are gainfully employed. We have um, a bunch that are in in retraining and um, and uh, the rest are uh, have identified some issue Interestingly at the same time that we were delivering these skills to the unemployed It was also a scalability study and we were training five facilitators so that they could leverage it forward And continue to use it in their respective social agencies. So we had the Nancy i'm gonna pause you right there because you just hit a great point And we have like 15 minutes left of the webinar You had a great point because you actually even though you weren't on this webinar You brought us back to that original this original thing we talked about. Yes, this is low tech guys But that gap right so So uh that gap of I don't know. Could you see that of of what were those original client goals and she defined it 85% of the participants will be one of these three things So there was that concrete. This is what it matters in the eyes of the client So I think it's a great time to send people back into the breakout rooms for your last five minutes to Talk with each other unless Nancy. There's another jewel. You want them to take you? Well, I was gonna do I I started it 10 I started it like five after I had like a couple minutes to do the anxiety study the employability Study and then some of the bias that comes in when study But if you want to do breakouts and come back we can totally talk about that after eric Whatever you want. Let's let's do Discuss a little bias which might be relevant as well. Okay to the breakout rooms. I'll stop. I'll I'll uh I'll cut short the employability study and give you an acronym friends chore C-h-o-r-e the following five are the most common research biases that come into The type of study that I've done and many others do Um in in essence, it's it stands for cultural bias The halo effect I did this. No, it's hang on We did The order of our questions in the qualitative the the order in which we ask questions The researcher bias also known as confirmation bias and the elaboration of Of the questions that we ask as well because when we Much like I think it was melissa who talked about um, you know, why are you standing in this space as opposed to Um, you know, uh, tell me more about your choice or I wonder, you know, coming from a coming from a place of curiosity and positivity opens up a conversation in in a much more true and authentic way and Very often facilitation does that is a traditional facilitation is supposed to lead and direct and apply to improvisation as you know is That type of traditional facilitation can really lead to that type of bias So the cultural bias to go back to our acronym cultural bias is how we Understand and we look at things through our filter of our Of our own bias the halo effect is that which we have when you know, we view someone because of one particular positive trait we view them to be more knowledgeable and Um, and we receive them better because of because of that Um, the order of our questions in the in the questionnaire that you're giving your workshops careful how that becomes um, uh leading, you know And uh confirmation bias researcher bias is that when we have a hypothesis We look for evidence to make it so and oh my god. Am I ever guilty of that? But that is just the truth and then the and then the elaboration of our questioning, you know, uh, As I said before how watch how our questioning can Put words in people's mouths and we don't want you know, we truly need uh, who they are truthfully coming So sure and I'll send that out to Over to you air That is awesome. Isn't this exciting? Questions being it's so exciting because you know, you get just sends us into more, um Distinction about our thinking and inquiry and curiosity I know that this is not enough We are going to send you in a breakout room to maximize your chance to talk to each other And we'll be back with Nancy and carolin in five minutes There you go Okay Everybody Welcome back all Yeah, so we did have a question in here like what if what if we want to ask leading questions? And I think the main thing is we definitely want to be aware if we are asking leading questions And ideally I think we do want to be mindful and careful about that because it's like the quintessential leading Question in my line of work is the professor asking a lecture room full of students Do you understand and all of the students going? Right when they don't so it like if we do want to again not fool ourselves I think it is important to be if we were for if we're doing it with design. That's one thing but What it's very very easy to ask a leading question Yes To reinforce confirmation bias Yeah, the equivalent in my world might be this happened. There's a leadership team. I had people answer questions I not you know on online form I present the results It's been two months later. I it took two months to get the meeting I present the results and the leader is like, oh, wow, you know, these are from like two months ago I think everything's so much better now I I can understand why they used to look like that but everything's you know now things are really good Total silence in the room So I'm like, you know, would you be interested in finding out? And they're like and I like take out a piece of paper number your paper one to ten We asked the same questions right there in the room Then I took those questions home repotted the graph. They hadn't moved at all, you know So that was like, how do I you know? That was a way to like use data to reveal the system to the person if they're in power They're going to get biased answers not a single person spoke up But then I brought the data back so So interesting Any other uh client come we have a couple minutes. Yes. Yeah, so so Practicing what we preach we had I put a feedback form into the chat room If you please fill that out very quickly, you know, you can pick and choose the questions That would be great. So we can get some more feedback. This has really been a great start I feel to a conversation. There's it is indeed a start. What more do you want? We could get into so Nancy's gonna put in to put in uh put together Summary of per second study and add it to a shared google drive that we will send out It's a little bit proprietary I'll get the permission from a client and I'll I'll send it out in generic form But if you want to have a conversation with me about it, um, feel free to reach out and I can uh, and we can Do so in more depth Yes, I'll share the slide here real quick of the other activity. I was describing We're using images And then putting words to images and then people individually giving actions and then dot voting about priorities and themes So there's that I want to just give you a quick snapshot What else I look forward to watching the breakout rooms the zoom account these breakout rooms It's a free part of anyone's zoom account. So we can continue using breakout room Yeah, thank you so much. This was really helpful all of it Thank you for we have two whole minutes. Do you want to tell us what was helpful, davin Please Oh, just thinking about as sedrick and I were talking the breakout room Um, just the importance of like one of the things he talked about is how do we get compliance from our client? and the meta thing for me today is I can't wait to talk with a specific client that I talked with him about of What more another conversation checking in on hopes and dreams and then what are what are outcomes? And then how can we measure that and working with her so that there's more chance of a compliance if she helped You know do that with me Yeah That's I have a note from So it's yeah, it's it's us walking our talk of co-creation right and using our skills Um, I'm sending it back to paul to close us Yeah Thank you, erica. Thank you very much carolin and latterly nancy And thanks to everybody for their contributions and the chats the breakout rooms and speaking to the group the recording Will stop in a couple of seconds and then we can say what we really think