 Selena said I'm Joe Hastings. I work next door. It's a great pleasure to introduce Jamie. I've worked with Jamie since 1993. Our orbits have kind of crossed here and there since then. Jamie Bell began his career in formal science at the Exploratorium in 1985. He began working there as an explainer and by 1987 and through 1995 he ran the high school explainer program. And that branched off into an UCLA which went nationwide in terms of a youth development program in science centers. After completing a master's degree in education at Harvard Graduate School in 1996, he joined the Exploratorium to manage several projects including facilitating the framework and the exhibit and program development project based on California state science standards. And then also the conception and development of two exhibit projects. One of them is AAP, Achieving Active Prolonged Engagement, AAPB, Exploratorium, great with the names. And those are some great exhibits by the way. Some of the best hands-on exhibits you'll ever find. If you look to that book, Active Prolonged Engagement, it's a good one. And then also listening, making sense of sound. Jamie has also worked as a consultant in the Boston area as the content developer for the Essential Science for Teachers, Physical Science, Professional Development Video Series with the Harvard, Smithsonian Center for Astrophysics Science Media Group. Get all that out. But also a little easier to say, the Math, Lumen, and Science Center project at Turk. In 2006, Jamie was invited to Malaysia to develop and establish a center of learning at Petro Science, their National Science Center there in Kuala Lumpur. After three years in Malaysia, Jamie returned to the US to spend a year as a visiting scholar at the University of Pittsburgh Center for Learning in Out-of-School Environments, up close, working with Kevin Crowley and others. In 2010, he accepted his current position as the project director and principal investigator of CASE, the Center for the Advancement of Informal Science Education at the Association of Science Technology Centers in DC. Jamie is a true lifelong learner and educator, as informal as they get as an educator. And he lives the philosophy that the best answer is often the question. He is an inveterate partner, collaborator, and improver of his organizations and the organizations he connects with. And I'm thrilled to introduce Jamie Bell. Well, thank you so much for the opportunity to be here. Today, I'm going to time myself so that I allow enough time for some dialogue if you will. I work really today and I learned some things I'm sure will influence the presentation a bit, so I hope I'm not too redundant with the things I said upstairs. But I'm here as the project director and PI principal investigator of a project called CASE, which is an acronym for Center for Advancement of Informal Science Education. And it's an NSF-funded project that's based at, and speaking of long sentences, at the Association of Science and Technology Centers, which is the professional association for science museums and centers worldwide. It's an international professional association based in DC, and put it on a conference every year. We made one of the conferences here a couple of years ago. I know I did. There were some wonderful hosts here who hosted that museum. These two, and some guy who packed my bag to Texas. Anyway, but so in that role, we were housed at Aztec, this project, but we worked 100% of our time, when I say we must have, on this project called CASE. So we're not involved in putting on the conference. We're not involved in the Aztec website, for example. We work with them very closely, but I just have to make that distinction because people say, well, what's your relationship to Aztec? And we have a relationship, but we attend a conference like many of you do, actually as a client to some degree. We actually have a booth there. It's a CASE booth. It's not the Aztec booth. You may have seen that. So that's the relationship, and it's a very wonderful synergistic kind of relationship because, of course, the science centers and museums are the anchoring kind of sector for this field of informal science education field. So it makes sense for us to be there. And it also makes sense for us to be there because the grant was written there eight years ago, and that's the fiscal agent for the grant. I also work with a few co-PIs around, co-Principal, and that goes around the country at University of Oregon, Oregon State University, I should say. That would be a bad mistake. Oregon State University, John Falk. Kirsten Allen Bowman, who's now the CEO of the Great Life Science Center. Kevin Crowley, my co-PI at the University of Pittsburgh, where as Joe mentioned, I spent a year prior to coming here. And Sue Ellman-Cann, who's the executive producer at KQED Public Media in charge of the Quest programming there, the Quest Network, which is also an NSF-funded project. And you see up there, there's three staff and myself, and so we're responsible for the whole project in terms of staff. So we're called the Center for Advancement of Informal Science Education. So what does it mean to advance this field? So I always steal this line from one of our colleagues, I think colleagues, several people here as well, Dennis Schatz from the Pacific Science Center, who was at NSF for a few years recently and just went back. He said, you know, to advance this field, what we need is data, we need stories, but more important than that, we need relationships. And I really agree with those three components being what we need as a field to sort of advance ourselves, because people don't know, sometimes I understand what this field is. Joe joked about me being an informal educator and one of my program officers at NSF always says, you know, the informal is in front of the wrong word. It should be in front of education, not in front of science, because people always go, especially if they're a scientist, research scientist has to say, a natural or physical scientist, they will say, well, what's informal about science, right? I mean, that's kind of insulting, right? So, but I think, you know, the idea that it happens anywhere, anytime, as Joe said, life long, life wide, life deep, to quote the National Academy's recent document. But advancing really does require that we do have data about what's going on out there, what people are doing and how they're being impacted by experiences from places like this museum and the one across the street and many others who are represented in this room and other providers of these type of experiences. But we need stories too, because stories are very important and we need relationships. So all three of those things I'll try to weave into this presentation a little bit. And the one thing that I particularly need that may not be so much in the presentation are also images. Because images also help people. You see those photos of, like, on the first slide, people engaged and varied, obviously, in a pleasurable way in the experiences with STEM topics and natural phenomena. So, I'll go into the presentation here a little bit. So when the project first started, my co-pian John Faulk did this sort of landscape study in the field to kind of figure out, he interviewed a couple of hundred people that he fought, he and his graduate students at the time, thought were sample, a good representation of the field of people who were doing this work that was back in 2006 when they started it. This came out in 2007, 2008. And even some of what we call these different sectors now has changed over time. But the whole point of this was to kind of develop this grid that would show, depending on which part of informal science education you're working in, the degree to which your goal was STEM understanding, the actual understanding of content and process, and the degree to which you identified as an informal science educator. So it's kind of interesting at that time that people at the Natural History Museum, Science Center, and the Aquarium very much identified that way, but people at other parts of what we would also consider the field then through this survey anyway, the sample, identified in different ways as to their roles. And so I just give you that as an image of sort of where we started the field. And one thing that's certainly true today that I'm sure you're aware of is that the number of different types of settings in which informal STEM learning happens now is expanded just exponentially. It's amazing. And I'll talk about some of those things. So case in terms of a visual to how we position ourselves and think about ourselves. So again, we're funded by the National Science Foundation advancing informal STEM learning program, which is out there in the center. And the types of sectors, and I, you know, use that word with a little bit of trepidation because people sometimes refer to the whole informal world as a sector and the formal world as a sector, and that's fine. We just use it as a convenience within the project to think about the different areas in which informal STEM education takes place. And, you know, that happens in citizen science projects. It happens in museums and science centers. It also happens in film. It happens in after school. Our roles as we see that as a resource center for the field, are to, we seem to like C's and a lot of people I think do like C's for some reason, convene, connect, characterize, and communicate. So we bring together groups of people who are doing this work who are professionals, typically from the different sectors. We try to have a lot of sector representation. So film producers, researchers, evaluators. We try to make connections between what people are doing. And then we, as a result of those kinds of events and gatherings, characterize what we think is happening in the field with regard to a topic. So if it's sustainability science, for example, it's an area that we've had convenings on. If it's an actual topic area, like nanoscience, for example. You're probably all familiar with the nanoscale and formal science education network. Or if it's a way of working like professional development. So there's a lot of projects funded by NSF that are the purpose of which is to do professional development for people in the field. And then to communicate those via our various channels, the website, our newsletter, and other communication channels. There's, I'm going to spend a bunch of time today on the website and the repository, which is one quarter of our initiatives, the web infrastructure, there's a site called Informalscience.org. But we're also involved in evaluation capacity and practice and research connections. And that has to do with practice in informal science education and research on that. So that could include a number of different social science research areas, like learning sciences, for example, and broader impacts in informal science education connections. So those are typically scientists, again, research scientists who are working to develop broader impacts plans, activities, or they're working with their staff, or directors of education and outreach to develop activities to address broader impacts goals for their research and their proposals. So as I said, the website is the key repository for us, the key tool that we do all this through. And there's one thing I'm not going to do during the talk is to actually go in and out of the site because that would just take up too much time to distract you all. However, while I'm talking, if you want to look at it on your phone, your iPad or your computer, or you want to tweet or anything like that, that's fine. I'm used to that. You can, you know, you can have it kind of like this and that's totally fine. Because that's the way things are in the world. Which is actually a good thing, I think, for all of us as long as there's some communication going on. But I say that because if you want to check out the site, you can always talk to me afterwards about specific questions or write to me. And also the network I've been with earlier got more depth breathing on the site. But we're always happy to do that on a webinar basis, on a one-on-one basis. So there's three types of projects, three types of resources on the site. Project descriptions, research and evaluations. So there's descriptions of a whole bunch of different kinds of projects happening in different settings. As I mentioned in all those sectors, there's research on those kinds of projects. There's foundational research that is informed and designed in those projects as well as the documentation of them. And there's evaluation reports, everything from funding to formative to summative to remedial evaluations of those projects. And there's, right now, there's about 10,000 records on the site that span over the last 30 so years of work in this field. We don't have everything in the world that's there, but what is there is pretty rich and it's added to every day. So that we have additional library and curates and catalogs, materials every day. We rely on the community, people like yourselves to submit things and tell us about things we don't have. So you play a big role in helping us strengthen this resource. There are also news posts, blog posts. The news typically has to do with upcoming funding opportunities, whether they be NSF or other federal funds as well as private foundations. And as well as professional development workshops in different museums or science communication courses in the UK or anything that comes to our attention that we think might benefit the field of people working in this field. And the newsletter comes out once a month. If you join the site, here comes my big ask from the top, please join the site. All you have to do is go to Informalscience.org. You can do it right now. And join as a member only for a couple minutes. If you want to fill out your profile, fine, you can do that over time. But once you become a member of the site, you automatically get the newsletter and you automatically have access to resources there that you wouldn't have. There's a couple of things you wouldn't have access to if you're not a member. It doesn't cost anything. That's brought to you by the National Science Fund. You can also, I won't go into much on this today, but it's possible to start a group on the site. And that means a discussion group of some kind of forum. And I can talk about that a little bit later on. Period. So the three biggest new things on the site that I should say is right up front. I mentioned these upstairs. I didn't mention one of them. But I thought that would be more appropriate to join the group. The evaluation landing page, we recently completely redid our evaluation landing page. We click on evaluation on the home page. All our evaluation resources are organized, I think, in a very accessible, understandable way. That includes reports, but it also includes directories, handbooks, different kinds of tools, instruments, organizations that support and strengthen the formal science education evaluation. That's another thing. The outreach for sciences page is on the home page. Also, you click there, and you can get into all the records in the repository through disciplinary categories. So sometimes scientists go, you know, I want to work with informal, but they go to the site and they go, how do I find something that's related to, you know, I'm a material scientist. I need to find out what's been done in this field, what's been successfully done for primary school students or something like that. And by the outreach, using the outreach for sciences landing page, you can get directly into your own content area first, and then start to search for different audiences, different approaches, different strategies. The research agendas landing page has to do with some research agendas that are happening in the field of informal STEM education. That is to say groups who are trying to develop big questions about what we need to know to do this work better for grand challenges for the field of children's museums. Or for zoos, or for natural history museums. There are people working in these sectors trying to identify questions that if we knew the answer to, we could all do better work and our audiences would benefit from. The case perspectives blog, every week on the site there is a new blog piece that tries to be, we do our best to be current with current things happening in the field. For example, I mentioned upstairs a couple weeks ago, Triple AS in San Jose, the citizen science association, which is now a formal professional association had their first conference. It was a wonderful conference. They were overrun by members. They had 3,000 members online as soon as they established themselves. They had expected 300 people at the conference. They had 660 people at the conference. There was a lot of energy there and we did three blog posts on it because we thought it was important to have a lot of sort of currency. But every week you'll see a blog post on something, doing the field, and those blog posts will have links to records in our repository that we think are useful. That's a little bit what the evaluation page looks like, those categories and resources I mentioned at the top. There's also a guide to working, using evaluation and working with evaluators in informal stratification arranged by chapters here. You can just click on it and go right to it. This is a book that Case wrote with in collaboration with the Visitor Studies Association. In collaboration, I should say. And if you want to just find out about the key elements of evaluation, this is more for people getting started, understanding theories of change, different aspects of developing the evaluation plan, proposal, and activities. The research resources I mentioned include peer review articles, connections to the research agenda that I mentioned earlier, and we have this thing called the Evidence Weekly. I didn't mention this upstairs. But under the research category, if you click on ISE Evidence Weekly, there are a number of short articles that are framed as evidence statements for what we know from evidence, whether it be evaluation evidence or research evidence, about how people learn in these different informal settings. And that's intended for, again, a brief, easy way to find some statements that might support some proposal you're writing or to allow you to look up what's been done before around that particular area. So say it's girls in technology or it's field trips. And field trips, by the way, is one of the most popular evidence-winky articles we have. There's one article there about your field trips. Isn't that a surprise? The value of field trips is one of the most popularly downloaded articles. I wonder why that is. I guess people have to look for reasons to justify field trips these days. Or maybe it's like reading a book. So that's that. So again, please join the site. When you join the site, you can join an existing group. And I'm going to say something about that today. This is also a litigation. We do forums in the group section of the group of the site. And today we started a forum. And it's called on shareable measures in informal scientification. And to do that, you need to go into the groups and you need to go into KC Evaluation Initiative Forum group and you can ask to join. We'll immediately say yes. And you can join this discussion which is about measures that are being shared across projects or actually could be better shared in the future across projects that are studying or evaluating the impact of informal science education experiences. So if you're in your project in the program where you exhibit you are claiming that people are developing a stronger identity as someone who understands science or can use science or you say that you're catalyzing motivation or curiosity. Those are all things that people are working on in these pockets all around the country and they're using sometimes the same words to refer to the same construct but sometimes using different words. So this forum this week is an attempt to try to get people on the same page about well what do we mean by this identity? Motivation, perseverance, self-efficacy all these things will be claimed that we are evoking of people who are part of our experiences. So that's a week on forum I invite you to join it and if you have any trouble getting on, send me an email. As I said you can search all these resources and you can find potential collaborators once you join as a member you can look at all the members search all the members and if they have a profile you can tell how they do it and where. Okay I'm going to talk a little bit about networks because the informal science education network group the New Mexico informal science education network invited me here and we have done some stuff as the same case as done some work around networks but I just want to give you a real brief few comments about my own experience with networks in this field Joe mentioned youth a lot this is a network that exists still in regional areas around the country but it was a pretty amazing 10 year a network that was funded by DeWitt-Wall's news I just found in the 90's that led to lots of professional development for people like myself and at the time Joe said I was running the high school explainer program and for me professional development wise this was a key vehicle to connections in the field to greater knowledge, to skills in working with youth and it was basically there was a lot of discussion about how many museums were actually ultimately in it because there are still some in the regional networks but region was about 72 exhibits and they were programs and they were organizers leadership programs there were expansion programs and technical assistance programs and all of this was developed by the association of science and technology centers in cooperation with DeWitt-Wall and so there were nine museums that had been identified as being really strong in serving youth especially adolescents 10 to 17 in the country that served as models, different models because the exploratory one and the New York Hall of Science one and the one in Evanston, Illinois all had very different models but very successful in their own way Museum of Science Boston was one also actually provided help to people who were trying to expand programs that had been started but you know needed some ideas about how to get more youth involved and how to serve them in a more quality way and then there were also a third tier which was called technical assistance which was people who really wanted to start a youth program that didn't have one yet and wanted to know what was out there this was a really key initiative so I think in the field it has had so many ripples since then tens of thousands of youth as direct participants and that doesn't mean coming into the museum for a one-time visit and leaving and they were in programs where they worked as explainers or they could volunteer activities, led lots of activities they're putting very much in the role of a leader and a teacher in some cases facilitator of experiences but then there's hundreds of people at the leadership level where they were program leaders managers developers those kinds of folks who have gone on to do all kinds of things in the world including one of them standing up here and directors of museums who were part of the leadership of the program at that time and the nice thing about it too is that there's been some documentation and a retrospective look at it through the lens of Portland State in Oregon Kerry Snyder who you may know from used to be at Lawrence Hall of Science for many years had wrote a paper with May Burke that sort of talks about the legacy of it what it did, what the initiative did and sort of where people are now so that's an example of the type of resource that you'll find on the site that we hope to is a rich one whenever you're thinking of a new idea so the object that the network shared and I use the word object in a very general way in this way actually to say that what was in common among all the projects was they were working with youth working with youth 10 to 17 and they were typically targeting underrepresented youth in museums and science centers and that became of course that's a fascinating topic it's a very challenging topic but that was the content of the network when we met twice a year for professional activity we were mostly talking about how to best work with that age group to get them engaged in science and STEM but in science and in engagement with the public in science and basically in their interest to stimulate their own interest so that common sort of boundary object where it became this endlessly fascinating topic that could look at from a number of different perspectives and over the years each year we had meetings a year one at the Aztec Congress and then one in the spring where we had lots of professional development opportunities many people in the field now are guest speakers like Eric Jolly who's director of the Science Museum of Minnesota and lots of people came through that network and are doing interesting things another couple of networks that I've been one particular I've been involved with was on XNet at the Exploratorium where I worked with Joe and others which was a very different type of network one institution running at one science center basically the hub for it but interestingly in this case the content is the exhibits so the exhibitions that travel around and Joe and I were just talking for the talk about how many was the upper limit we had at one point so we had international members who were about a dozen sometimes give or take received exhibits on a rotating basis throughout the year I think every four months or six months you've got a new set and with that you've got also professional development so facilitation on the floor you know how to work with teachers how to work with various audiences from the perspective of the exploratorium it was really interesting to see the kinds of things that happen in a network like that when people are working with the same exhibits but at different times different ideas about how to use them and also in some cases how to extend so innovation happened often in terms of how people used the exhibits and what they learned from that and what they shared another very different model traveling exhibits at New Zealand Science Teams that was an NSF funded project where a group of exhibits came together a group of museums came together to develop exhibits together and actually exhibits and traveled them around to the museums in the network about eight or so and NSF funded that actually three times to get started initially and then to further develop the model and to do some research on it we're going to move a little more briefly now so in terms of case with networks with networks was one area I mentioned we convene so when we looked at the portfolio back in 2011 and we said what are all these projects that NSF funded what's some common themes here a lot of people have networks or we're trying to form networks so what is it about that it's interesting what can be learned from it so we went kind of through this process the first meeting people in 2011 then the follow up PI meeting that year that discussion continued some of the networks are listed up there that we brought together and I've discussed a few I mentioned before and out of that of all the convenings we've had this one so far has had the most legs so I don't know what that tells you about the interest in networks is not waning I guess but people continue to propose the topic at aspect sessions and pre-conference workshops up until this past year on this topic and we try to continue facilitating that discussion through the website as well as an evidence wiki article that kind of summarizes a lot of what we learned through the project now those wiki articles are submitted from the community so it actually wasn't one of us who wrote it but it was someone who's an evaluator based in Chicago who wrote this article for the wiki to kind of continue building this knowledge based on what we know about networks the typical kind of things I'm going to go through these parts a little bit faster simply because being upstairs and seeing what's going on with this network I realize you guys are probably processing a lot of this kind of stuff so I don't want to go into a lot of network theory and stuff like that but there are things when you bring it together with the people who are doing this and we have about 10 at the beginning you start to find out that there's this common issue so what is the definition of a network housing different companies other things like consortia, partnership alliance, collaboration of course there's a lot of overlap with some of them what type of a network is it intentional, intentionally formed or organic, cultivated or cobbled to remove what existing people do the same thing is it very top down, bottom up is there a central hub or distributed leadership so on so all those kind of things we're talking about and kind of embedded there one of our colleagues who some of you know, Sam Dean the director of the Macy in Arkansas Arkansas he came to the convenience you know the thing I learned here is time, trust and togetherness and it seemed to kind of sum up what what people learned and he's actually speaking about text which I'll mention in a moment as an example of a network that they spend some time together all these kinds of issues the purpose of a network the sort of shared purpose is it knowledge building is it about leadership is it transforming a field is it professional development is our evaluator, Mark St. Jones is getting better and getting better which he actually stole from somebody but it's a very good point and a good reason for having a network innovation it's not a replacement for innovation it's not a guarantee of innovation to have a network but innovation often happens in a network as I mentioned in the X-Net you have these exhibits going around and people will get an exhibit and they will think of something different to do to what nobody else ever did before and sometimes that innovation spreads out through the network scalability is kind of these days in education it's all about can you scale it we all know that the best things happen globally I mean that's where it starts it starts globally and so there's this impetus you want to make now we're going to scale it and that's not an easy deal sometimes so somebody it's more spiral building you know to work and finally maybe get up to the level that makes sense sustainability I was at science and learning centers which are resource centers but very similar to networks funded by NSF but from a different division the social behavioral and economics they met in DC a couple weeks ago because they're 10 years funded and they're all kind of going what are we going to do now I mean I have no cause to extension but how do I keep going maybe we should create a little website actually some really good work is going on and it's documented on their individual websites but somebody said well it takes 10 years to do that I think it does at least this network stuff maybe you can learn it faster innovation so yeah you were supposed to be online but you also have to bring in new people there was some talk about this upstairs about you have to seek out the outliers that's what's going to bring a new idea it might not be comfortable are they doing it definitely I'm not sure there's an accelerated feedback loop for ideas in the network because you could just bounce it off the members if everybody's responsive you can try out ideas and get things adopted quickly so there's this idea that sometimes innovation happens in pockets and you may not be aware of it whoever's the sort of coordinator leader or hub needs to be aware of where the innovation is happening and make sure that it's communicated out to others and of course the innovation manifests in products as well processes the niznet folks how many people have been part of the niznet they I'll talk about where they are in a second but they developed all this stuff these great kits and nanodays and guides for working with scientists and scientists working with educators and all sorts of fantastic they spent a lot of time on these activities to make sure that they were good and tried and proved true and then they get out of the world people do them differently than what they intended they combine what they run in niznet with the CSI exhibit that they have before work and there are stories like that and actually have these wonderful things so sometimes it's not the product it's actually the process people go through to develop learning opportunities this is the last one taking network management seriously we were talking about this upstairs you must have someone who takes it seriously maybe it's not their full time job but when they're on the job they've got to really be overseeing everybody as much as possible and making sure everybody's in a ratio in the network though there's providers consumers at various times and sometimes someone is a consumer for a long time steps up one day and guess what they're a provider it just sometimes takes a while to figure out what your role is what you have to offer in the field of this but to hear people say it from a number of different and as that front of the networks was interesting there's innovation but there's also integrity so you do want some integrity about what you're doing and not be saying we've formed this network to innovate so where's the innovations it's like Joe and I used to work with the physicists for a tournament recently and retired with Thomas Humphrey and he used to always say I don't know what else to say but the point about that is innovation is wonderful but catalyzing it is tricky and there's an integrity of quality that's just as important if not more so it was that tension came up in the discussions at that meeting fidelity, brand for some of the networks there like Quest came to be it was really important to them and they did be branded as Quest it had to be branded as Quest that's maybe a function of the fact that it's a media outlet and that's a pressure from their organization they have seven other media outlets around the US that work in the same way where they're in a locale and they have like 16 partners in San Francisco Bay Area there are either science centers labs, other providers or actions in case of student scientific research but have a big outreach component so branding is important in some networks some others not and then there's this idea of things disappearing or the need the need to do the network was meant to the need that the network was meant to meet that was hard to say that need disappears and then what do you do? you can reach other friends working pretty good sometimes the right thing to do is to close down I'll talk about that in a second there are not organizations on the cheap I like that one of them there are still things to say networks so a couple of scholars really influenced us we gave out lots of readings for the convenings the most popular ones were by Louis Gomez from UCLA and Rebecca who's at UMass Amherst whose name used to be Gajda so you see a lot of stuff published by her under the name Gajda these were the most popular and useful so we heard from our participants pieces and actually when Aztec was here two years ago at the was it a pre-conference networks workshop Louis Gomez spoke there we actually invited him and sponsored him to come and his idea is getting ideas into action he talks a lot about network improvement communities and it's a very rich body of literature I want to go into details about it now but the idea is that we all have lots of good ideas but getting them into actions is really hard and network improvement communities are a resource for doing that but it involves lots of understanding of how things work level A, B, C work level A is what happens in the museum on the floor with visitors level B is what happens with the museums that have everybody who's doing that be better at what they're doing and what we'll see is what you do with your other museum and other parts across the street and then the idea about fidelity comes up variation, I like the statement is a resource to be understood it's something to be studied and learned from not just know that variation is not going to work here so that's why I talked to Louis this is his and he works with Anthony Bright and Alicia Brunel and this is their planned study act we've never seen any difference and then Rebecca talks about looking at your old materials about these levels of integration cooperation, oh yeah we're all cooperating and then you start actually coordinating your activities well you guys are going to do this next week so we're going to do that the following week but then you start with I got an idea let's do this together I noticed that in yours there's another level where it's called coalition and that's very interesting there's lots of variations on these theories maybe we can talk about what the coalition means in the QA so I really recommend her work Rebecca Woodland or Gajna she talks a lot about the intense and shared purpose of the network that's her little wheel so a few network have a network convention just to where they are on their trajectory so XNet was a successful model with that model where the sports were leading something to get in and you just started TexNet so NSF funded a smaller network that was led by the Exploratorium and one of the Texas Exploratory XNet members to be like sort of two hubs for a smaller group interstate Texas network lots of lessons learned exhibits that live on, program ideas still going I was in Texas last week the director of the Loretta Museum was there talked to her a bit there's lots of rich material that came out of it the network itself doesn't exist anymore it wasn't meant out of funding but it's one that has a nice evaluation report that you can find on the site this is one that recently sunsets they would know about coalition for science after school so they you know existed for about 10 years as well and just like you know they put a lot of effort into a lot of events a lot of resources, a lot of connections the connectivity as I mentioned the time work it was funded by time work and they decided you know what, there's a lot of people doing this work now and they're doing it pretty well taking different parts of the landscape and running with it so we're going to sunset so they actually had a passing the torch summit does anybody read it? anyway we're done, we're going to have a meeting basically saying thank you go forth, do good stuff yourself and they were also very smart to like make sure that their resources live somewhere so we have all their history stuff on InformalScience.org and so it's interesting to go to that page and read about what they learned and you know the process and where some of the previous players what they're doing now I want to mention one thing there's a lot of information in the actual slides because in all the notes everything I refer to is the URL there's a whole list of URLs in the notes so you'll have them there, I just don't want to have the clutter open screen, portal to public who's a member of portal to public great project, right work with scientists in your place network annual for how to do it growing network expanding beyond science centers lots of good documentation nice then I mentioned they're currently thinking about what they're doing next they're going to have a big meeting in May the idea that this network was formed around the topic here is pretty pretty unique in our field ten years on nanoscience pretty basic, lots of amazing materials lots of connections 500 plus members different levels great reports, great evaluations great approach to evaluation you can't afford an evaluator what they call it team-based inquiry team-based inquiry is a great way to evaluate a program or project you can't afford an external evaluator science vessel alliance growing network really expanding they have a ipset conference don't ask me that it's international public events science something called science vessel also includes cafes all kinds of science events this thing they grew in ways they didn't expect they grew they tended to have six in three years and 36 members in two years not all of them are the same level where they're actually funded but there are members and members that are being mentored by members of the science vessel anywhere in the world and you can get help doing it living lab this is a great model NSF really it's an interesting model where researchers in early childhood learning do experiments on the floor your family comes in and your child goes over to explore learning about a particular object material spatial reasoning there's all kinds of experiments where you can actually receive it for sure if you're a parent you can go home and observe your child doing all supported by the museum staff they have partners around the country they're doing in other cities like Baltimore do you have where's that you're difficult this is in the middle two things I want to slide something to say about network if you only have because I was joking upstairs everybody has tons of time to look at lots of websites and read books right we don't have time to do that so because you do there's two evaluation reports one is this one navigating the future of aboriginal science this is a five year project where SRI networks or what they were hoping would be networks in aboriginal programs in California they found some really interesting things lots of missed opportunities pockets of strength but some really interesting issues about how people thought about partnership or didn't even think about partnership there's an after school program where engineers in a water facility with educators they never thought we could do this we could invite them over things like that it's very struggling in that way to read about what can happen even though there's a lot of rich resources going on whoops I'm supposed to be the one in there oh I'm going to say one more thing about this the reason I had this one in here too was because this is the other evaluation in terms of being informative about networks many of the networks I mentioned have evaluations on the aboriginal science network website but because a lot of them aren't finished yet they don't have summative evaluations but this one has a nice some data that's being gathered and it's just really overwhelming to try to make sense of 10,000 records on the website especially as I said for the time I don't have so I mentioned to Selena that I brought the form of a website on my computer and I handed out the top 10 research papers and top 10 evaluation papers that have been downloaded from my site over the last year as well as like seven curated ones that we think are really, really worth the time to read that I'll make available for people to print out on the tape I'm going to do one other thing real quick when I was in Texas I was thinking you know what am I going to talk about support happening in the field right now? and I started thinking well maybe it came citizen science and so what I did was I said for every letter in the alphabet I could think of something that's important or trendy right now I'm not going to go through them all but what I did was kind of put them all on these slides and then I said in the notes there's a link to every single thing so activation lab like for A I know how many people work on this project it's funded by the Gordon B. Moore Foundation I talked about shared measures before this is a project to look at what activates a child to learn science what do they have to have in their metaphorical backpack to be successful in persevere so they've been doing these studies in various Lawrence Hall of Science University of Pittsburgh and they have a website called activation lab and they have been the researchers at a point now where they're about to come out to the evaluations phase so it's something to know about that may produce again some ways to think about how you're measuring what's happening in your programs acts of pro-on engagement Joe mentioned the Explore Touring Project one thing that doesn't happen enough in this field I can tell you from Warren's statement we get a lot of things submitted it's researchers and evaluators researchers and evaluators I love all this people but not enough practitioners write about what they do if you do a program your program leader designer if you could possibly make some time to write in a reflective way about what you do that is really, really important that's why I think that book is great because it's got seven different developers to say I was trying to make this exhibit that showed square root they used to have this exhibit to show it if I just added this piece or made it bigger or whatever and it explains everything they tried and how they worked with the researcher and did videotaping and then developed the exhibits and made them better so I highly recommend that anyway, I go through the whole alphabet here I want to say one more thing because it's on the it's under E here in addition to networks and centers ecosystems of learning so when the learning science and informal environments came out in 2009 it's been a very influential book first time it was a consensus volume to capture everything we know about learning in informal environments and it's been sort of the standard thing, I guess right about every first it was surrounded by science which was the companion book that went along with it it's getting a little bit old in some ways it's out there because it's been a lot of research since then so there's been a lot and in that book we talk about ecologies of learning so all these settings in which you're interconnected something that's not so interconnected in which learning happens this isn't developed into this whole thing people are talking about ecosystems now so there's a couple of papers that are included in the notes page here funded by the noise foundation on ecosystems that have promised in different regions of the country where the formal and the informal are working together in very intentional ways creating experiences that are more coordinated sometimes more laboratory but more sort of cooperative but this is something about the topic also came up upstairs about how to get in the door of the formal you can try to get in the door but you can also think about what's already in common there's good practices happening in schools there's good practices happening in all the settings it's a matter of identifying those who are trying to connect them so there are a number of projects right now that are looking at ecosystems of learning and one of them was funded by the national academies a couple of years ago STEM learning is everywhere you can go to the national academy site and get the entire communication for what happened there you can go to the foundation find out a paper by Saskia Trill and Kathleen Trappay that does a case study of 15 different places around the country where they think it's promising their findings, what they think it takes to do this what kind of leadership what the issues are what the recommendations are so ecosystem is something I really see on the horizon but that said as with any at that STEM learning is everywhere communication Carol Tang she's now at the children's creative university before that she was a program officer at Bectl before that she was at the Lawrence Hall of Science before that she was at the California Academy of Sciences she's a very smart scientist actually and she studied ecosystems as a graduate student that's fine but don't forget the ecosystem consists not just of a problem of carnivores but also of the ecosystem such as bacteria and sometimes the critical species in the ecosystem is not a sea art but a sea storm similarly a large overall community organized over the year may end up being one as far as significant change also ecosystems are not efficient they follow a very long time period and they constantly change when we think about this work we try to get it to easy to screen and I just like that as a way to think about being attentive to all the different parts and you know sometimes actually die away and that's a good thing you know things that aren't working in the form of science and that some things can often do like in the beginning it's real Chinese so there's a number of more of those alphabetical things in there but you can just thank you very much fine together you're geographically distributed distributed network how important to other kinds of non-basically connected I hope so I don't have I mean you know case is not a network but we a lot of people are involved in what we do and we do a lot of conference calls we do webinars and you know that sometimes works but it's always usually that's why the convened part is really one key part of our role because we found that if we have an in-person convening even if it's only one on a topic as long as we nourish that and cultivate that discussion that can go really far but you have to do the stuff after you know so I think it's key and it also was mentioned upstairs by another face to face and of course it is you can keep interest and participation in a topic or something going but you have to really you know keep cultivating and nowadays I mean in this effort so it seems that a lot of the networks have a tendency to fold once the large funders go away so are there examples of networks that have sustained themselves with other kinds of funding models or you know just is there hope once a large engine so some of them like I mentioned before the way they did was just regional members and once they did that they could get local funding to have like an Atlantic region and sometimes that works does anybody know of any examples of oh I have one so it's a long term a lot to do on a recording L-T-E-R with funding no but they're funding went away and a lot of it have got they've got to stop funding for a long time I mentioned upstairs there is something and under the bees was a broader impacts network there's now a network so I mean this is third year National Alliance they have a NSF grant in the category of RCM research coordination network I'm not sure what all the requirements for that are but what that grant allows them to do is to have activities for five years it doesn't pay anybody salary or anything like that but it pays for them to have summits so it does get it keeps that person thing going and count on a summative so that's a route that they used of course they weren't coming out of a they weren't ending that's what they started but it made sense for them to just fund the activity so it's a vehicle that NSF has I know that some they figured out how to transform they were a cooperative agreement before but then they changed to a project they just wrote a different proposal so that they could do a different thing with what they had already been doing but not have it be a cooperative agreement so you can be creative with what the agencies offer to keep going but you bring up the points at some point you're going to need a situation of you know and sometimes what survives is not the network it's all the good stuff that happens that's okay and people do get used to like working with the same people and they want to keep working with them and it's sort of a phenomenon and sometimes that could be good I don't know many other examples well so luckily Celebrolessiancia that had funding to do a lot of stuff but one of the things was state fair and people still go to the state fair in that day if they started a science Celebrolessiancia that was sort of the love same players in the kind of the network here the museum collaborative council that was a network actually still is there's several exhibit networks like the the science museum exhibit collaborating I think they spoke in some form partifying and sharing you can't find you know they don't have a website they know each other we started with their science six years ago in England and France in Canada but keeping it alive is the same thing a shared group of people talk about the same thing or just struggle if you find it enough to do something I mentioned that citizens science started their own professional association so that's also a way to go is to start a professional association a community practice essentially to keep doing a certain kind of work and establish a 501c3 and membership and views and that's a lot of work sometimes that's the right people you mentioned that researchers and evaluators are kind of hard to come by and I noticed that you know a lot of journal publications are willing to share the research that is out there do you or does anyone in this room know of local University of New Mexico or state researchers that are specifically studying this in the Department of Agriculture Education outside of that do they're studying informal learning do you know what's here in town maybe right anyone who studies for me though I'm just wondering about is this here in town right and there's a group called Apex but they're from higher they're local and TBS has a pretty TBS out there they have a pretty hefty research and evaluation group but once again how do you connect to that to get there do you know has anybody ever worked with Slover Linnett and Sam yeah there's a guy I don't know Slover Linnett it's called audience something they're in Santa Fe they do really great work in fact one of their evaluations is on this talk it's an evaluation of an art museum actually it's one of the best evaluations I've ever seen and Peter himself also runs something called culture kettle has anyone heard of this it's not the only one they put it on a culture kettle did you hear that culture kettle by tea kettle a couple years ago just two years ago at MIT they ran a conference called the evolving culture of science engagement and they actually it was a lot of people from our field including and other people in our field but also like Neil Tyson was there and they produced this document that is really really interesting about the new frontiers to think about engagement one of the things that science centers sometimes aren't able to do because they can't be as nimble as smaller organizations or projects and it's available, we have it on our website but you can also go to Slope and Leeds website and it's called if you just search for the evolving culture of science engagement it's a really good report and he continues to be involved in that so I don't know if he'll do other ones but he's not that far away so excuse me your email address I'll make sure you get the notes from today's talk also Karen I wanted you to announce that she has many copies of these seasonal sightings birth migrations along the middle of the American calendar which is really beautiful so what's that going to be? CQ her medical bag of them I've got more in my car I still need to go I've got to stop your video