 You know what, we have a lot of really good discussion that is just waiting to happen. So even though I'm sure a couple more folks are drifting, let's get going with our topic, which is increasing collaboration in civic tech research by acknowledging the potential for there to exist divides in research, across research tradition. The inspiration for this conversation. Really has crept up on me as a result of being somebody who trained in an academic tradition who has explored doing impact research in terms of evaluating the effects that a civic tech project has on the world. And working, if not directly with, at least close to design researchers who are doing work that I observed was really useful, really interesting, and in some ways overlapped with the methods that I had been taught when doing qualitative research. So, you know, the idea that we have a lot of research that's happening in parallel and possibly not in conversation with each other is something that I thought would be nice to kind of explore. And so I brought together a group of people who have a lot of thoughts on this and who come from a variety of research traditions. And I thought that that might be true of this group as well, that we would have, you know, at your tables an opportunity to explore how your approach to research aligns with or is dissimilar from other folks who share an interest in the same topics as you. So, what I'm going to do is I'm going to go down the line and ask people to introduce themselves, to also introduce themselves as researchers, so to say what kind of research... Actually, let me take that back. Three questions. Who are you? What is research? Because that's an important part of this, right? And what kind of research do you do? These are Shella. One at a time. It's built in. So, my name is Christopher Wilson. I don't really know what research is. That's a hot mess. But I like to think that I have an idea about what research should be, and I think it has to do with answering specific questions that we will use the answers for and doing it in some kind of methodical way so that we can trust the answers we get. I am a... I'm kind of a civil society refugee in academia. My background is in the UN, where I did a lot of, you know, impact assessment style research. Then I worked for an engine... an organization called the Engine Room for a number of years, where we did a lot of practical applied research. Lots of it ordered by donors who wanted to know how their work was playing out. And then I started a PhD, and now I'm in the academy. And I struggle. I spent a lot of time trying to figure out how to exploit the academy and all the expertise there is there for everything it's worth for practical application, because right now, as we all know, there's this dramatic divide between traditions. There's a whole host of academic disciplines doing serious research on the stuff that we do, the types of things that we do, and it's fragmented according to the research object, whether it's studies, e-participation, or open government, or voting, or civic engagement, and it's fragmented by discipline, whether it's political scientists, or communication scholars, or anthropologists. And then it's again fragmented by methodologies. We're in quant, people doing qual, and everything in between. And none of those disciplines are talking to each other, and we're not talking to any of them, except for Tic-Tac, where we get one or two every year. And so figuring out how to make the most of that, because at the end of the day, the methods that get applied are important. They are how we know that we're not just finding the answers we're trying to find. It's how we know that we can trust the results of our research, but they've also become incredibly onerous and difficult. And all these things that happen in the Academy that gives the Academy a bad name, things like IRBs and peer review and slow publication processes, these are safeguards that have ossified over time. They were put in place for a reason. I like to say it's very similar to the rules in large institutional procurement processes. We do it for a reason to avoid corruption, but it becomes this terrible thing that everybody wants to avoid. So what I want to figure out is how to extract all the good stuff. How to learn to apply methods in practical applied, quick research that will give us meaningful results without getting sucked into all the quagmire that is the Academy. If you have to choose a word, or two words, and the reason that we're going to choose a word or two words will also be an object for you so you may think about this too. Internoper. Methodological fundamentals. Okay, good. Good morning, everyone. My name is Che Chi-Sum and I'll just come with a mic that will make your life easier. I'm from the OECD. More precisely, I work for PISA, the program for International Student Assessment. So I wonder why, how I define myself as a researcher. I would say I'm a free rider of all the wonderful work that they do. And also like an active consumer of knowledge and passive contributor. I will define it that way. This is why it's just because I put a little disclaimer. I'm not in academia. I have two masters, but I don't have a PhD. So I'm not trained as an academic researcher. But why am I here is that we have done, at OECD, we have done a very interesting, two interesting projects last year. The first is we call PISA-based for schools that we give individual school a tool, an assessment that they can compare the performance with other countries. Which is pretty new in the sense that it is first time schools have their own international data which they can do comparison for school improvement. The second thing that we did last year was called PISA for you. That it is an online collaborative platform that we group teachers and educators from all around the world together. They work with each other. They come up with a common challenge, a common solution, an implemented solution and evaluate it. Innovative in that sense that we give the voice and data for individual teachers. So why I am here is that, again, I'm providing all the conversation here, is that I want to know how to define so-called success for this kind of civic technology. What is success like and how can we conduct research to measure such success and what kind of mechanism that we can use to scale up to the impact of such kind of project. So, yeah, I want to share now and look forward to a discussion with you guys. If you had one to three words to describe yourself as in on a hat, what would they be? Good morning, folks. I'm Martin Wright. I work for my society. My job title is designer and that probably gives you the clue that I'm not actually a researcher. Actually, my imposter syndrome is inflamed at the moment. My role is very broad. I do everything from branding like this stuff, all the way to talking to users, understanding how they use our online tools and talking to our own staff, funding out the issues they have. So research for me is not a thing I am. It's kind of, it's more like a tool. So I'd liken it to a magnifying glass or like a periscope, something if I don't understand something, like research is the thing I use to understand it better. It might be early on, so it might be a problem. I don't quite understand a problem, so I'll use research to help define the problem better. Later on in the process it might be, we might have a solution, so I might use research to judge the success of a solution. So my one word would be designer I guess. Hi everyone. My name is Anna Coulomb. I'm a first year PhD student at the Open University. But before that I've been working for around eight years in applied research in the charity sector. Specifically for most of these years at BBC Media Action which uses communication and technologies for development. So I've come from working in the applied research world with quite a fast pace and a specific set of structures and ways of working. And now I'm into academia. And in academia I'm working across four different disciplines. So I'm technically registered in the politics department. But I have a team of three supervisors that are from three different disciplines, sociology, psychology and development. So I'm saying this because it shows the importance and the importance I place myself in not working in one silo or compartment but actually across. So I would myself perhaps also define as a free rider or perhaps like a chameleon where you try and apply basic principles in different contexts. And talking about different or basic principles, I would define research I guess as a systematic inquiry it's replicable in the sense that the process has to be transparent. And I think as long as you are transparent about your position, the methodology and the rationale for what you've done and why, then you can have academia questioning it or improving it with more rigor or you can have different traditions applying its strengths I guess. So I guess we'll talk more about that later but that's what it is for me on the basic level. So I guess, okay I'll use the word chameleon I don't know if it's the same as free rider but we can discuss later. Hi everyone good morning I'm Nicole Anand, I'm the director of strategy and learning at the engine room So I'll go with that first one, what is research For me I think at the broadest level research is essentially just information, collection, synthesis and analysis but to what Christopher was saying about good research I guess there's kind of two elements that I would talk about. One is around this idea of methodology that everybody has touched upon and I think it's worth just kind of unpacking what is methodology, what defines methodology and for me that really the main thing is about intentionality and talking about what your intentions are so that you're intentional about your objectives, you're intentional about the people that you're asking questions interviewing etc so it's really just that component is what makes up a methodology The other part that I think is worth talking about is something that we've also kind of touched upon which is okay so you collect and you synthesize and you analyze but then for what and so I think with good research there is this component, you can call it what you want but I would maybe call it engagement not so much an after thought sort of communication strategy or dissemination strategy but even kind of throughout so how are you engaging your respondents in your interviews from start to finish so for me those are kind of the important elements of research in terms of what type of research I am I think 10 years ago so let me start with current I guess I suppose some people would call me a design researcher now since I teach design research at Parsons in New York but I think 10 years ago I might have said I do academic research 8 years ago in India we called it action research at the civil society organization that I was part of action research or participatory research so I'm not sure but I do know that my role at the engine room right now we at the engine room we house a lot of different types of researchers so we have user researchers, we have user experience researchers, we have design researchers we have what we call practitioner research which is the core of it is like really focusing on digestible accessible information so for me my role is to kind of look at all of those methods, find the complementarities between them and bring them together so I guess my word would be mixed methods researcher maybe hyphenated, that's two words right? Thank you there was a lot of wonderful scene setting here because the intention is to explore our relationships to the concept of research and how we kind of connect to that as an identity because each of these processes of learning is also in a acculturation, a socialization process so for that reason and the reason I kept narrowing people down to the one to three words is because now there is the craft section of the workshop where I'm going to see if people would be willing to take their one to three words and put them on a hat so my words are going to be academic and evaluation it is entirely up to you whether to assign meaning to that the way that you put them on if you are open to that if you do not want to wear a hat you do not need to wear a hat you can also choose to label yourself with a sticky note also if you are at a small table you do not need to do this at the moment you can also do this as a discursive practice in a little while this is an experiment in how the hats affect our conversation to some degree too but to put them on what you can do is take the two ends turn one around what we're going to be doing is kind of slotting them in by cutting by kind of ripping halfway if you turn it back you can slot them together I am not a designer but this is my first experience with an effort at a group paper craft design so if you don't have any at your table there should be a couple up here at this front table if you are interested in participating I have no words to say what we're interested in how do you define yourself as a researcher or what kind of research do you do ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?