 to have him with us for the year and have him with us for the evening. And so with that, I'll turn it over to you, Eric. Thank you, Scott. It's really great to be here. My name's Eric Gordon, as Scott just said. And I've been here for this year at MIT and really enjoying my time so far, so getting and speaking this forum is really just an honor. So let me share my screen and hopefully this will go well. Hold on one second. Do you see a towards a meaningfully inefficient smart city slide? So you see? Yep. Okay. Good. And let me try this. All right. Wonderful. So, yeah, so my talk tonight is entitled towards a meaningfully inefficient smart city. I will explain all those words in a bit. Also please be patient with me as I do that. As Scott mentioned, I direct the engagement lab at Emerson and we have focused in the intersection of play and civic life for some time. And the work that I'm going to talk about tonight is both emerging out of the work that I've been doing for nearly a decade now, but also firmly tied to a recent book that I published in March, which I'll talk a little bit about. So let me start by telling you what you already know that on May 25th, George Floyd was murdered by Minneapolis police officers. The next day, a video of the crime was circulated on social media and the city erupted in protest soon after. And within a week, throngs of people demanding racial justice took to city streets all over the country and the world. Soon after, a majority of Minneapolis city councilors pledged to defund the city's police department. And they said, we are here today. This is their pledge. We are here today to begin the process of ending the Minneapolis police department and creating a new transformative model for cultivating safety in Minneapolis. And since this time, it has largely fallen apart. There's such a sense of urgency around trying to start some sort of community engagement process that is staff-led and better embedded in the city enterprise, says Councillor Bender, so that it isn't just the policymakers having these separate conversations and our wards that aren't recorded or captured anywhere so that we can help to build confidence in our community that we are moving forward in a way that reflects the complexity of the multiple layers of crisis. One resident of North Minneapolis, a predominantly black and brown neighborhood, had this to say about the effort. They didn't engage black and brown people, referring to the city council members. And something about that does not sit right with me. Something about saying to the community, we need to make change together, but instead you leave this community and me unsafe. So the decision was given over to a Minneapolis charter commission, a mostly white appointed commission that decides matters of changes in the city's charter. And then last week, just last week, the commission has put the ballot initiative on pause and the city's relationship with the police department is about where it was in May. So this story is a complex morality tale. It's a story about the success of the movement for black lives, the mainstreaming of racial justice as a legitimate and essential concern for elected officials. It's also a story about the failure of procedures of governance to facilitate changes in policy. The failure of institutions to adapt to sentiment. My talk today is about institutioning. The process by which interventions, big social movements or small programmatic designs transform the institutions that enable them. And while especially in the media disciplines, we tend to focus on large social movements and specific media interventions, the process of institutioning is often where lasting transformation is located. So Hoybrecht said all argue that the engagement with procedures of institutions is necessary to repoliticize participatory and co-design practices. The missing link for progress is the transition from movement politics or specific interventions to sustainable institutional transformation. The example of Minneapolis is not unique. Over the last several months, US cities have challenged their ability to govern with their stated values. And because of widespread social movements, these values are often newly discovered by policymakers or they feel newly discovered by policymakers. It's one thing for elected officials to call foul and make a decree. It is quite another to generate and enable the trust and social capacity necessary for systemic change. So this is the focus of my research. And I'm lucky enough to have it also be the focus of my teaching this semester. The co-design studio I'm teaching is focused on designing the mechanisms for communities to effectively co-create with city government in Boston. We have counselors Andrea Campbell and Julia Mejia are partners in the course and both have expressed similar sentiments. The city council in Boston is more representative of the city population than it ever has been. It's majority women and nearly half people of color, but the mechanics of governance have largely stayed the same. So our studio class has 11 community partners and it brings together students with city staff and community leaders to co-design mechanisms of collaborative governance in Boston. So why am I speaking about this in a media department and not a policy department? Because local governments governance is mediated by digital and analog communication from voting to town hall meetings to information dissemination and advocacy. And because media studies scholars, especially folks like me with a background in the humanities have a great deal to say about the interface between expression, technology and power and the processes by which it is integrated into institutional logics. At a time when trust in government is at an all time low where nationalist and populist leaders are trafficking and misinformation campaigns and diminishing trust in institutions is a feature not a bug. It is difficult to have much faith in government. I acknowledge and share that concern, but I also understand the necessity of public institutions to provide access to goods and services to represent the various publics they govern. We might say this breaks down at scale and in the contemporary environment of the United States, it is difficult to be an institutionalist on any scale. But when we examine progressive politics in the US and beyond where struggles of democratic representation are inching towards legitimacy, that is happening on the local level. Cities are where nearly 60% of the world's population lives. And they are and have always been sites of contestation and evolution in ways of living and governing. They are sites of social protest. They push boundaries of difference and tolerance. And they are sites of technological transformation that have perpetually altered how humans live together. As Georg Simmel said in the opening lines of his 1901 essay, The Metropolis and Mental Life, the deepest problems of modern life derive from the claim of the individual to preserve the autonomy and individuality of his existence in the face of overwhelming social forces, of historical heritage, of external culture, and of the technique of life. The city is the locus of phenomenological and corresponding institutional transformation. Simmel argues that the metropolis is where individuality through capitalist pursuit is placed in tension with the management of external stimuli and the throngs of social difference. I just realized I used the word throngs twice in this talk. That's special. He explains a coping mechanism that he calls the blase attitude, which enables the city dweller to adapt to an unprecedented inundation of stimuli. The city is a site of constant transformation. Individuals and groups develop filters through which to cope. And institutions need to evolve such that they are capable of governing within the conditions of rapid change. While Simmel doesn't use this phrase, this reality implies a certain intelligence defined by situational adaptability. It implies a smart city. Smart cities. It's a designation that has become a marketing catchphrase that positions competing municipalities in the global marketplace of innovation and creative economies. The label tends to invoke images of responsive technologies and efficient infrastructure. There are global smart cities competitions and mayors around the world are desperate to put this label on their place. I don't want to engage in a debate about the relative value of the smart city label. The critical arguments in the space are well rehearsed. I do, however, want to expand how we think about the emotional intelligence of cities, mostly as a site of experimentation of collective pursuit of new ways of living and governing. The city is an important unit of analysis because it marks the ability for institutions to make connections, act in good faith, build trust, be held accountable and cultivate responsibility for something resembling a public good. Smart cities tend to focus on systems, prioritizing certain values of efficiency and ease. But if we go back to the example of Minneapolis, the promise of efficiency and ease placed in conflict with the messiness and complexity of the demands of emerging publics led to institutional inaction. Which leads me to the overarching question of this talk. What are the alternative logics of intelligence that might better explain and compel democratic urban transformation? So the shuttering in May of Google's Sidewalk Labs initiative in Toronto is instructive. The single provider model of a smart city where Google transformed an industrial area of Toronto's waterfront into a completely digitized, responsive and smart environment was an impressive vision of future urbanism. It's highly efficient infrastructure, near complete social connectivity, usable interfaces for information access was a compelling representation of a smart city. Sidewalk Labs even held design workshops to involve members of the surrounding community to help shape outcomes. But as Shannon Maydern suggests, these workshops were largely performative and the well-funded operation overlooked the relational work necessary to distribute a sense of ownership and the process and build trust with those who had every reason to believe that their participation would not translate into having any real impact. Without an opportunity to be heard and have impact on the outcomes, it is not surprising that within the social restructuring prompted by the pandemic, this model of a future smart city would prove not to be desirable. The logics driving the vision of the future city were focused on optimizing for efficiency around understanding human behavior and were largely absent of values that would lead to a city built on trusting relationships and systems of caring for people's needs. In March, my book Meaningful and Efficiencies, Civic Design and an Age of Digital Expediency was released. This book profiles civic designers or people within public sector and other public serving organizations who are typically working against dominant organizational cultures to craft human systems guided by relationships and care. Gabe Mugar and I tell stories of people from small news organizations seeking to transform the interface between audience and newsroom and people from municipal governments who are desperately deprioritizing a focus on streamlining service delivery by designing inefficiencies into systems as a means of shifting focus to the time and labor intensive work of building relational trust with historically marginalized communities. In February, I understood these practices as fringe, as subtle acts of resistance inside of public organizations. But today, the act of pursuit of a values-based transformation has been mainstreamed. The pursuit is in no way straightforward. As we see in Minneapolis and Toronto, it is expensive. It often runs counter to common sense practices of incorporating technological efficiencies into antiquated organizational practices. And as a result, beyond initial platitudes, it is politically difficult to execute. Unlike past waves of public sector innovation, it cannot be addressed by a small batch of design thinking workshops or by developing a new app. Cultural sensitivity training is not enough and quotas for increasing diversity fall short of addressing structural processes that perpetuate inequality. What is necessary is not just changing the appearance of governance as we are reminded of in Boston. It is necessary to change the logics that guide it. And so I point to meaningful inefficiencies. Systems deliberately designed with Slack in order to hold space for a diversity of stakeholders, viewpoints, and emerging forms of governance. I compare this logic to that of a well-designed game, a system with clear goals, consistent feedback, and room to play. Opportunities for people to fail safely, experiment with solutions, and build relations within a constrained environment. Meaningfully, inefficient programs and processes not only make engaging experiences, but when inside these systems, one can build the trust and relationships necessary for communities and organizations to thrive and care for the issues that matter to them. This is not an argument for more public meetings. The goal is not simply more dialogue. That is part of the goal, but it needs to be situated within a capture device that is capable of allowing reflection, learning, and growth. We need to be thinking about the systems of cities that enable them to become smart, to learn, to grow, to generate wisdom. So consider the Talmud, over 6,000 pages of rabbinical interpretation of the Torah, which has served as the guiding legal document for rabbinic Judaism. The ancient document captures interpretation born of a cortex filled with ellipses, such that meaning can be made of the system's logic. In the center is the core text. On the right is Rashi's interpretation, and on the left is yet another interpretation, and then surrounded again by commentary on the interpretations. This spiral of knowledge capture is a structural solution to containing and learning from inefficiencies. So what does this look like on a city level? How can cities learn? They certainly can't be smart unless they are able to learn. And currently, the institution's mediating public life are not set up to be smart. They contain the mechanics of public life, which might include voting, protesting, tweeting, but they lack the ability to contain the interpretation of these mechanics, the listening, the evolution of novel ideas, and to agreed upon practices. So I have introduced a logic of meaningful inefficiencies to use a different, more familiar term that is descriptive of institutions that enable meaningful play, or we can talk about meaningful play. And importantly, we're able to learn from and evolve from that affective, collective experience. So the philosopher Bernard Soetz explains that to play a game is to achieve a specific state of affairs using only means permitted by rules, where the rules prohibit the use of more efficient in favor of less efficient means, and where rules are accepted just because they make possible such activity. Playing a game is the voluntary attempt to overcome unnecessary obstacles. So games are, by definition, then, inefficient. It is, in fact, the inefficiency of these systems that cultivate and support the experience of play. So Soetz's famous example is the game of golf, where he talks about the most efficient way of getting the little ball into the little hole is to pick up the little ball, walk over to the little hole, and drop it in. But instead, all these unnecessary obstacles are placed in the way that are inefficiencies that enable that play to happen. And by agreeing to play the game of golf, you are voluntarily entering into a system so that you can play, so that it enables play. And so what the conclusion here is that the play is the affective quality of modern urban life that is necessary for shared understanding and collective care, reshaping the mechanisms in which cities are governed to recognize and develop collective play is the goal of my research and design. So now I want to talk about two current studies that are exploring emerging forms of governance in cities. The first is an experimental design project in the city of Boston, which is in collaboration with the mayor's office at New Urban Mechanics, that explored a collaborative process for making decisions about new technologies in the public realm. And the second is an evaluation of an effort in Cluj-Napoca, Romania, where in a loose structure of multi-sector collaboration is being formed to advance citywide efforts to enhance the well-being of youth. So I'm going to talk about both of these projects back to back. So the first one is Beta Blocks. So Beta Blocks is a project that came together in 2018, 2019. It was funded by the Knight Foundation. And as I mentioned, it was in partnership with the mayor's office of New Urban Mechanics and the architecture firm, Supernormal. And what we set out to do here was to address the core problem, which was the way in which technologies find their way into the public realm. So normally, technologies, cameras, sensors are installed in neighborhoods without any kind of public consultation around that technology. So there is the public sector, the government decision makers. There are the technology companies that tend to make deals with one another. And the community, for the most part, is left out of that decision making as to what technologies are in their space. And here I'm talking about public realm technologies, sidewalks, publicly held spaces. So Beta Blocks essentially was designed to invite some friction in this process. And it was a process to bring residents and organizers into the conversation between government and the technology companies. And I'm going to explain how we attempted to do that. There were five primary components to this intervention. An exploration zone, which was the site for experimentation. An advisory group that had oversight of that zone. A public exhibit that was designed to encourage dialogue, a specific youth curriculum that was designed to bring youth into the process specifically. And then ultimately, recommendations, which was the collaborative governance function of this prototype intervention. So let me go into more detail. So in the city of Boston, we identified three neighborhoods where we set up exploration zones. There's Lower Alston, Chinatown, and Codman Square. These areas were chosen based on a number of factors. We looked at need. We looked at the data to determine where there was need. But we also, there were also political factors, which was the government. In the case of this particular project, the government helping us to decide where friction may be more or less present and where we might have more traction within communities. So it was a complex process of selection. I can talk more about that later if people are interested. So the exploration zones were there essentially four square blocks that were identified in the city wherein permitting would be relaxed so that technologies could be temporarily installed within that area for the purpose of public scrutiny. And as I said, each of these zones was governed by an advisory group that we set up over the course of the project. So now the advisory groups were created in a number of different ways in the different communities. They were designed to include community leaders, volunteers, and youth. They met monthly while the zone was in place. The goal was to match local priorities with tech and to identify locations of the tech to be installed and provide feedback for those installations and then data policies and then also feedback into business plans. So the part that I didn't mention is that the other actor or set of actors here are the tech companies. And the tech companies had their tech that they would essentially lend to the project that could be temporarily installed. So they were other stakeholders in this process. And the community feedback was also part of the business plan was open to feedback, which is an important feature of this program. So one of the companies that we worked with was a company called SUFA, which is a startup out of Boston actually originated out of MIT. And it's essentially a solar powered digital billboard that the great feature of SUFA signs is that they don't need an electrical outlet, so they can be placed in, you know, you have more variety of where you can place them. And so they just be plopped down and then they feature sort of local advertising, they feature municipal data, it's very flexible as to what it is that can be featured. So SUFA partnered with us and lent us a number of their signs. And then these were located by the communities, by the advisory groups. Another company that we partnered with couldn't be more opposite, it was Microsoft. And in this case, we worked with Microsoft and they had developed a new air quality sensor that was located in each of the different zones. One of the things I want to point out here is that, as you see here, there's a sign that is next to the air quality sensor that was on the light post that the air quality sensor was hanging on that says this is an air quality sensor, this is the one in Chinatown, as you could tell. This is an air quality sensor. This device is temporarily installed here to help understand its value for your neighborhood and then it includes a phone number and a website to get feedback. And this was in addition, of course, to the official ZAG or zone advisory group. So one of the tasks of the advisory group was to locate the technologies. So we had the groups would meet and then determine the location of the sensors, the location of the signs, and we had other technologies as well as part of this pilot. And these were robust conversations often because they involved like it was a sourcing of problems. Why would we need an air quality sensor here? If we think we might have an issue here, then we should locate the air quality sensor. In this neighborhood or in this corner, why would we need a digital sign here let's locate it where there is particular pedestrian traffic that would actually make use of the sign. So these are the kinds of conversations that were happening in the zone advisory groups. So OK, I'm going to leave that, I'm going to sort of bracket that. So we had the exploration zones, we had the zone advisory groups, and then we also had a traveling exhibit. And this was designed to encourage public conversation about the issues of technology in the public realm. One of the features of the traveling exhibit was this nine by 18 inflatable structure that we call the beta blob that we could locate throughout the city and it became a site of exploration of these technologies. So wherever the beta blob was located, you would first be greeted by this rendition of a future and past city. And we would ask questions such as, how do you envision technology in your city or how do you use technology in your city? And we would sort of open it up immediately to some sort of dialogue. As you see here, we would put the blob in public spaces. So here it is in the Rose Kennedy Greenway in Chinatown Park. We sometimes would call this a hug object because people would come up to it and hug it. And it attracted people in that way. It had a very inviting presence as it turns out. And it became a place to kind of open up the conversation about technology. One of the most popular features of this exhibit was this board game that you see being played here that always sort of do crowds as we had the blob up around the city. Now again, the idea here is we're trying to create the sort of interface of like how we can create these kinds of conversations on a public scale that would actually lend itself to changing the way that technologies are procured within the public realm. Here's another example in Covenant Square where the blob became a site of picnicking as well as playing the game. So it sort of picked up multiple functions as it moved about the city. The interesting thing about the blob is that it was quite difficult to locate outside as you see because it could be blown away. So we had to tether it down here to I think with 6,000 pounds of bricks that it had to be tied down to that had to be sort of moved every time the blob would move. So it was a remarkable amount of effort to move this playful object around the city. And then it culminated in a installation in City Hall where we had the blob and this was really for policymakers to see the kind of input that was happening around the city. Not only the results of all the zone advisory groups but also the results of the game and then the game was there for people to play as well in City Hall. It was placed there for over a week. I'll just say a little bit about the youth curriculum that also operated on the side. This was a four part workshop series for youth 14 to 20. It focused on data storytelling and visualization and then youth at the end of this created an infographic driven story using quantitative data to sort of tell these qualitative stories of how people are using data and technology in their place. At the end of the zone advisory groups there were recommendations that emerged and this is an example of recommendations that came from specifically about the Sufa sign in Chinatown. Profit sharing with or financial contribution to the community, oh sorry, what conditions need to be met for Sufa signs to be valuable for Chinatown? Profit sharing with or financial contribution into community, display relevant information to residents, community events, job opportunities, info about local businesses, clear and bilingual labels and then operate at no financial cost to the community. So this is just one example of a community response to a particular tech. Now I just want to remind you tech was placed temporarily because permitting was only temporary. It was there for 60 days and then it was removed. And so after that 60 days, there was this opportunity to provide this inputs after this lengthy process. Other things that I won't go into was a data policy that was also drafted by the zone advisory groups that looked something like this were principles that were collaboratively authored by the group. And then these were to happen in each of the zones. And so this over time could lead to a data policy that was emergent from the different neighborhoods of the city. Okay, so here are the outcomes, some of the outcomes. It was years in the making, we developed a valuable model that is currently being explored by other municipalities. However, the prototype was never able to integrate into governance structures. And partly because our primary partnership was with the mayor's office and we weren't able to fully integrate into other units within the city. So what was interesting about this project is that we focused on the infrastructure. We focused on the logics of the way that governance happens when it comes to making decisions about technology and the future of cities. The result was mixed. The result was we created an interface but the kind of connective tissue wasn't there in this prototype or not yet, I should say. And it's one of the interesting things about doing work like this is that even though our design goal was focused on governance, they may not have been the, those weren't the immediate impacts of this project. And again, I can talk more about this later. Let me move on to another project. And I just wanna, I won't, I don't have as much to say about this. This is a new project, a project called Our Cluj. And I'm gonna see if, actually, I don't think my audio is connected. Can you hear that? No, okay. All right, I won't mess with this. So the Our Cluj is an initiative that was started by a foundation called Foundation Botnar. It's actually a Swiss foundation and they made a commitment to the city of Cluj-Napoca for a 10 year commitment to Cluj, and which is a sort of unusual thing. So the capital city, so Cluj-Napoca is the capital city of the Transylvania region, which is the city has a population of about 700,000 people. It's about the size of Boston, has a large concentration of universities and a vibrant youth sector. And so for the last six months in collaboration with a Romanian based research team and also with the foundation that I mentioned, I've been exploring the structures of multi-sector collaborative governance on the city scale and trying to understand where alternative logics are being experimented with and deployed. So, so far we've interviewed over 40 people in government, academia, civil society and the private sector twice and I'll explain a bit more about that in a second. What we're beginning to see in Cluj is a tension between local government's concept of itself as reformer of a post-communist system and civil society's concept of itself as connector. But each is paying significant attention to the complexity of aligning relationships and the slow process of building trust between stakeholders and the need for better mechanisms with which to do this. So instead of focusing on individual tools for transaction, there is increasing attention in Cluj to emerging models of collaboration, which is represented well by one particular effort that I wanna talk about, which is the Urban Innovation Unit is a non-governmental R&D program aimed at strengthening collective action and response to the city's strategic challenges. This unit sits inside the Cluj Cultural Center, which is a consortium of over 20 organizations in the city. And now the Urban Innovation Unit is working in parallel with the Center of Innovation and Civic Imagination, which is a newly created unit that exists within the municipality. So what's fascinating about this example is that these two organizations, one in government and one without, are working as a mirror image of the other. They have duplicated staff, which enables them to both operate in a nimble extra governmental structure. While at the same time working within government, with all the access to direct service provision that that enables. They are focused on building out new mechanics of civic participation, ones that would not have been possible from within government, as people still tend to distrust government, especially on the national level. As such, they had the foresight to develop this unit outside of government with a plan to transfer it fully to the municipality in 2022. So what I'm seeing in Cluj is a self-perception of multi-sector work that is unique and attention to the mechanisms of governance in and out of government with all the messiness that entails and a desire to learn from it. While it is far too early to report on any substantive findings from this research, the novel governance structures being built in this secondary city are well worth paying attention to. So I'm gonna say a bit about methods here. So in order to understand how people see themselves within these collaborative structures and how they might understand the experience of playing an institutional context, we're actually using novel research methods to capture additional insights. So during each of the interviews, I have two to five artists sitting in to creatively document the process. Within two weeks of the interview, we share the completed artwork with the interviewee. And then we have a follow-up interview to get their reaction and interpretation of the creative representation. So the use of arts in qualitative research opens up spaces of interpretive play. The first interview, the first interview, I asked people what they think. The second interview, I asked people what they think about someone else's representation of them. Art is filled with ellipses. And when incorporated into social scientific methods, which offers a capture device to learn from its generative discourse, it can be tumultic in design. This method creates productive spaces for interpretation and self-reflection that would have otherwise been impossible. People respond to the way they are represented, the imagery used, the emotional suggestions of the artwork. They see themselves at a distance, creating a space of strangeness that begs for interpretation. Some other arts that emerged from these interviews. This is one of my favorites. That represents a man walking down a dark hallway and then battling a bureaucracy monster that elicits quite remarkable conversation. And I will say, actually, I'll just briefly back up to these images. And these are of two different people. And what's interesting about these images is that they create all sorts of dialogue in the person that this represented. So now they're able to see themselves. They're able to see the way that they interact with all the pieces of the puzzle that they've mentioned. And a lot of people felt, again, with the sort of ambiguity of these particular representations, it was incredibly constructive and generative as they talked about their sense of self within this emerging ecosystem. So let me come full circle to play now and conclude. When I talk about my work, people sometimes mischaracterize my interests as gamification. As I hope I have made clear in this talk, I'm not interested in gamifying public life. Gamification, which is the incorporation of game elements into non-game systems typically has the effect of increasing efficiency, motivating one's path to predetermine goals. I am concerned with games only in so far as they are model systems to enable meaningful play. And as such, they become a useful metaphor through which to understand the transformation of cities and urban governance. So let's return to the example with which I started this talk. When we think about the crisis in Minneapolis, the question is not how can we design a game about police violence and systemic racism. The question is how can our systems of urban governments accommodate the needs and interests of multiple publics by enabling them to explore, discover, and augment the systems that contain them? This is the key question for smart cities. And the only way we answer it is by looking at the logic guiding institutions, not only the individual policies or programs. This is my smart city agenda. An agenda I call the civic smart city. At a moment when emerging technologies are transforming every aspect of everyday life, from how we live to where we live and how we live together, research and design interventions need to be directed at the logics of the mediating institutions such that appropriate forms of governance can emerge to meet the demands of the governed. Thank you very much. Thanks, Eric. So we want to, maybe if we, if you unshare, like invite everyone, if you're a panelist, just raise your hand and take your question or use the hand raising feature under participant. If you're a guest, feel free to ask a question in the Q and A, on the Q and A piece. I know it usually takes people a while to fire up. So I want to start, I'm really intrigued by the work in Romania. And I'm wondering, was there anything particularly surprising either in what the artists produced or in the ways that the interviewees engaged with that production? Are you speaking specifically about the methods, Scott? Yeah, yeah. Yeah, yeah. That's been really exciting to do. It's been a learning curve as we figure out how best to do it. So let me, I'll tell you what was surprising. We have, I think, 12 different artists that are part of this group called Art of History. It's an artist collective in Cluj. They're mostly students. And so what we do is when we have an interview, we coordinate the artists and we have two to five, as I mentioned, come in and sit in. And what's interesting is that all the artists have different styles, right, of course. And so there's nothing particularly scientific about the approach. It's all the artists have different styles and some of them are much more representational than others. And some of the artists do kind of, you saw a couple, right? You saw the sort of comic style and then there was, you know, then the other style that was much more somber. And then there's some were even more representational than that, more like, you know, kind of infographics. And what I found, perhaps not surprisingly, because of everything, based on everything I said today, but what I found is that the greater the ellipses, the greater the conversation, right? So when you clearly say for people, here's an infographic, here's what you just said, and you do the thing that we do in social science, which is member checking. We say, what do you think about the thing that we think you just said? And then, you know, they say like, good, you know, that's good, maybe I changed that detail. That's mildly interesting. But when you have something with a bunch of ambiguity in it, tons of ellipses in the representation, then all of a sudden people fill those gaps with their interpretation. And then what we get from that is this kind of remarkable conversation about their sense of self, like so many of the people that we interviewed have talked about, you saw in some of those images of the sort of the men that were, you know, kind of big and looking over the city, people respond to that. And what's interesting is that people were like, I don't see myself that big in this ecosystem. I'm uncomfortable with that. I'm uncomfortable being represented that way. Why am I looking down? You know, like there's all sorts of things about the kind of ambiguity in that, that again, gets me to my questions, which is how does this multi-sector collaboration actually take place? How do people actually work together? How do they see others within their collective goals? And how are they reimagining governance? And I can ask them that. And that's, again, mildly interesting. But when we get it in this sideways way, all of a sudden our responses can be far more interesting. So that's been really just delightful, actually, in this project. Have you started to think about a framework for sort of evaluating the responses or is it too early for that? Yeah, it's kind of too early for that because it's so emergent. Like it's even the method. I mean, again, I'm familiar with, you know, photo voice and other sorts of methods that one uses to elicit, you know, representation from people that you are talking to. But this isn't, I'm not pulling directly from an existing method that I know of. So if anybody knows of something that's aligned, I'd love to hear about it. And so I am just, it is a bit emergent how we're gonna understand that. I mean, it's all interpretive at the moment. We're coding everything. So we're gonna be looking for, we're still looking for, you know, we're looking for aspects of, again, collaboration, self-perception and decision-making. Like those are the main sort of themes that we're looking for, but they, and they're getting at it in different ways and we're coding for that. Cool, thank you. So we do now have some people lined up with questions. I think we'll take the questions from the panelists first in order and then we'll take the questions from the Q and A. So whoever raised their hand first should feel free to just unmute and ask the question. I see two of you there. I didn't see who was first. I can bounce first if we just have someone to go. Hi, I'm Mal Satter. I'm a CMS alum. Oh, and you're muted too. I'm muted, hi. Hi. Hi, I'm Mal Satter. I'm a CMS alum and currently an assistant professor at the University of Maryland. I've also done a lot of writing and research on the Sidewalk Toronto project. And I was really interested in sort of the idea of inefficiency, specifically deployed in a political and policy context because one of the major criticisms of the Sidewalk Toronto project was that the procurement process that the RFP had gone through to create the opportunity for that project was, you know, according to many critics, myself included, wildly corrupt and compromised and therefore deeply efficient for Sidewalk. Like it was very efficient for what they wanted to do. And so I'm interested in approaching the concept of inefficiency from perhaps more of a work to roll perspective and simply a matter of actually enforcing existing policies. And so I'm wondering if the projects that you have going forward sort of interface with that at all, either through like responses to corrupt, to true corruptions and existing policies or merely adding more layers to existing policies to create that inefficiency? Thanks for the question. And by the way, you have a chapter in the, in my civic media book. I do. Yeah, yeah, I know. Hello. Yeah, hi. So now the problem is the policy, right? The problem, and so the challenge here and where inefficiencies matter is actually in shifting those policies. Like the decision-making process in the Sidewalk project was already determined. It shouldn't have been. You know, we could have built structures to change the way that those decisions were made. Now, maybe it couldn't have been otherwise because of the nature of the project. But I guess what I'm curious about your face and I want you to say something back, but the issue is not so much, we have policies, the policies are sort of efficient. There's an efficient path towards corruption which is what you just said. And everyone gets in line for that, right? The challenge is not simply to disrupt that, the challenge is to actually change the way that those decisions get made in the first place, which is what we tried to do in the Boston project. In the Boston project, I mean, yeah, we had a lot of sort of like playful elements to it, right? But at the end of the day, the goal of that project was actually to change how procurement decisions about procurement get made. It was about changing how value propositions get oriented and where smart happens. So all of those things were at the core of that project. And so if we get to the point where the fastest, you know, the most efficient path is corruption, and it's already broken. And so we have to sort of back up and create some inefficiencies in that process to enable that kind of collective decision-making to happen. So now explain your face. I will explain my face, which I made on purpose. So the issues that occurred, and this was eventually brought up before an ethics committee, Parliamentary Ethics Committee, was it included allegations such as the Waterfront Development Authority had released details of the CFP to sidewalk in advance of releasing them to other bidders that the CFP was actually crafted for sidewalks specifically at the request of the Liberal Federal Government. Like it goes all, like it went all the way down. In addition to the fact that the Waterfront Development Authority, which is Waterfront Toronto, was a well-known corrupt development authority in Toronto to begin with and had existed as sort of a very compromised entity for a long time. And one of, so Bianca Wiley, who is sort of the major critic from this perspective, she's the one who knows the most about procurement policies in Toronto and Ontario specifically, but her major argument was that it never adhered to the policies that were brought forward. And like we can have a, you and I can have a long conversation about sidewalk Toronto because I know way too much about it at this point. But so what's interesting to me is the ways in which the inefficiencies that might be implemented might best look like sort of traditionally anarchic or traditionally labor-oriented slowdown activities and that type of political resistance. So, you know, one of the distinctions that we make in the book that I didn't mention today in the talk is the difference between a mirror inefficiency and a meaningful inefficiency. And that's really important, right? And a mirror inefficiency can be bureaucratic slowdown and it can be intentional bureaucratic slowdown, like all of those things. A meaningful inefficiency is, again, one design that can cultivate outcomes that are desirable and not simply anxiety inducing. So I think what you're describing, again, like I guess I would come back to you and maybe this is a conversation for another time, but I would come back to you and say like, where are, like, what are the broken pieces prior to the beginning? As you mentioned, corruption on the federal level or the national level, corruption in the development agency, all of those things existed and also a huge desire to partner with Google, right? That was a huge win for Toronto. So all of those things took place. And again, some of these forces are so large we're not gonna be able to mess with them. That Google comes in and it's not like we can convince cities not to take that deal. However, I think that if the proposition was posed differently, for example, if we were able to say, no, a smart city is not a World's Fair style showpiece but a smart city has to be located within neighborhoods, right? Like it's gotta be where people live. It has to emerge from lived experience. We have to actually build the capacity, that again, to start from a place, not from zero, to learn from the, and to learn, to have that capture device that I was talking about. If those were the preconditions, then maybe there would have been mechanisms to say, no, Google, you can't come in. And that was sort of our desire when we started BetoBlocks. And I know that's crazy, but the desire in doing this work is to say, like, look, if we establish the rules collectively early on, then we can avoid some of the problems that we've seen in the past. We can learn from some of these examples. But yeah, I think there's, I'd love to talk to you more about this. I can just make one point, and then I will stop talking. I think that the mirror inefficiency versus meaningful inefficiency is a super great distinction. And I would actually classify the long-winded co-design process that Google and Sidewalk engaged in in Toronto as a mirror inefficiency. Like, it was clear that nothing that came out of those meetings was ever going to have an impact on anything versus other types of meaningful inefficiencies that might have been implemented, like at the city council level or other levels. Thanks, Mal. Hi, Eric, great presentation. So my question is more in terms of the general reception of play or what your experience has been in terms of the reception of play as a concept. And I first was thinking about this when you showed us the blob. And then also later, you know, when you were showing us the representations by the artist. And when we were looking at the blob, there were predominantly, I mean, there are people of all ages, but it did seem to focus much more on children or maybe young adults. And I don't know if that's just because it was this really big colorful object, but I think that that's part of kind of what this question is about is the idea of play, is that stratified by age? And does that ever come across as do people kind of trivialize the idea of play? And is that ever a roadblock in kind of how you're trying to facilitate these conversations, particularly when you're presenting the images I think of the artist to the people you're interviewing? Yeah, thanks for the question. Yes, I've never had to convince, spend any time trying to convince a child to play. So children play and they see a thing that appears like a play object or even one that doesn't and they will play. Adults, not so much, you know? And so for adults, there is a stage of convincing because one of the things that play does is that it levels the playing field. If in play conditions, there's supposed to be equality or at least in game conditions. And so what's interesting here is that, so I've done a lot of projects that involve play that have sort of high level decision makers. And some of these projects that have happened in say like in countries in Asia where there is much more of a sort of strict hierarchy involved in how these things take place. And what's interesting there is if you have a boss with an underling in a situation where you introduce play, the boss is usually very unwilling to play. Because that would suggest a kind of giving over to the possibility that they may not be on a higher level than the other people that they're playing with. So, and I think the same is true for generation, you know, it certainly plays out generationally. So back to beta blocks, there's a lot of things that we learned in putting that blob out. One is that kids are attracted to it, would come over to it and the other is that adults were suspicious of it because they thought someone was gonna sell them something. So it was, so people would sort of very, they would be trepidatious, they would walk, they would kind of look a sconce, like that's interesting, I'm gonna keep walking, not gonna go over there. And then we learned quickly actually, because at one point we were out there and we, stupidly, I had, you know, me and my team were out there with a clipboard, you know, it's like, what were we thinking? Like we were standing in front of it, like with a clipboard to take notes. And like, of course people saw us and ran the other way. So we got rid of the clipboard. And then the other thing that we learned is that the blob was very programmed. So there were actually stations around the blob, we had designed it for like activities in each of the stations and we thought that was the way to go. And again, people would look over and they would see that there was a thing and they would be a little bit unsure about how to engage in it. But then one time, we were actually at Fish Pier in Boston. Blob was there during a, strangely, it was during a fish festival, not fish, the van, but fish, the creatures. And so there was lots of fish and food. And then there was our blob. And people would come over and talk about technology with us. But the most use it got was when we had cleaned up our stuff and we had moved all the tech out of the way and we had, we had moved it into the van and the blob was still there. And then it was an unattended blob and then people were climbing on it and jumping on it and like jumping through it. And it was like all of a sudden it had a completely different use value, right? And it became a pure play object. And then people weren't afraid to move up to, even adults weren't afraid to go up to it. And so I think that was something that we learned about. Again, play is something that, children are willing players, adults you have to convince. But when you're introducing play into a space where it's unfamiliar, then you really have to design for that uncertain, the kind of uncertainty that comes with entering into an unsafe space. So that's been incredibly intriguing. Other times I've experienced this thing, this concept of sort of generational divides is that creating conditions for youth to work with adults, right? When adults feel like they're willing to play as long as they're enabling a child to play. But if they have to be the primary player they're less comfortable with that. So they'll step away from the play experience. So that kind of intergenerational connection has been a really important learning experience through some of my design work. Thank you. We have a number of thoughtful questions in the QA. And I could read them out loud, but I thought, Eric, can you open the QA easily? Is that in front of you? Because if it's not a problem, it might be better for you to quickly look at them and answer them directly. I was gonna say it might be helpful to read just because the future viewers of the recording. Oh, okay. Sure, then I'll be happy to read them. I just didn't want to sort of like, yeah, yeah. So the first question is, how does the move to a civic smart city and the forms of data and information flow around public policy and decision-making sit with parallel developments in media architecture by urban design firms, perends the physical structure of cities? Are these comfortable alliances? Parallel developments in media architecture by urban design firms. I mean, I guess I don't, I'd have to understand the example a bit more. I mean, certainly, I mean, I want to as much as possible use play here as a generative constructive space, right? Like that's how I'm understanding play. Now, often play is used as aesthetic, so we can imagine sort of the design of sort of you know, kind of Silicon Valley office spaces, right? The sort of play aesthetic that gets used as sort of a theater of fun. That's not at all what I'm interested in. And I think, and I don't know, Eric, if this is what you're referring to, but I think that there's certainly in the architecture and the media design or the urban design space, play is an aesthetic. And I think that that differs from the kind of institutional in the kind of institutional logics that I'm trying to direct our attention to. So I wouldn't, so I guess to answer your question, I think that is an uncomfortable alliance. And I think I would sort of point to one being an aestheticization of play and perhaps another one not. And maybe this would be a better conversation if I had, if we had an example, and maybe you can put one in the chat, Eric, to talk about, but we can move on to another question and come back to that. So Barbara Bulk asks, what would it take to create connective tissue in Boston for beta blocks to achieve lasting transformation? Ooh, that's a tough question, Barbara. That is, you know, it's funny. It's with projects like this, right? That are prototypical in nature. Where, you know, I think what it would have taken is a different approach from the very beginning. And my colleague who I think is here listening in, John Harlow will, you know, we work on the project together and one of the frustrations from the beginning was our, was the lack of ability to sort of connect to other offices within the city. And so understanding the procedure of institutioning is something that needed to happen earlier on. And I think what's interesting is that now that this project is completed in Boston, I actually have more faith that it'd be picked up by other cities because it becomes kind of representational in perhaps a productive way. So, you know, in the city of Miami is interested in this because it's a blueprint for them and they can say, okay, here's how, well, let's start again. Let's use this blueprint and see what we can go with this. Whereas I feel like in the city of Boston, it might be harder to do that at this point. So, that's my sense of that. And it's also, it's interesting, again, if you've done projects like this before, sometimes they function like conceptual art pieces, right? Sometimes they, sometimes at the end, the documentation of the project and hopefully the relationships built and the good faith built become a kind of conceptual model that sort of energizes discourse, right? And I feel like at this point, and I don't want that of any project that I don't set out to do conceptual art. And, but I also know that sometimes that's what happens. And especially within university environments, you know, we're good at that. We're good at turning things into conceptual art, maybe. We're better at that than we are the other. And I'm more interested in the other. I'm more interested in the finding paths to persistence and institutional transformation, which is the goal of the work. But I'm also becoming more comfortable with sometimes we'll pull out of it with some conceptual art that can have an impact beyond the original location of the prototype. So that's, so I think it's a complicated question, but that's as best as I can do at this point. Thank you for that question. We have from Nicole West Bassoff. She's a PhD student in public policy and SDS at Harvard and works on smart cities. And she said, I often find myself asking questions about what makes it possible to say no to public-private partnerships or technology installations versus what it means to bring people to the table to participate in decision-making around planning and implementation. An example of this contrast was made plainly evident in the sidewalks labs case, as well as, for example, Amazon HQ2 in New York. How do you see the role of play in putting no on the table versus in widening participation or just slowing things down? Oh God, good question. So again, I think this goes back to Mal's question about the political power that's involved when a Google or Amazon comes to the table. Getting to know seems impossible. However, however, I do have faith again, and this is why I'm a municipalist. I'm interested in cities. I think cities have, there's a lot to gain from learning from cities. I do think that there is possibility for cultivating decision-making processes first. So I'll point to one example in Boston actually, which was the Olympics. When the Olympics was coming to Boston, there was a pretty significant anti-Olympic movement here in the city that did say no to the Olympics. Now that's different than saying no to Google because one is a more direct transfer of cash than the Olympics, but it was definitely a sort of revenue stream that was rejected. So I think it's possible, and that's one possible case, but what I guess what I'm saying is that if we had a clearer foundation of ways in which municipalities can effectively partner with communities and neighborhoods and groups, if we had that worked out, again, I go back to what both Andrea Campbell and Julia Mejia have said in their own governing, which is, how do we create a city government that not only looks like the people that it's governing, but actually has a different value structure from which to govern, right? That's really what this is after. And can we do that by building the mechanics of governance in a way that is more participatory, that is more collaborative? Now, this happens through mechanics, and so this is what we're doing in my class is that we're thinking about these sort of mechanical elements of co-creation in government. And sometimes it feels really small, right? Like we're looking at a particular place in which we can intervene in the process of decision-making, but this is all about building models for to change that logic, right? So again, if the model typically in the way that cities say no is through protest, where there are decision-makers, then there's a protest, there's something that rises to the attention because of masses of people taking to the streets, and then officials are pushed into changing their decision, like that's often how governance works is that there's like here we're gonna do, we're gonna make a decision, and then uprising, and then okay, we're gonna make a different decision. That's not very effective, right? Now, it works temporarily, but as we saw in Minneapolis, it doesn't create models for actually persistent change. What it does is that there's a lot of lip service paid to the acknowledgement of the value transformation, but we don't have the structures in order to make decisions differently. We don't have the structures to make decisions differently about policing, let's say, either in Minneapolis or in Boston. So again, the city of Boston is also attempting to reform its policing structure, and we can do that through policy changes, so we can make different rules and we can make different policies, but until we change the way that policies get made, we're still gonna find ourselves in the same old spiral. So really good question. I have to believe that there is a path to know, but I think that that's a long haul, and that's a really sort of focusing in on government structures and changing the way that happens across cities in the United States, and then also learning from cities around the world, like Barcelona is a leader in this. You have a question from Hamid Reza Nasseri. Thanks for the great talk. I'm curious how you think Allende's Project Cybersyn in Chile connects to such multi-sector collaborative efforts in governance. One, does that project even come up in the conversation in such co-design settings? And two, what do you think about how that groundbreaking, but unfortunately unfinished project can inform these more recent efforts? What a good question, and no, it doesn't come up. It doesn't come up as often as it should. You know, I don't even know how to respond to that question because ultimately what you're asking is how can we learn from that project, and maybe the answer to that to your question is similar to my conceptual art answer, right? Is that we learn from these things by through storytelling, right? This is the sort of generative discourse of these projects. And all too often these projects will presumably fail, and then they'll get talked about as a failure, right? So here's another problem, like you have a project that happens, it fails to do a thing, and two things can happen. It gets buried because it has failed to do that thing, or the people who are doing the project can't talk about failure because they wanna continue getting funding, or they wanna continue getting support, or do more business. So there's just no room to talk about failure. So we never get to talk about why things don't work. We only get to talk about why things work, and so we don't learn from things. So the Project Intelli is a great example of a project that failed in certain respects to meet some goals, but not in others, and it doesn't get talked about enough because we often don't make the room to talk about projects that can function like conceptual art, that have worked in other ways, that have created relationships, dialogues, networks, that have done all the soft aspects, I guess, if you wanna call it that, of this work, but we don't have the sort of business language to talk about it in most settings. So it's another reason why I think that the sort of, if universities can find a way to sort of move the critical discourse out into other spaces and make room for failure, I think that's another wonderful function for universities in this public discourse. Thanks for, thank you for that. I think the one last thing, unless people still, they'd like to raise their hand, there's still another moment or two, but Eric did expand on his question, the first question. My question is a bit about trying to think of cities as built spaces that have to be undone to develop relationships with their constituents. I wanna think of cities as built spaces that have to be undone to develop relationships with their constituents. Yeah, so it's not a question that you have to answer. Thank you for saving me, Eric. I mean, it's an interesting question because it's true, I mean, it's one of the problems with the way that cities get built is because it's expensive and they get built and there's hesitation to unbuild things that are built. And so I think that it is an issue and this is why I think there's lots of opportunity in thinking about media spaces or thinking about sort of soft urbanism where there are possibilities for transformation that don't include bricks and mortar or breaking ground. So that's another way to think about cities. And again, like beyond the built environments, I think we have to think about cities institutionally as well, which is another thing that doesn't necessarily require the raising of buildings to transform institutions. It might, I mean, some buildings need to be raised to transform institutions, but I think that mostly they don't. Tomah, you have a question. Oh, I think I'm better. I wanted to ask you first. So I think you're the, I think you're the only... Sure, okay. I'm better. Tomah, well, one first and then the other one. So I was, thank you, Eric. I assume that different communities have different needs and when you like do these like workshops in different areas, I'm wondering how the government can fulfill like different needs in the communities or which like, how can they decide like which things can like function for like different communities, let's say. How can governments decide what can function for different communities? Like when you create like these different workshops, I assume that different communities have different needs or like prefer certain texts, but like, I guess government doesn't have enough funding to like, you know, fulfill different needs. So I'm wondering how could they decide which ones are the ones that they will pursue? Yeah, I mean, I think the answer is that they shouldn't be deciding on their own, which one to pursue. And this is another reason why it's so important to find mechanisms of collaborative governance, right? That when governments are left to make a decision alone about where the need is within any kind of place, those decisions tend to be bad. So when there's opportunities for collaboration, then it is about sort of daylighting some of the concerns that government often conveniently ignores and that, you know, again, sort of brings that to the table. So I'll bring it back to Kluge. So in Kluge, you know, the most marginalized populations in Kluge are the Roma and Hungarians. And, you know, and they have largely been systematically excluded from governance of all sorts. And in fact, the sort of bias against them is somewhat accepted, right? And so it's, you know, in some ways it's different than the discourse in the United States where, you know, again, at least in appearance, those biases are not publicly acknowledged most of the time. And so in that case, in that case, what that requires is that it requires the resources and the effort to bring people to the table that have been systematically excluded from that table. It requires will from, it requires political will from government bodies. But once, but more so than that, it requires the mechanisms to accomplish that. So for example, in Kluge, you know, the mayor is invested in building, integrated housing for Roma populations, which was pretty radical for Romania. And so like that example is a really sort of interesting one. And now coming from the mayor, it's one thing. It's actually building that's not necessarily gonna get it done. What's gonna get it done is actually putting in the resources to bring those impacted communities to the table. And that's what's beginning to happen there. And that's what takes time. And that's what requires that sort of, you know, that level of trust where, you know, if you've been asked to come to the table over and over again and every time you come to the table, your finger gets cut off, you're not gonna come to the table anymore. You know, so if you have a new, if you have new leadership, then you have to slowly build that trust with people to say that, yeah, we're inviting you to come to the table and we're actually gonna feed you this time, you know, like it's a completely different relationship that is being built. So I think it is political will. It's commitment to develop, to commit resources. And then ultimately it's about the sort of long-term trust building that's required. Amber, sorry, I hadn't seen your hand. Okay, Thomas, it's your turn now. Thank you. So I'm wondering if we think of collaboration as the way for institutional change. I wonder if we should rethink also a delegation on accountability too. Yes, we should, Thomas. We should rethink. So now again, accountability is an interesting question. Accountability tends to suggest a personal accountability, right, that one is accountable for their actions. And in that sense, I think that there's, you know, really interesting literature that looks to sort of transferring accountability to responsibility. Responsibility tends to be more, implies a kind of reciprocity, right? That we're responsible, that when we're responsible to something or to someone, that's different than being accountable for our own actions. And I think, you know, that governance needs to be able to, like, yes, we need to rethink governance, but we also need to shift accountability to responsibility. It's not about government being responsible to the governed. And it's not about the governed being responsible to the government, but it's about a mutual responsibility that is absolutely necessary. And again, a mutual responsibility to one another. And this is tied to, you know, the other thing that I didn't really get a chance to talk much about is the frame of a care ethic, right? That, you know, if we start to sort of shift, you know, if we move some of this language to, you know, away from, say, impact and more towards care, right? Then we're realigning priorities by assigning a different kind of value to the work that's happening in cities. And if we think about what happens in cities as a kind of mutual network of care, and we're seeing some of that bubble up because of the pandemic, right? Whereas care is kind of rising to the surface as something that people are actually talking about in cities. And so can we transfer some of those bubbling up actions to the values that are gonna be guiding the systemic change of these institutions? And I do hope like that's that we can start moving a lot of that language into a different space. There's a book that I haven't, I only read the first couple of pages of, but it's the Care Manifesto just came out. I don't know if anybody's seen that yet, but it promises to be fantastic. I'm gonna recommend it before reading it. So there you go. Thank you. Thanks for the question. Thanks. I think we, obviously we're coming near the end of time, but we do have one more question in the QA. So I thought I, for those who can stick around, from the perspective of municipal government, how should they measure the effectiveness or performance or otherwise evaluate the products of meaningfully inefficient projects? And is there a risk of missing the point? Yeah. You're doing that, Larry. Yeah, what a great question. Yes, indeed. And this is how like in a culture of evaluation, and in a need for measuring impact, yes, we have to change the conversation about this. I'll bring up, we, I developed with colleagues a tool called Meter, which is M-E-E-T-R, and it's M-E-E-T-R, I'll put it in the chat here. Actually, here it's, so that, so I developed a tool called Meter, which is specifically about evaluating different kinds of questions. And so in what Meter does is that it looks to, it looks to create a kind of reflective evaluation process for practitioners so that they can understand along two primary axes, that they can understand their work along social infrastructure, building strong social infrastructure and longevity. And so, and we do that by breaking down four activities, and these are described in the Meaningful and Efficiencies book, but four activities, including network building, holding space, distributed ownership, and persistent input is what we call them. And these activities are, as we describe them, these are the, this is the work that people are actually doing in order to accomplish the kind of goals that I'm describing. And so the challenge is we're not gonna get anywhere if we continue to measure the same things that we've long been measuring, that we have to start measuring different things. And sometimes that, what those things are, are these mostly invisible practices that people are engaging in, in order to transform their organizations or their places. And so, yeah, so I'm very interested in this problem. And like I said, we've developed this tool, this is being used by journalism organizations. It's being used by folks in cities. I'm actually, we're all using it for some of my active research projects. In fact, in Cluj, we're having the practitioners that we're working with use the meter tool to self-evaluate their own practice so that we can begin to get at a different kind of metric for understanding the value of this work. So I think along with sort of changing the language that Tomas was talking about, in order to change the language, we actually have to change the criteria by which we evaluate these things. And so it has to be a parallel practice. So I appreciate the question. Well, thanks everybody for coming. Obviously, Eric, I thought this was great. And clearly there was lots of interest in what you had to say. So really glad you were able to do this and thank everyone again for coming this week. Look forward to seeing everybody next week, same time. Thanks very much. Thank you everyone. Bye-bye. Bye.