 Welcome everyone so urban outfitting civic tech for the city context. I'm afraid we have to start with some sad news which is that one of our speakers Jose Alberto Gomez is not able to make it yet a last-minute incident. Following that with some happy news that will give us a little bit more time to dig into the presentations of our other two sets of speakers who are going to be from OECD Antonio Kenyamas and Fabrice Merta. Antonio is from the city's urban and sustainable development group in OECD and Fabrice from the statistics directorate that's right. Correct. And following their presentation our other speakers are from the World Bank US branch Claire Davann and Stephen Davenport. So let's we will have up to 30 minutes for each of those presentations and then 15 minutes together at the end for any questions and discussion. Let's start with Antonio and Fabrice who are going to speak on developing an urban barometer. Thank you chair. Good morning everyone. So I'm going to introduce this very promising OECD project called the Urban Barometer but Antonio will do the bulk of the presentation. With my well-being hat let me simply introduce what is the policy context. There are now many well-being initiatives both at the national level. New Zealand for instance is taking the well-being issue very seriously. There are other national level as well as some national level initiatives in the UK, France, Israel and many other countries and the SDGs can also be seen as the global equivalent of those well-being initiatives. What all of those initiatives have in common is that they are human centric. They really put people at the centre of policy action and putting people at the centre means two things in practice. It means looking at average outcomes across a number of well-being dimensions, income, health, education and so on but also looking at inequalities in those various outcomes. So income inequality, health inequalities and so on and so forth. Also following the publication of the Stigridsen FITUSI reports there is growing recognition that both objective and subjective outcomes matter for a good design of public interventions. It turns out that inequalities in the objective outcomes are in general well documented even at quite granular level thanks to the rich statistical efforts and the wealth of administrative data that local governments can use. However special inequalities in subjective outcomes such as subjective well-being or people perceptions of public services are largely undocumented. They are of course available at the national level and sometimes at the regional level but generally they are not available at a more granular level such as municipal or city level. However they are very important, they are very important for policy making. For the following reasons they can help prioritize public investment in terms of where do we need to invest targeting the most disadvantaged local areas. This is important to promote inclusiveness, reduce social tensions at the local level and build social capital locally. There is also an efficiency argument by which economic flourishing of disadvantaged local areas needs to be boosted. And finally it's also a way to engage with citizens and build trust in local governments at the very local level. As I said local governments generally have a rich data architecture already on objective outcomes, income, jobs, firms, health, but there are scarce and subjective well-being outcomes. There have been some survey attempts in the past such as mappiness but in general their scope is quite narrow or they have been operated at quite a reduced scale. So what we want to do is actually fill this gap and build a survey whose scope would be sufficiently large and whose scale would be sizable as well. And we believe that local governments could benefit from this kind of database to assess people's perceptions and well-being in order to improve policymaking at the local level. I will now pass the floor to Antonio. Thank you Fabrice and good morning everyone. Well I'll present a bit more in depth what this OECD urban marometer would be and well I'll start with the why. I think we are now all convinced by Fabrice's excellent introduction that there is a need on more data at granular level particularly looking at subjective areas of well-being and what we suggest at the OECD is to develop a tool that can help us collect this data at metropolitan level but also inside the city to look at inequalities within different parts of the same city. So what we propose is two things. One is to look at what already exists there. We don't want to reinvent the wheel and we can use already existing social data. Fabrice put an example of an initiative called Mappiness that already does at to a certain extent this collection of data of subjective well-being but we believe that there is a place to develop an innovative digital survey tool that can help collect that data directly from citizens with whom and this is where the civic engagement part comes into place is that we would need citizens really to look at our survey and give us providers the the data through innovative sources. So what would be the potential data sources for an OECD barometer? First as I was saying is the use of traditional surveys is introducing subjective aspects of well-being into existing surveys. For example in the province of Córdoba project I've been very involved in we redesigned a household survey that the province in Argentina already had in place to introduce questions that could help gather subjective well-being data from citizens. Second there might be place to develop an innovative survey and low cost that is very easily accessible to citizens and that could be distributed through some social media platforms such as Facebook, Twitter or others. And lastly is to develop new partnerships. There are now actors that produce data and that traditionally have not been involved in supplying that data to local governments that local governments can actually use. Many regulators have already data on citizens' perception on public services or even service providers directly who could supply that data. So what we've done is in a partnership with the between the OECD's and SPO and Facebook we have already done a pilot test of this innovative survey and online survey. It was it was conducted in the city of Paris in four around this month and five municipalities. Around this month meaning inside the city of Paris municipalities outside the core city of Paris. So the pilot study yielded around 3,000 responses in this in this four around this month and five municipalities. And what we observed is that actually there are very very significant differences in life satisfaction between different areas of the same city. For example, we see a large difference between a traditional municipality that has good well-being outcomes with Bobigny. These differences and we cross-check that with the Gallup survey is as big as the ones found between Ukraine which is one of the low performing countries in this survey and Switzerland. So as we can see there are really really relevant differences within the same city. Secondly what we found and that could probably explain this well-being gap is all the all the circumstances around citizens and their daily lives that really affect life satisfaction. For example, we found a very we find even larger differences in the satisfaction with public services related to access to green areas, public parks or even the feeling of security but also on job satisfaction and satisfaction with housing. So and just to conclude on what we are looking to do during the next month is the we see these looking for partners it can be foundations, it can be local governments also the private sector to run a pilot test in in five cities and we aim to cross-check if this online survey is really accurate and for that we would need some cities that already have surveys looking at subjective well-being to cross-check the results of those surveys. So thank you very much. Thank you Antonio and Fabrice. As I said we'll have a joint session for questions and comments at the end of this session. So let's welcome our next speakers from the World Bank. Claire Devann and Stephen Davenport speaking about the open government partnership multi-donor trust fund research on the impact of open government participatory democracy. Can I bring up, do stand if you would like. Do I need to turn this on? Yes please. Hi Steve Davenport, World Bank and my colleague Claire. Quick question before I start is does that raise your hand if you know what the open government partnership is? Okay great. So I can probably, so open government partnership just for those that don't know it's an international platform or partnership with 70 plus governments and civil society partners that every two years make open government commitments. They started around 2011 through its life cycle and determined the steering committee decided that maybe it's a good idea to set up a set-aside funding mechanism to invest in potential open government implementation of commitments and research activities. So they approached the bank, the bank then set up this trust fund and then we're at the stage now. So in Pretoria in South Africa the idea was identified and then now in 2019 took us a while but we finally got the trust fund on its feet and we're in implementation which is one of the reasons we're here is we have an interest in public participation participatory democracy in the areas of research and what better conference to have this conversation. Steven apologies for interrupting before we go any further could I get you to stand on the other side so we can share your talk with posterity outside the room as well thank you. So really the and I'll go into this in more detail but I'll probably try to skip towards the end where we talked about research in particular but along the lines of the principles of OGP the trust fund is focused on again country-level commitment implementation because rates were low people were making commitments but they weren't getting implemented. We'd like to build the research base on the impact of open government that was determined as a gap that we'll talk about today and then provide support to the OGP itself. It follows that mirrors the two principles of the OGP. First is OGP actors in the driver's seat. Every proposal or kind of award that comes out of the trust fund needs to go through the multi stakeholder forum or the mechanism locally between civil society and government that agrees on commitments and implementation and then it has to be an equal representation of civil society and government and we have a governance structure to the trust fund that ensures that we've got that parity. We've got a pretty good representation 31 countries and nine locals so 31 national governments and then nine subnational local governments that are part of OGP that are eligible for this money. I can give you the full list so this is a bit verbose but the intention is to share a copy with everyone so they can follow up because we do have a call for proposals coming up and I wanted everyone to be aware of that. The reason it's taken some time is we were in a fundraising mode so we got commitments from the French, DFID, and Global Affairs Canada and now have secured the funding for implementation and it resulted in the following three windows of funding all of which are at different stages of implementation. Now country support that's directly targeted at at funding implementation of commitments on an OGP action plan and the co-creation related to national action plans so that might be of interest to you. The one we're going to focus on today is the research and thematic priorities so there is an amount of resources for impact research that I'll talk about in a minute and then also communities of practice on particular themes that are about to be released so open contracting beneficial ownership topics like that will be a kind of a global working group or community of practice within OGP to share knowledge and then also very specific research awards and then the last is the support to the support unit directly. Examples of the country support again it's implementation of commitments so and this could be anything that typically would range from fiscal transparency to participatory budgeting to beneficial ownership to open contracting so this would provide $500,000 to governments over a two-year period to implement a commitment. Co-creation is a big thing OGP does every two years every government has in civil society have to reinvent their national action plan or carry it forward. We will have resources that we've already issued a call and awarded a first round of co-creation and we're about to do a second one that's roughly 25 to 75k for civil society organization to kind of facilitate the co-creation process. We also have a window of countries that are about 50 percent or higher in terms of eligibility score to join OGP there are some resources to help a government to become a member and what's relevant to the audience today the window one is the largest is the largest window it might be of interest to you in terms of involving some of your civil society partners or government counterparts in civic tech reforms but the one that's I think the more germane to this discussion is cross-country research and thematic priorities so the one that's the most relevant at the moment would be what's coming up in May which would be a research grant call for proposals up to 150,000 to to focus on participatory democracy and do impact studies and that would go typically to a CSO research institution a think tank potential partner some of which you might be interested and then what we're about to announce in the coming month is five to eight thematic partnerships so these would be a two-year award for a research institution or a think tank to run a community of practice or a thematic partnership on a particular topic and that would facilitate global workshops knowledge and learning peer-to-peer learning etc very specifically when we started this trust fund we needed to map the landscape of who's doing what in research impact research on open government so I encourage you to go to open impact open gov impact dot work where we have this online map that maps all the potential partners j pal etc e-gap a lot of the players in this field and where they have focused and that's what helped us to identify who to work with but then also what areas to work and that's where public participation was obviously a gap that we wanted to fill in terms of research again it needs to follow the ogp principles have some replicability facilitate it has to be tied to some commitments it needs to go through the msf or the local forum for approval but it falls into the two categories the research itself so that would be again to a think tank or an academic institution to do a one-year exercise and then the thematic priorities which will come out very shortly the research question that we zeroed in on what impact does public participation have on improving policy government responsiveness and or accountability in ogp countries there will be a call for a proposal that'll come out that will be clear on this topic and there's also some flexibility it needs again needs to be tied to ogp commitments it needs to build on the evidence-based it's roughly and this is a indicative it's roughly 150 000 over over a 12 month period what we're particularly interested in is national ogp commitment on on public participation and then what is its impact sub-nationally and what is the variation across sub-national units or in this case sub-national regions or municipalities that's the that's the research question but again we're going to ask for there can be some variation there and typically this would go to a non-government agency to do the research in designing this window in particular we came up with eight principles i think there are eight um and we worked with several global partners on trying to identify the best way to to do this work in a different way so again embrace complexity things are complicated i think we all know that but embrace complexity avoid testing solutions some of these some of these approaches don't travel well so this is another way of saying context matters which i think we're all maybe a little tired of talking about co-produced with policymakers the ideas why don't policymakers take up some of this research is because we don't do them do the research with the policymakers so the we're encouraged to work with governments directly draw from a menu of instruments so rcts are what people want to do but sometimes a mixed method approach is better um and use the appropriate technique for the appropriate research question approach multiple sources of data as we know there's an influx of data from all over make every effort to use different sources as they become available feedback loops let's um i know this is full of uh jargony terms but basically make sure it's relevant to the to the government car apart and make sure it's it's solving problems that they have and that if they have if they want to interact with the data and respond have that be part of your research process engagement with the client and be opportunistic so a lot of times we have a research question and we think we know where it's headed but another opportunity might present itself where you're actually answering another question and be aware of be aware of that and then focus on positive deviance a lot of times let's just highlight let's not focus on what's going wrong let's focus on what's working so again these are general principles that we're trying to enforce when we award some of these grants so we'll be really what it means is we'll be looking for multidisciplinary teams their experience in rct style research but also in the whole adaptive management adaptive process i don't want to say storytelling be honest uh the process is pretty straightforward we'll ask for an expression of interest initially where you can put together a small proposal of what you intend we'll then take a look at those short list them and then ask for a full proposal which will be technical and financial proposals etc in more details but that process will kick off in mid-may we'll ask for that expression of interest again gives you time also to look at the country lists of eligibility look at civil society and local partners that you might want to want might want to partner up with and then also engage the local ogp process to see how how open they are to some of your ideas then we'll do the short listing as a council and again that council is the bank and the ogp working together um then we'll ask for full proposals which is a more involved process evaluate and hopefully over by the course of the summer in the u.s of course we'll be able to kickstart it in august september so again this is a an opportunity i think for most of this for some of the organizations here to do something practical on the ground in terms of impact research on public participation and participatory democracy given all the conversations that are having right now i think it it might be something that some of you are interested in but again i can answer questions we have there is some flexibility it's not open until may we will be asking around to see if people have ideas on how to how to position this but um again i would reach out to ogp or to myself if you have creative ideas and we'll take them under advisement again we'll probably release the call for proposals before the ogp summit in ottawa which is at the end of may so we'll probably launch the expression of interest period before that so that we can have conversations with people that come to the summit about what our expectations are but anyway thank you i'd like to open things up to questions and discussion and if you could press the red button in front of you if there's one conveniently placed i've realized that's a little harder for people in the back that would be really helpful um i'd like to kick off actually with a question to you antonio you talked about um looking for partners could you say a little bit more about what kind of qualities you're looking for in partner organizations so i think what we are looking for is really local governments that care and that have the vision to include subjective well-being data in policymaking i mean for us at the obcd our role is to really advise governments on how to improve public policies so engaging in with cities that don't really believe or don't really want to use after that after we leave they will become barometer wouldn't make a lot of sense for us so yes well i talked about cities private foundations or other partners of course it would be would follow the same logic and institutions that really care about subjective well-being and that are willing to use the data after we leave for their own policymaking processes questions for other people i'm from integrity action and my question is to go to cd um what will you do differently to build trust between citizens and local authorities in order to get citizens to really provide you honest frank feedback rather than co-create yet another platform that might be perceived as a bit of whistleblowing our colleagues from my society have amazing tools that um you know complain about your potholes and all the rest but that they build that trust with everyday person so if i want to know about my streets i won't ever write their platform i will not respond to anything that the local authority does so i think i'm a sadder this morning in many race sessions such as in it as well so i'm very excited to hear from you that's a good point that was raised during the the committee the city committee earlier on i think and we take this concern very seriously actually the pilot on paris was presented was run through facebook so one could have some issues about the statistical representativeness and the selection of the underlying sample which is the reason why we want to have this cross validation exercise and check that our distribution of data and answers are in line with higher scale surveys that have been done in very few cities around the world so the the survey which was run through facebook was presented as an academic exercise implemented by by science for paris so i'm not sure that people would be uh that reluctant in uh in that uh in that academic endeavor but uh i think that we're really at uh at the start and we need to test several models in order to to be able to say what works what is the model that should be put in place several cities of this sort of uh internet platforms could they be used for for that purpose again with the concern of selection we we need to to explore that deeper it was enough in facebook so that people clicked and were redirected to another website were they feeling 20 question survey of like three minutes plus they had to enter the zip code and that's how you tell localize the the residency of every every response thank you thank you for asking my question i was just curious if the survey was actually on facebook and then what the information was about thank you very much for your presentation i want to do to please speak a little bit more to your definition of satisfaction and how people might have represented this to you in your research um because i my name is amina san some of the communities where i've had to work we find that um access to service is one way of testing culture but also the dignity or lack of it also matters to people so how do you be able to understand that to do with your tool and quickly thank you for the experience i just want to ask and and make a comment that i hope that um when this grant is made available that two things will matter one would be that um the world bank diffusion colleagues will be very interested in a deeply shared attribution um so if you give a grant to one group of people it doesn't matter that they're able to share the the success story across many fronts um in instances where there's always been a concern about the person getting the money being the one to prove they have achieved people tend to protect and we don't get to maximize opportunity second is as well getting the resources for the work it's also important for us to think about long-term sustainability of this organizations especially the civil society organizations how do they survive on a day-to-day basis and some of the grants we have same usually will give money for the work but not resources that help you reach your overhead demand my other foundation is learning how to do this a little bit more in addition to the work just but to show what to take on the you can go so in the field of subjective well-being there are several questions of different types that can be asked broadly speaking there are three categories positive and negative uh effects so we ask people whether they have experienced stress or worry or happiness yesterday life evaluation types of questions so we ask them to look back and in general on the scale of zero to ten uh how do they evaluate uh their life or how much they are satisfied with their life and there is a third type of question which is uh odemonia so sense of purpose and meaning about life in the literature the life satisfaction slash evaluation questions are by far the most used by people because uh they have received the highest degree of external validity if you want uh in particular uh pure happiness questions so positive or negative uh effects have been demonstrated to be extremely volatile and to follow uh more the the stock option or the weather than uh we know the bread and butter of what makes a good life in contrast life satisfaction does capture some structural determinants of well-being such as job quality income level health quality of health and so on and so forth uh odemonia it's probably a field that has been neglected and uh we could uh we could look at that as well so we are this we made this choice of focusing on life satisfaction as a non-com variable and we are also very interested about what makes quality of life from a public policy perspective so provision of public services satisfactory public services such as uh green areas uh access to to transport security with satisfaction with security services and so on and so forth we could have uh expended that that list and ask questions about uh satisfaction on education system or welfare services and so on and so forth the list is long and uh in another survey on the determinants of trust we have expended the number of public services that we have looked at and we could obviously make the same choice here except that we're a bit constrained by the duration of the survey which needs to be extremely uh quick otherwise uh we will lose a lot of people sure I just wanted to take a moment to respond to your comments I think in the case in the case of shared attribution we did have this issue with our co-creation grants we gave co-creation grants to several countries to civil society and we had struggled with them actually including other civil society you know who was running the show kind of issue so in our second round of co-creation we're going to emphasize although you are the recipient of the grant the process is inclusive and includes all civil society government so what we're trying to learn from from doing in terms of long-term resources it's a bit easier for us um as the bank to fund government sustainability as you know I have a call with a budget in Nigeria this afternoon with all the sun and team to talk about and their first concern in civil society country is improve funds civil society sustainability in the OGP process so no easy answer to that coordinating with the local foundations and local donors to provide civil society funding for the process is always necessary I think um in some cases in terms of OGP itself we found that CSOs are more inclined to take resources from the government if it's through the OGP process but again that that's very country-dependent country-sensitive so sustainability is always on our mind especially for CSOs the bank is not always the best at that so I have a question hi I'm Victoria I'm from MIT from Yoruba and I have a question mainly I think it goes for a larger sort of discussion towards the narrative of using data for policy making and my question regarding and the limitations of data data has certain limitations it's a temporal snapshot it's context-based but yet it holds a lot of authority in terms of like it's been received as neutral yet it's not and there has been some issues when like Paris Surveys and mapping and for example areas that are poor versus areas that are rich are mapped differently areas that have younger people versus areas that have older people are mapped differently there are some examples in gender differences for example in Wikipedia it's widely edited by men not by women and women and what my point is that there is always people who is hidden in the data it may be non-binary people breastfeeding mothers etc and if a decision is made using only the data the social differences are going to be calculated and I wonder if you are going to highlight the limits within the data that you are collecting for decision making to avoid perpetuating those differences I think this is an excellent point which touches again upon the main limitation of this kind of exercise which is the statistical representativeness of the sample there needs to be large enough even in this granular setting but will always be limited to unobserved variables that affect the participation of the people to the sample so with this point being well taken I think that the main issue that the data shows is the enormous degree of spatial inequality within a very small radius so Bobbini and Neuilly are far other distant by only I would say 10 kilometers still you have the same degree of inequality as the one found between the country with the highest life satisfaction Switzerland and the country was one of the lowest degree of life satisfaction namely Ukraine and we could have taken Laos as another example so to me I would use the data as a kind of warning system but the data in itself doesn't tell you how to doesn't provide any concrete solution for how to to build economic development and improve life satisfaction it simply provides some indications look in Bobbini people have very much concerns with their security as compared to to people from a richer neighbors so that could perhaps be one area of intervention but the data itself doesn't tell you how to intervene do we need to build more police offices or I don't know so it's a it's a way of illustrating the importance of the social problems that policy makers want to tackle for me the decision makers that hey this data has hidden things like there is this data has its limits and so informing them so that they know that the data is not as neutral and as powerful as they think because I think there's an important role in the companies that are doing this and informing the policy makers. Yes point well taken this is why we really want to have this cross validation exercise which could help us correcting the statistical bias ex-ante if we discover some regularities in terms of lack of representativeness in poor areas for instance we could make some assumptions which would be of course completely opened and spelled out in order to correct for these statistical biases. Service I use service all the time so I'm familiar with its pros and cons of all the easy things about service they're easy to gather and easy to find so I'm sure of policy makers back in the days is incredibly important but one of the issues that we see is that first of all the quality of data is not that great so we just clean down things that people can respond to bias we have a lot of problems. The second problem which I think is bigger is the idea of creation of data which is as a sort of research we do over time which is we ask the question and so how do we decide the agenda it's something we believe in or the policy makers are concerned about or something so just when we kind of get into your thoughts about whether you were considering using found data so what's already there and it could be social media could be something like that so rather than you asking people questions seeing what people are talking about and there's lots of research on this versus the United States about police violence and data mining and all that so instead of directly asking which is which means that you frame the issue which means that you also frame the response options which are only three or four problems but you see what people are talking about so you're asking questions versus about poverty so maybe poverty is not an issue maybe it's security so just your thoughts whether this would be also acceptable. Yeah thank you and that's a very good point and that's why the OECD would like to say that the process is as important as the outcome I mean what we do usually in this kind of projects and we when we have policy dialogues with cities or countries is that we have a sound involvement process of different actors so for example applying it to the case of developing an urban barometer we would partner with five local governments of cities we would then go there for a first trip where we have a scoping mission and where we sit around the table all kind of factors that would be involved or benefited by accessing new data and this includes NGOs the private sector service providers different departments rather than all our main contact in the city together with them and that's what we did in Córdoba is we decide which are the areas of well-being where they need or where they have a need for more data to guide their decisions no and that would shape at ultimately the with the tool which is the survey that would go online I cannot I can follow up on this actually I just published an academic paper looking at well-being in the US assessed through Google trend searches which is very interesting because you can have as you put it you look at what people say so you're completely open and then you can check whether that correlates with survey measures of well-being in order to calibrate your your model and identify the people mains concern in the US we found that job search was really the the key concern of people together with health activity healthy activities that is interesting a very interesting approach but it is also high highly time-consuming we devoted a lot of defaults and research into that and I'm not sure that this kind of statistical it is not viable at the as a statistical superstructure for policymakers to get a quick snapshot of the situation on the ground you see it's it's a tool for for academics which has pros and cons but it's not viable for for policymakers in my view I just wanted to follow up on that I guess I'm not sure I don't think I understand why it would be not valuable to have like the found data versus survey data which I think both of your points are kind of saying that data feels pretty faulty right like there's a potential of that because when you're self-reporting or answering a survey on his point about the framing issues but then the second point being when you're answering a survey it's a completely different feeling than if I'm just searching by myself completely unobserved like I feel unobserved when I'm searching data even if I am or not which is why like you know clinical trials right where it's like oh we put you in a room in a laboratory and there's a two-way mirror and you know that you have a two-way mirror but like it's okay I'm still going to go about my business whereas when I'm being asked a question now I'm thinking about my identity I'm thinking about what you think what you I know you were going to answer this you're going to be looking at these answers so I guess I'm trying to like understand how your view is if you have a smaller set of data that is very could have a lot of biases from either the framing or the answers versus something where it's something that people have found you're right it's time consuming but if it's a policy making decision that seems like a worthwhile investment versus making a quick decision that will ultimately have a lot of impacts on a lot of people that may be edge cases that aren't just the majority so social media data are time consuming to to cope with because they are extremely noisy so they have their own statistical issues and I can tell you you need to detrain all time series that you're you're looking at otherwise you could be completely random you have huge spikes and cliffs and you don't know anything about what drives them for instance it's just anecdotes the keyword divorce in the US when you look at the time series there's a huge spike at the time of the divorce of Kim Kardashian with the football player so this anecdote tells you that you need to invest a lot of time in cleaning the data and even after having done so you need to assess whether the the correlations that they give you with well-being is indeed something that goes in a plausible in a plausible direction so plus those data are generally not available at a very granular level and this is also what we're bringing on the table with this type of surveys we want to be so large that we will be able to enter within the cities it's not possible with this google trend type of data we add the granularity dimension to the thing at the cost of having some statistical issues but I believe that this is worth doing because we know that well-being outcomes are very very unequal at the local level and this degree of inequality has not been that well documented in the past for subjective outcomes it is documented for for life expectancy for instance you know that when you do three or four metrostops in London you lose five or six years of life expectancy but you don't know this kind of information for for life satisfaction for instance and that is worth addressing any further questions we have one of it uh which was great by the way um I was thinking that you said that you have a lot of learning from what work you do work with the trust fund is that something that's going to be basically um put up forward to people to be community and also react to the feedback or not and if so how do you think of doing that because for me it feels it was a lot of stuff you haven't seen but you haven't seen the what made it come the way it is so getting more research involved it's a good question I mean we have a learning program with the OGP they have a unit that focuses on learning um we are learning from our trust fund rounds yes uh we haven't really discussed public you know publishing that learning from the process itself but it could be something we consider that we'll be learning in the awards themselves as well but programmatic learning we could share for comment but we've been pretty knee deep and getting everything up and running at the moment but in the next year that's a good idea um to the the issue of um data and um challenges with representation I mean one of the we saw this coming so one of the reasons we're connected to OGP commitments at the local level is to be more problem driven um we also will be working with clients more directly in terms of policy makers so hopefully we would anticipate any misrepresentations from data analysis and um then we will also have uh given where the we work very closely with governments we will have access to some data that that they have that they're that may bring other data sources into play that are normal research companies so that's one of the value added of the bank is our access to that information so we'll see how it goes any further questions feels like we've exhausted um just remains for me to thank all our speakers and everyone who participated for a really insightful set of questions thank you all