 Special thanks again to Astrid. Thank you to the Open Justice team who helped me flesh things out. This talk will be a bit dense because I have a lot of the material to go on, but it's all being recorded. So I thought I might as well go all in. So open digital innovation in the public sector, what works, let's begin. First off, I'm Petria Montens. I'm a software developer with a law degree. I've been building digital services for over 15 years as a civil servant, as a digital startup, a data engineer, a freelancer, and currently as a contractor for the Direction Interunitaire du Numérique in Paris. In my free time, I also built digital services for the public interest. A Belgian unofficial journal that has 15,000 visits a month, I found an on-profit working on digital transformation of justice. And in writings, I try to shine a light on what the peculiarities of innovation by and for the public sector imply. But beyond that moderate obsession of digital innovation, I think I'm just mostly a regular person. Before we start, I wanted to explain what I understand and mean about the topic of today. There will be some startup like jargon, like agility and lean. I had to use those words because otherwise I had to define them all over again, but just know that when I use them, I mean them in their like most purest and original sense. And should you have questions about them, we can always clear things up later on. So what do I mean by open? Well, there's the values of transparency and collaboration tools, practices you may know of open data, open access, open source and whatnot, documented APIs. There are a lot of benefits working in open source, but it's also more than that particularly in the regard to the public sector. There's the article 15 of the declaration of rights of man and of the citizens. So I'm a Jewish, so I like that kind of stuff. It's from the French revolution still, but it still has constitutional value in France. And article 15 states that society has the right to ask a public official for accounting of his administration and transports to software. That means that software written by and for the states means the source code has to be accessible. It has to be open. So there's a lot of meaning behind the open word. Public sector, well, public sector is not the private sector, but public sector has a couple of core values that make it really specific. There's a value of continuity. So there must be no interruption in the service you provide. Equality, the service must be for everyone. You cannot choose the public you target. And mutability, which is adaptability, which means that a service from the public sector has to adapt and can stand still in regard to technological change. It has to be state of the art. That's the definition of it. Well, private sector can use those values too, but then it's the choice. It's not imposed. And these values mean a public sector can be merely analyzed or built with the usual concepts and constraints of a commercial service, for example. You'll of course understand that I'm a staunch defender of letting the public sector innovate from the inside, but that's not a matter. Innovation is anything that's new, be it a technology, an approach, a way of doing things. It's something precious because it makes things more efficient. It makes people lives better. But it's also something very fragile that can easily be lost and destroyed, ignored. And that's why it needs its own space and time. But I'll come back to the aspect of innovation later on. And digital culture, well, that's the use of... I see it as a particular context of working digitally with virtual tools, services, data. For example, there's the scale and speed of diffusion of a public service, for example. If you can serve 10 people, you can easily serve 100 times more with almost no additional cost. And each one of those 10,000 people can personalize their service to their own needs, tailor it to their own wants. And suddenly, those 100,000 people start expecting more of you and your service. And if you're not careful, tomorrow those one million people might sue you because you leaked their personal data by accident. Though there's a lot of unique challenges within the digital realm, but it's also filled to the brim with potential. And if you have the right tools and practices to make use of it. So the digital world is a whole new one. You can't just expect to call and copy old paper services and written protocols to the digital realm and expect it to be sufficient. I think there are a lot of people think because it's quite a hot topic for the moment. There are a lot of digital native ways of doing things that are more appropriate than practices and customs that can be traced back to the industrial evolution. So this part will be in three parts. This talk will be in three parts. Innovation by chance in the Belgian public sector, innovation by design in the French public sector and open and digital culture applied to a public policy problem. And as a disclaimer, what I expressed, it's only my point of view. And at no point I blame any individual groups of individuals because they themselves can only act within the bounds of a system or process and can necessarily act outside of it. So innovation by chance in the public sector, by chance I mean there's not really a planning to make, to build innovation. It's an outcome, but it's not the desired result. It's like ordering a new service from a menu but without necessarily realising the scope of your ambition, the needs and obstacles you might face. And as a subject, I'll be talking about digital proceedings platform for the Council of States, which I've built. It's a unique service, not because I've built it, because it's unique in both its breadth and its complexity. I've heard in years since about others, other digital proceedings platforms in Europe, but none of it come anywhere close and the service has been live for over seven years. First, a little introduction about what the Council of State is. I'll go right fast over it and here's the view of the backyard of the Council of State, for example. But Council of State has two main sections, administrative litigation and legislation. If you want to dispute the construction of a wind farm or a school board decision, you go to the left side to administrative litigation. And if you're a lawmaker with a new law or decree, well, the right side will tell you how good or how bad your text really is. And they call that advisory opinions for a couple of years, and they have finally remained public. There are, I can't go into detail of this fascinating institution. There are other organs that are equally important, but it had its own practices, its own customs and its own history, also. Speaking of history, it has a history of digital services because the Council of States is widely regarded within law circles as the premier institution regarding developing new tools. They had their first websites in 96, fully published all of their case law from 94 onward and kept on building new digital services over the years to facilitate access to doctrine, to the opinions, to the arrests. And that has created, for anyone interested in administrative law, a unique environment in Belgium because nowhere else there's so much in any and so and so easy access to the information. And they have a set of tools and services that's quite incomparable in our country and perhaps even in Europe because at one point the Council of States decided that they needed to provide an electronic proceedings platform. And I can't really remember the details but there wasn't many much study about it. It was just like, okay, we need this. There's been an ask about it. So let's start, let's get things rolling. And it was extremely ambitious for a small institution of under 500 people with a small IT service because we had no particular experience in providing a full two-way digital service. And they left the task to a junior developer. We had no particular guidance or exercise in the matter. But still some things, the Council did really well. It set up user groups so we could see each other multiple times in a kind of development period. We used iterative development cycles. So we started really fast. After three months, we had our first working prototype which the users then used forward and we went back to it and we iterated on it. And I think I rewrote every part at least twice at the end. And I also had a lot of freedom as developer to contact users externally, internally. And there was also no preconception about what the service would be in the end. We concentrated really on building functional software, working software. And once the software, the service was ready, we sat down and wrote the rules and the concepts of the service that would be for the users to apply and rulings also. Okay, that's good. And we released it in 2014. We released the service free and for everyone with multiple interfaces, electronic ID identification, electronic signature, et cetera. And the animation you see on your screen is the back office, in fact, which regular citizens shouldn't see, can't see it's an internal tool. But it's specifically built for the registrar who has the mission to handle incoming and outgoing documents. And it was built to allow them to manage all the cases because the service was built to allow for two to 4,000 active cases at the same time. And the service was a success. Once released, and despite the need to continuously improve and adapt the service, usage just kept growing. And during this COVID-19 pandemic, usage is even 100%. Every case is now fully electronic and there are over 10,000 active cases on the service at the moment. There were some Celtic stories during development because the council wasn't really prepared to provide such a vast and evolving service, to provide the support. I remember at one time I was on phone providing support for a Norwegian ambassador, for example, and I had to debug a bug because the system might go down and then suddenly there was someone in my office requesting me to come to his to change his 20 toner, for example. And a bunch of stories like that, because we weren't just prepared, we weren't ready to provide such a service. And I think we were very lucky it didn't crash in times like these and the service kept evolving for four more years after that. And these are three examples on the inside. That were built by listening to the needs of the users. In the center, there you have like the overview of dates. That's from a document because a judge realized internally that he didn't know the status of a case at one point. So we were in full discovery mode. We were, okay, we're going electric. What are the lessons you should go about it? And during the audience, we realized that the lawyer had just uploaded a new file just before the audience, because that was possible in an electronic proceedings platform. You can do it in paper, but in electronic, you can. And so we had to work on a new document. I went to those people and we spoke together. We said, like, oh, look what data do you need? And we put together this document. In first was a bit complex to understand. So we added a column with some details around it. And that document became like the missing link between the digital and the material world. And it ended up replacing the very big manila folders from the paper proceedings that moved around the Council of States. And it was very interesting. And on the left, there's this dashboard also in the registrar's office because people would phone in and ask, hey, I can't upload the file. Is there a problem with the platform? And instead of forwarding the call to the IT service, we installed small dashboards with health status indicators and new activity on the platform. And just could just look up and say, no, everything's all right. Or just, okay, there's been some updates. You just have to wait a couple of minutes. So that was all lessons learned during development that helped us going forward. And all lessons learned because the status information would otherwise be hidden to the people that were responsible to contact the users. We even had in 2015, we had a completely interactive dashboard tablet to provide digital access to people who came to the Council of State to consult an electronic prosignal file. And in the end, what were the reproducible approaches we used and it worked really well? Well, there was the user centricity, of course. The collaboration we could have with internal and external users, the iterative development cycles. We just, we started a bit small and then we grew over time, bigger and bigger and bigger, added new aspects, new modules. There was a lot of autonomy and a lot of trust placed into the development team. And there was a big focus on working software. But these approaches alone don't necessarily prepare the service to remain in good shape for the future because security environments, user behaviors, kept on evolving. Just last month, I realized, we realized that users uploaded movies, video files on the platform. It was never made for that. But users, people on the internet, are now accustomed to upload video files. And so they respect it. Well, why not upload it there? And so while providing the service, you continuously have to work on it. And in the end, it's literally, and that's a bit of a digression, but it's a state of mind about digital third service development. And you can find that in the mind everywhere. IT developments are often viewed as just as projects, which start at the moment and then they end. And once released, they are mostly abandoned because you did what you had to do. You check the box on your to-do list and you go up to the next project. But that doesn't work for anything beyond a particular scope. You end up with tools that grow old, technical depth exploded, the users grow dissatisfied in the best of cases. And such developments in fact, such services must be thought of as products, which start, but only end when they are put out of circulation. And suddenly you have to consider the whole lifecycle of your service, of the evolution and adaptation to change. And that's how you should build and maintain a digital service provided to the people. Second part, I left the council of states and I went into the innovation sector. First, as a private data, big data startup. And then I discovered a unique organization that's a state-organized startup like incubator. It's called beta.gov.fr. Simply put, beta.gov is a particular organization within the DINEM, Directional Internationale du Numerique, a service of the prime minister. And its goal is to improve, of course, the life of the citizens, enterprises, public agents, and improve the effectiveness of existing public policies and help build new digital public services or organizations, mostly by tackling complex problems who don't have necessarily a straightforward solution because if you have a problem with a straightforward solution, you just write the spec and you post it to the IT service. No, this is for complex solutions which don't have a solution right there. Beta.gov doesn't work on its own. Most of the time it works in collaboration with other administrations who wish to work on a particular problem. And so they detach a person which we call the entrepreneur. So someone who innovates from the inside of an organization. And that person will be coached, assisted, and will be responsible to make a team, to bring a team together and in charge of his project and he will be completely free and full autonomy to develop his service. And beta.gov is just next door to Etelab, which some of you already know. There's a lot of probability between the two. The two cells. But before going on, I removed a lot of digressions like this from the talk book. I really wanted to keep this one because it talks about the nature of innovation. Like I said, innovation is something that's very fragile. It starts with a rough idea and at the beginning it's very easy to destroy it or to just plainly disregard it. You have to nurture it and let it grow so it can prove itself. There's a book I can recommend, Loonshots about the subject. And there's this little graphic inside the book about the separation between your operational part of your organization and the more creative parts of it and how you must have an exchange between both. Like for example, if I want to illustrate it, if you want to test new things, new approaches, emerging ways, you need like privateers. People who go against the flow, they put things into question and they are eager to test new and emerging practices and technologies. Or they're just very motivated to make a problem go away. But as an organization, which has a certain size and specific mission, you also need sailors who follow orders. So you can concentrate on running not only your boats but the whole fleet. You need both groups, but mixing those groups is dangerous because at worst, if you let the privateers run wild, they would just bump into the sailors and they will be at each other's throat in no time. And at best they will completely ignore each other and nothing will happen. But if you separate them completely, the privateers won't provide any benefit to the rest of the fleet and any gold or treasure they'd find would be completely under-trapped somewhere. So there's a clear need to have some balance between both. You have to separate the groups but keep them close to each other and let some sailors become privateers, let some privateers come back to sailors. So there's an exchange of ideas, discoveries and treasures. And that's really the mission of an incubator. It provides a space of freedom, of discovery and experimentation, but close to where that discovery and experimentation, close to where it's needed. So it can be useful. And that's what it's all about. It's making new ideas useful. So what is this space of freedom and openness? It's a community, it's an open of autonomous teams within the incubator, very diversified people from all kinds of backgrounds. And the space itself of the incubator of the community is completely open and self-organized. What you see here on the pictures is one of the weekly rituals. It's called the stand-up. It's when every Wednesday, now it's Thursday in COVID-19 and at the distance. So it's the moment when all the teams gather and they share their news, they share a car for assistance, they call for help, they talk about what their actuality is. And they also talk about future investigations and possible investigations and possible initiatives. And what BetterGoof applies is very simple. In fact, it's just plain best practices from the startup sphere. It's data and impact-driven autonomy, freedom of the teams, agile development cycles. And I mean like real agility, like in the manifesto, not some other watered down version. And of course, user centricity. One of the core values is that the needs and wants of the users, those of the citizen, are more important than those of the administration. And some regards, that's the whole revolution in itself regarding to public policy, solving a problem of public policy. And this is the life cycle of a public state startup. It starts with a pre-encubation period, a call to innovators, a specific order that's followed by an investigation period. During investigation, one of the main objectives is to see if the problem is a good problem, if it's a problem we're solving, if it's a whole of the qualities of being applicable because, well, it should be, first of all, a problem that can be solved by digital means. So once it can be constructed in small, it can be easily scaled to the whole nation. And once the investigation period is over, you go before the launch committee, where it's decided if we effectively start with the construction of the solution. And during the first periods, while the entrepreneur assembles a team, starts contacting users, works with them, builds his first MVP, the minimum viable product. That's a fact that responds, that's just the basic functionality needed for his service to be proven. And even often, there isn't even a digital service at that time. It can be fuel concierge mode. So you just call and do the things yourself by hand, just to check if it's worth constructing. Because I saw, I thought it was data driven, impact driven, but there's also a certain value of being beholden to the taxpayers because the money that is spent in this case, in the incubator is taxpayer money. And we don't want to continue building something that would just cost money because that would also be our money. So we want the money spent to be well spent. So the service has to be efficient. And we also want to make a positive net difference. It just must be another service more that will just be another obstacle on some other point. It must really have some value for the end users. After a first period of construction, if the team can prove it has an impact, there can be successive periods of six months where we go before the investment strategy committee, where it's decided if we go or we go further or not. And in Italy, in the end, while your service has grown so large, it can be deployed to the whole of the nation and it can be integrated. It can be integrated in the original administration that pushed for it or it can become a fully autonomous organization. A lot of different things may happen. And even if the issue is not as happy and you don't succeed in attaining your objectives, well, there are always the lessons learned. There's a post mortem about what were the failings of the project and there was always things you have learned, people you have met. And because you built your software and open source, there is something that can be reused for other things. And often other teams and other projects and investigation starts on the remains of previous investigations. The community of BetaGroof has grown steadily over the years. I started in 2015, when I joined, there were just 300 people and now there are over 500. There are 80 digital services that are being deployed right now, developed. Of every service is completed transparent in its objective and results. Open statistics are fully available. There are multiple success stories. Some of them have become autonomous organizations. Some of them provide a crazy service like demotorizing any administrative process in a few minutes. Others are simulators for parliamentarians to estimate the impact of new amendments before voting them by changing brackets and viewing the impact on different populations. The incubator has also its own network because other administration starts to copy the initiative. There are about nine at the moment and more being created. And internationally, you have Germany, Canada, United States, United Kingdom and Singapore who also have those government-run organization who internalize and learn innovation from the inside. So you don't have to depend on companies who may not share your services, culture or long-term objectives. And so what worked in regard to Beta? Well, innovation can be organized and accelerated in the case of the incubator, but you need to problem, you need your team and you need to sponsor behind you. And the basic concepts, the formula for innovation can be inscribed in other administrations. And when you do that, the people with vision work, they are happy and motivated and productive because when they're put in a situation where they have a meaningful impact, there's a lot that's going on. And I recommend a book on that, The Bullshit Jobs, because David Grabler talks about the happiness there is in having an impact around you. And, well, those formula, can we reuse them in a problem of our own, for example. And I went on to discover that with a nonprofit open justice here in Belgium. And the road to open justice was this, there's an article in our constitution that says that any judgment has to be made public in accordance with the term said by the law. But that's not an old article, but at the moment actually, while the case law is produced mostly by the lower courts, the publication is to complete reverse. If you look for a case law online, you will mostly find decisions from the upper courts. And less than 1% of the decisions are actually published. And the message we had as a population, but we received from the media wasn't very glorious. And our own history, of course, if you know it, digital information of justice had its own fair share of issues. And so there I was with my experience and of open source developments and seeing these things going on. And I said, well, maybe we should maybe try to develop something in open source. And maybe that could inspire or like go at least progress on the issue. And without realizing it, I discovered a good problem. A good problem because it's a real problem. It's worthwhile. It's worth to spend time on developing because there will be a real achievement if you do it. It's timely because it's the right moment to work on it right now. It's also a problem that unites people. That was a great discovery because a lot of people were interested in working on this solution. And it's actually a problem. I can do something about it. And other people can do something about it because what do I know if not building a digital services? So what we applied? Well, we applied the basic best practices. There are people joining the initiative. So we had a team, we collaborated around the project. We did a lot of communication about what we did. We started small experiments, reiterated on it, we developed it. We were of course fully autonomous, because we were autonomous nonprofit association. There was a lot of users in Christie because there was a lot of users within the community which could directly ask and tell them what's your use and how would you use it and what can you do? We shared a lot of what we do and we tried as much possible to connect with other associations and see what they've done and how they could help us. How we could help them. And after six months of applying these methods, the allergies, completely in our free time, we are at our second MVP iteration. And so right now we have the complete solution to upload a digital decision, scan or see it, anonymize it and pseudonymize it. We have a complete and documented API and that API is used by one of our search engine that offers a 30, 60 degree view of the Belgian case law because it's not only the decisions we collected but also the decisions published by the Justice, by the Council of State, by the Constitutional Court and all of that, of course, is fully in open source. As a community, we also set up our own rituals and spaces. And that has been another interesting part of our journey in that people started testing and experimenting and discovering new digital services and that allowed them to be more effective in communicating and collaborating digitally. And we shared news, we shared opinions, contacts, all the while building and organizing a whole new entity. And we had our B-monthly meetup and other rituals. Like I said, we had workshops, we made goodies, we had a lot of fun. We had also had post-op, posted workshops which did not only were used to decorate our walls but they provided real insights about the things we were trying to develop. We are developing actually. And of course, there's our GitHub page. What we also did was, that was maybe the most important of what we did is spreading the word because it doesn't look like much but bit by bit it kept on increasing. And at one point there was this law professor from the University of Brussels who said on LinkedIn she would provide her law course. She would publish it in open access on the internet and her reason was one of the articles we published. And that was the moment I realized that like them we were starting to have an impact. We are affecting some change. And it was really a wake-up call for me in the end because in the end it's all about the people. It's all about the people. And if you want to say don't put justice you're easy to find because there's this feeling we're going to make it through a coming digital transformation and revolution really. And it's all about people and ideas. And the best I can hope for open justice the best I can hope for open knowledge Belgium and other initiatives is to become beacons for a more efficient and respectful way forward and use the jump, the cultural shift to make society more accessible, more equal, more enjoyable for everyone in respectful and meaningful ways. And that's what it's all about in fact. And to conclude this last part what worked for open justice is we found a clearly identified problem. We gathered a community around it. We were spread the word. We applied open digital culture practices and realized that when you realize that when you create a space for privateers privateers will come to you. And to conclude this whole talk I wanted to fence with these three points. It's that's because I'm just over time. An unplanned approach to innovation works sometimes but plant innovation is a real accelerator for the digital transformation even within the public sector. And it's an effective in an economic and all the while being respectful to people and motivating them way of doing things. And believe me with the experience I have you really want to work with people who respect each other and who are motivated by what they do and by what they all allowed to achieve. So thank you very much for your interest and attention. Thanks again to everyone for being able to see through it. And I think I'm just in time for the questions and free talk part. I also must note that in the shared notes I've put some links to some of the things that I talked about. There are links to the council of state to contact me because if you don't have time to ask a question I'll always shoot a mail. I'll be glad to respond. There are already a few questions in the chat, Pieterian. Okay, I'll go through them. What is the two-way services part? Well, the council of state is used to providing like I built most of those, some of those two just a publication service. Like, okay, we publish or doctrine but it's just one way. Users, we don't care for the user to come and look and most of the time we don't realize people are looking at our publications while in two way there's a real interactions. People publish stuff and you have to get this movement. And suddenly, well, like there are services internally from the council of state. You have like a couple of developers for a service for 500 people that works but you can't have one developer for a service for the whole population of Belgium. You see, because there's a two-way service. You can if it's a one-way service like I have my own service that has 15,000 views per month but it's one way. It can be 100,000 or just 10 won't change anything for me. But if it's a two-way service, that changes a lot, of course. How do you recruit, involve end users and sector public agents? Well, that's one of the problems. The understaffing for it today. What we went, what the user, the not necessarily from within the public sector like I worked in a problem for users with a handicap. And these users, we made like we organized an open lab where we put all the stakeholders together, the different organizations, the users themselves, the persons themselves. And yeah, we just fix an open lab for example. And that's maybe that's the answer to your question. There's just open lab where we reunite everyone. And so we block the day to have talks around tables. And when you put the different stakeholders around a single table, you realize that there's a lot of things going on because people talk about their lived experience. And suddenly there's someone just across them who can tell them, oh yeah, but that's because of that. And sometimes there's someone from another organization under administration that says, no, no, you misunderstand. You have to do it because of this. And then you realize there's a lot of miscomprehensions and then you start also to see with the people present how a new service could help everyone move forward. But you're right. You can't really rely full-time on public agents, people of civil servants. And that's why for BetaGoof already is when you're an entrepreneur, you're detached full-time. You don't remain partly in your administration or in the incubator. You go full-time in the incubator for a couple of months. Belgium universities are funded to do this innovation. Well, they also do in France. If you do, if you keep the innovation within universities, it's very too much far away. You won't have the very pragmatic and it can be interesting, but we really concentrate on services and production. Like what I did for the KZDPO was one of my projects. After just a couple of weeks, we had a service in production. We were part of a larger initiative. And we were the front line because we were months before anyone else, because we were that much faster and really concentrated with this start-up way of thinking, of providing a real service. So we started small and grew and grew and grew. And if you would have been completely, completely separated from the KZDPO, for example, there wouldn't have been enough extra time in the KZDPO, for example. There wouldn't have been enough exchange between each other. And another thing you should miss about that is that if people are kept close together, you also provoke a kind of cultural shift. Because one of the things I had in the KZDPO was we were working on all on the same floor. And some people were just completely taken aback by how much fun we had while constructing our own service, because we were a very effective team for the beginning, but we were working in full autonomy on our own service. We could take up the phone and call anyone we wanted. We could contact the users. And when we saw all the subscriptions coming in, all the people following us, and all the great returns we had, and very, very motivated people, because the people finally felt like they could help build something, because you also have to consider that suddenly the citizens had someone who listened to them. And that was incredibly transforming for them. And so there's also this cultural shift that happens when you mix people, privateers and sailors. They see like, oh, can you have so much fun developing a service in the public sector? Well, yes, you can have. And it's extraordinarily liberating for the citizens and for the people building the service. I also suggested, if Marina wants to comment, that you can speak up. Please, please do. Yes, so I'm French, so. But the thing is that in Belgium, the governments ask universities to make studies, to test scenarios, to create technology, but not only, or to propose new scenarios for the society with a big S to go ahead. And I think it's not really the case in France, or maybe much less. And I was part of the really startup movement in Brussels several years ago. And I really saw that whenever you wanted to go into innovation, if you hadn't had a university partner, then this was not, you couldn't access grants. You couldn't access really being recognized. And in a various or any regional subsidies would push you to collaborate with universities. That's my only point. And I know also that in France, they want to do a startup nation stuff. So that's why they put me there. Startup nation came after Petagouf. So there's just this discussion in Petagouf. Should we still keep the startup monitor? Because it's quite deserving, because it's not really what was behind Petagouf. Doesn't want anyone to be an entrepreneur, or even. And what important is what you say is Petagouf is only one part of the digital transformation shift. There's also Etelab, which is an entrepreneur that's another approach. And I do think they have a lot of think tanks, which do like what we do here with universities. It's also important to know that innovation from within is also very important. You empower the people. One of the aspects of Etagouf is to have people from the public services to be empowered and to grow new competencies to grow. In fact, in what they can achieve, they break out of their silo and they work with other people and they realize, OK, that is how you build new things. So it's not only just the part of building a new service. It's also like opening up horizons and letting people grow and empower them to affect more change. But you're right about universities. Open Justice has multiple links to other universities. That's what they realized also. But the core values, the core seeds of the approach, which are user's interest, impact-driven and such to build your service without the much fat as possible, those can be transplanted within another structure and they can grow there. They have the basic tools to reinvent and to appropriate those different aspects from within. Whatever the organization is, because I also know in Belgium, for example, a lot of IT is now done by state non-profits because the framework for IT people is so non-adapted within the public sector that you have to create a whole new industry based on non-profits so they could recruit and attract the necessary people. What I've lived with Etagouf is that when you create a space for innovation, innovative people will come to you. And if the public service wants to have people with ideas or want to enable his people to have ideas, that's one way to do things. That's where it works. An open lab, well, there are a lot of words we use. An open lab is really when a state startup has, like it's a multi-day event with different stakeholders from all over the place, from all over the country too, because France is very big and so you go to multiple places and you have an open lab with local actors and you reunite users, you reunite people from the public sector and you're there also and you try to think and have their experience and maybe think of solutions to their problems and obstacles with them and co-construct the future solution. That's an open lab is. There is also a question on how would you advise to evangelize an administration from the inside to move in that direction? That's a good question. I didn't show it, but there's actually, Etagouf itself started with a book from a think tank in which they detailed their approach. And that was co-written by Henri Verdier, who is at this moment the digital ambassador for France. So he is like the first evangelizer of this method. And through this, I think now he even is one entrepreneur because he started within, he started a new startup against disinformation. So he was the evangelist but now he's also an entrepreneur. Well, you got to have motivated people and you have to have motivated people at the top because if you're not supported by the top management, so yeah, there must be somewhere an administration willing to take the steps. There are some initiatives in Belgium that wanted to work with a more startup inspired way of working, there was I think like the catch plan in Charleroi used some methods, but not completely. And it was not the whole same thing because here we really concentrate on building real services, not only on just piloting others. Oh, I see an interesting question, sorry. So if you want to know more about evangelization, if you're free to write me a mail, I'll put the documentation forward and maybe have a discussion about that. I'm very enthusiastic, yes, because I live by this, I live by this. Have you had experience of your products clashing with legacy systems and related budgets? Well, Betaguf has internal formations, courses for people. And one of the courses is how to survive in a hostile environment because as you are completely autonomous, you go forward, you are going to step on other people's toes. That's guaranteed. The state is a very pyramidical environment and in fact, the incubator itself allows completely horizontal teams to exist within a completely pyramidical structure. And it helps for Betaguf to be a service from the prime minister because you just can take up the phone and say, hello, I call you from the office of the prime minister and every door will open for you. But if Betaguf was just original legal office, it couldn't do anything of what it has done. So you have to have a sponge strip from high, high, high, way up high. Products clashing, my experience is there's always a solution. Like, I have my experience at Kesedeepur, which is a humongous administration. I think its budget is close to the Belgian gross national product. It's completely incredible. And they have very specific and it's extremely well organized. And I had the chance to be in the first startup data initiative from the Kesedeepur. And so we clashed a lot. It was a very learning experience, both for us and for them, because we came with new questions because they were like legacy protocols and ways of doing things, I would just went right through them. It was unconceivable for them to have a production team, for a development team, even before that, a research and development team, that's how they defined us. It was unconceivable for them to have such a team, have a public IP and publish a full production service without any kind of control in between. And so we had to open that accesses because we could force this opening of accesses because we were supported way up high by the hierarchy within the Kesedeepur. So you had to have the right sponsors to open up and to go through legacy objections. And there's always a way through. You hope so. You can also encounter difficulties. And that's one, there are startups who fail to go through these objections. And that's one of the reasons why your initiative cannot end successfully. Does link data? I don't know of link data, but I know that one of the first ideas of Betagoof, we're talking about Betagoof, but that's because it's quite interesting, I think. It started from a data goof because data goof is that open data platform for the Belgian state. And it was like a question, like how do we make the best use of this open data? Well, maybe we should try this new kind of services. And from that first idea grew the incubator idea. There's, oh, yes, let's, and as an incubator, we make a lot of use of public data. We make a lot of use of open source developed by other teams because each team in France, every service, every software written is open sourced. So each team is fully open sourced. And what happens a lot is when you work on a problem, you go see what others have done. And then you start building upon what they have did. And that's really interesting because if you build a job matching tool, you have the market, you have a person looking for a job, and you build a first job matching algorithm because you're the first one, but you're the second one, you go look around and you realize, oh, there's someone else who built a job matching algorithm. You know it, and so we can take what they built and improve on it. And after a couple of teams, you end up with a very good job matching algorithm instead of having tens or hundreds of completely mediocre job matching algorithm. You have something that grew over time and that's the basic tenant of open source. And that's what it's a big value for a beta group. It's to have everything fully open sourced and the data also. I don't know if there's other questions because we've already a bit past time. If you can also speak up, if you're connected to the audio, public apps, yes? There's some chat going on. Another difficulty is at the end of, where you built your MVP, but then there's also a transfer period where you have to adapt, okay, you test that, you proved that your project, your idea for new service works, then you also have to give it over to the administration where it has to be integrated by the IT service. And so that must also be taken into account. Well, no problem. It's the less I can do really. I don't always think I have that much interesting things to talk about, but when the R Street invited me, I thought, well, maybe there was one or two things I could talk about. And finally, I ended up with a lot of things. Mitigate risk staking, that's very interesting. There's a gradual risk increase. But I work together with, I don't know exactly what the, it's not state security, but it's close to something of an official security organism within France. And like, at first, there's not much risk because there's not much users. And in fact, the risk increase with the number of users you have. And so you can start your service like having very few rules about security or data protection because there's almost no data. And the constraints grow over time. So you don't have to build from the beginning a completely secured solution because you will end up your six month with nothing built. And you can also use a lot of no code tools just to go forward because everyone isn't a developer. Sometimes you don't even need a developer. If it's just a form, you want to push around. And so the security considerations and risks grow with the application. And at one point, you're gonna need developers. You're gonna need the people experienced building stuff. And that's one difference between EURG from Etelab and BetaGoof is that BetaGoof mostly relies on mostly relies on experienced developers because we build services and they must be autonomous and they must grow and be as fast in production as possible. So there's no time for to learn on the job while AEG is more like attracting new talents to the public sector. So it's really different approaches but to the same problem. It's to work and make the cultural shift and attract new profiles to the public sector and work on problems of public policies because those problems have a lot of impact and can affect a lot of change. And I can really better people's lives. So anyways, if there's any other question you want, you can ask, you can always send the email and I'll be happy to reply. Is it okay if I end the recording now, Petriam? Okay, so oh, it's was still recording. I didn't realize. Yes, it was still recording. Thanks again to Astrid. Thank you to the open just.