 I think we can start to get started. It's my pleasure to introduce to you Audrey Lomopoulos, who's going to speak on evaluating government policies using open source models. Thanks. Thank you. Yeah, for me, it really is awesome to be here today. And I'm Audrey. And I'd like to first share with you a story that is the precursor to a lot of this work. So yeah, last year on the second Tuesday of May, as of every other year, the federal government delivered its annual budget. And just like thousands of other Australian households, I watched eagerly waiting to hear those announcements come forth. Announcements like the proposed abolition of the carbon tax, changes to our social welfare system, changes to one of my passions, research funding and a higher education system, and a whole lot more. And I could tell, watching that, that there was going to be a new wave of public debate sweeping through our society. Here's why. Public policy is such a diverse range that they're bound to impact almost every one of us. And sometimes they affect areas of our lives that we feel really passionately about. And so I guess that this is why there's such a vibrant discussion about policy. So an Australian citizen, a mother, and a scientist, I had a whole lot of questions buzzing away in my mind. How does this budget affect me and my family? How will it affect our kids? How will it affect our future? Will the abolition of certain taxes and changes to public policy be sustainable? And here, the scientist part of me. What are these estimates based on anyway? What were the assumptions used? How is this modeled? Here's the source code. More importantly, how do we make our policy impacts? And how accurate are they anyway? So today, I invite you to step through the looking glass with me into a world where knowledge, data, and analysis is freely available to every one of you. Let's take this one step further. Freely available to use, modify, and distribute without restrictions. Now imagine a society where you could access any hypothetical questions using a simple app on your mobile phone. Where do you access? What would that mean for policy debate? How would this change our world today? In this presentation, my quest is to take you on my personal journey into a parallel universe where public policy modeling and analysis is a dynamic collaborative effort between government and its citizens. So fasten your seat belts. We are going on a sci-fi journey. My goal is to leave you asking even more questions when you leave this talk than when you came in. And that's because I believe that that is the real catalyst to progress. So before embarking, I should add that these are my personal views. They are not necessarily the views of the Australian government. There will be some time for questions at the end. And I really encourage them because I would really value your feedback and your input. The other thing to note is that some of the examples I use are specific to the Australian government. But the concepts can be widely applied to any other country. So it comes to numbers to quantify the impacts of public policies. What's the status quo? Numbers are powerful. We know this. They are usually used to give credibility to an argument to support policies, to support the rationale for changing policies, to introduce new policies, change taxes. So numbers and costings are pretty, pretty important. And at the moment, citizens are free to access the cost to see through their annual budget, economic outlooks, intergenerational reports, and other sort of publications. But times have changed. Today, as we progress to an even more digitally transformed society, we expect a lot more from the way that this information is delivered to us. And a static presentation of numbers in a publication is no longer going to be sufficient. In the world, reporting of information, forecasts, trends, and estimates of policy costs is going to be dynamic, and even an interactive process. And I'll take this step further. It's going to allow citizens to modify and distribute it. So the other day, I left my phone on to charge. And my three-year-old son was building some Lego. And anyone who's had any familiarity with three-year-olds, when things got really, really quiet, that's when you got to start to worry. And yeah, it went really quiet. And it sort of said, shiver's down my spine, so I went to investigate. And what I found was my son swiping through my photos, editing them, navigating them, like a pro, like it was almost second nature to him. And here's the thing I thought how to use this before. So look, being the scientist in me, it got me marveling at how far we've come with user interface design and how easy it is nowadays to do various things like create websites, using WordPress, and how much dedicated we've become today with our digital interfaces. And it got me thinking, if WordPress can make web design so much easier than the average, why can't we deliver that allows people to interact with their policy environment using a simple app? Wouldn't it be amazing if we had an app that allowed us to see how changing public policies impact us individually and as a nation, how it impacts the rich, how it impacts the poor, students, professionals, the old, the young, how it impacts us today, and how will it impact us 50 years ahead? So what we're talking about is creating a model that users can test drive through various virtual policy environments and make up their own minds about the sort of future that they want to create. I love this quote. There's a rule of thumb on the internet that says you should never trust a car analogy, so I decided to use one. So imagine you were given the key state of the art public policy model. After a little practice, you figured out the key indicators to watch the dashboard, the fuel indicators, brake lights, et cetera. And let's mirror this for our economy. The ABS has created this really interesting dashboard, which they call the economic dashboard. And it's got four main indicators on there, your society, which has things like health, safety, a whole bunch of things. Economy, environment, and governance. So they're kind of like your economic dashboard. And you can see that in some areas. For example, in the economy, there's a big cross against the resilient economy. So that's kind of like the danger mark there. Other areas, let's say society and health, Australia's doing fairly well. So this kind of gives you a snapshot of how we're progressing. But again, I think we could take this a step further. I think we need to make this an interactive dashboard and a dashboard where you could actually drive this model. So rather than a snapshot, we can actually take this a step further. So yeah, if you were driving a model, after some practice, you're fairly confident. You instinctively know how to run this machine, which maneuvers you can get away with. Because I know I do that when I drive. And yeah, imagine if you could navigate through the controversial terrains of public debate, because you are the one in the driver's seat. You know your track and you know your model really, really well. It's incredibly, incredibly powerful. Now let's imagine that each and every citizen had access to this model. Politicians, media, lobbyists, stakeholders, everyone. What do you think this world would be like? Moreover, what if the government provided these citizens the source code to use, modify, distribute freely? Where would we be? And this is my little pet. I call these policy government open source models or affectionately gozums. And gozums are really the heart of this presentation today. So what exactly is a gozum? It's an open source policy model. And the policy model usually involves the creation of a virtual society with its own economic or policy infrastructure. Ah, yes, this is probably sounding a lot like a computer game, some of you guys. But the main point of creating this virtual society is to test out different public policies and analyze their outcomes. And why is this useful? It's incredibly important for decision making. Deciding whether certain programs should be run, certain funding should be cut, certain taxes should be put in. It's a sort of environment you could test drive your policies in. So now I know what you're thinking. You're thinking how good are these models, how accurate are they, what are the specs? How complex or how complicated are they anyway? And being nerds, we tend to think, hey, let's get to the code. Let's have a look at the source code. Love to see what drives this model. But wait. Policy modeling and analysis are currently one of those jobs that you need to have a caution, authorized personnel only signed post at the front. For almost as long as we've had our democracy, it's a partnership effort between politicians and bureaucrats. But today we're on the verge of something new, something new and exciting. With government open source models, society and government has the opportunity to tap into new and undiscovered capabilities for creativity, innovation, and efficiency. And what we have is a three-way partnership. Gozums will open up the playing field for geeks. Traditionally, policy has been generated by a few sources like lobby groups, think tanks, academics, media, and government agencies. But I propose that we've got a third party there and we've got the geek in the story. So in the past, you may wonder, why haven't we been involved in this process before? And the reason is that the public in the past has not actually had access, such easy access, to data like we do today. Open data is a big thing nowadays, but about 10 or 15 years ago, it wasn't as successful as it is today. The other big thing about policy analysis is that it's actually hard stuff. Figuring out the quantitative information involved is difficult, it's specialized, and the average person usually runs away from math. But this is going to change. The specialized knowledge that we need for this kind of task is, I think, is broken down into three main parts. And one is economics. People need to be a bit familiar with economics. The second is a numerical kind of background. Math, stats, computing, in my case, physics, which is a bit out there, but hey. And the third part is knowledge about tax and policy and legislation. So there's three main components to this kind of work. And so the general perception is that this kind of work is pretty, pretty tough. It actually means that in public debate, when you sit there and listen to what's happening, people actually don't really question the numbers. They question policy, they question how things should be done, whether things are done the most efficient way, but when numbers get thrown out there, usually people tend to accept them. And this talk is about questioning whether or not certain policies cost what they say they're going to cost. And for someone like me, what the assumptions were behind those numbers? Where do they come from? The source code, essentially. So now we're ready to tackle on a case study policy problem. Remember, the same underlying concepts apply to any other topics, such as climate change, migration, pensions, the global financial crisis, but for you guys today, today's special case study, we will be focusing on a rather controversial debate. It is going to do with the event that Humpty Dumpty falls off his wall. In this example, let's, I mean, let's now ask just examining the tax consequences. As you'd expect, there's a fair amount of media coverage on this issue. Both sides, the skeptics who think that Humpty is on fairly safe ground and he ain't going to fall anytime soon. And we also have the doomsayers who say that his fall is going to happen any minute now. So, you're all policies as a model developer. Some of the questions you might want to ask are, one, what's the likelihood of this fall going to be? Two, how are we going to estimate the cost of this fall? And three, how would we be able to cover the cost of this fall? So, as with any data policy modeling process, the first step is information gathering. Being able to identify data sources, which information is important, information is irrelevant, which information is useful. And here we've got a diagram that sort of sketches out the process of how you'd go about modeling the answers to those questions. So, what you can see here is you've got three main components. One's your data, which is often collected by the Bureau of Stats, survey data, government administration data. Just look at your data itself. You actually get a lot of information that tells you about how things are going at the moment. And this analysis is invaluable. Today, government has employed very unique ways of getting the public involved in this process. And GovHack, as some of you know, has been incredibly successful. It allows us to visualize open data. And we've made a lot of progress in that area. But what I'm proposing is the second step, which is to actually release the model itself. So not just the data that gets put into your model, but the source code that actually does the virtual policy environment. So, why do we need the model? I mean, what does it do? So, I guess based on what I've seen, the models are mainly used for two reasons. The first is to test out your hypothetical policies that you want to propose and see how effective they are, whether they apply to the people that you want them to apply to, whether they get enough revenue. And the second reason is for forecasting, to see how things are going to be in 50 years time or for the next generation, for example. The accuracy of your model depends on various things like your data. And finally, after you run your model, you get your output, which is a quantitative measure of how effective the policies are. But of course, like with any program and any model, it's only really as good as the data you put in. And often, the output that you get out needs to be moderated or benchmarked. And that information then feeds back into your model to make your model even better. And so, it's like a bit of a cyclic process where you fine tune your model based on the output and based on your observations of what's happening in society. So, output. Highly contested. This is the stuff that newspapers run with. This is where you get people arguing for both sides about the number, but mostly the numbers are just accepted as is, but this is where I think we need to start questioning. So, what I'm proposing again is to sort of get these gossums out to the public. So, we want to have a look at how, as a developer, so we've kind of talked a bit about how useful it is for users, but as a developer, what's inside the gossum? How do you write one? So, a gossum consists of four main cogs in general. The gossum is used to consider what language do you use, Python, and exactly, do you write? So, it's code, which is a translation of legislation. So, legislation is a bunch of... And you're translating that language, in a sense, into your code, Python, for instance, to add up this virtual environment. You have a few economic assumptions, like, all right, what are the assumptions? How is our population going to grow? What's the migration level? What's the fertility rates? What's going to happen to our GDP? And there are your assumptions that drive the model, which you can change, and will give you a different output. There's your behavioural economics. Things like, with the Humpty Dumpty case, for example, well, if you change your policy and put a really high tax on sitting on walls, maybe you won't have eggs doing that, and that would then change the output of your model. And last part is the policy implementation. So, it's nice to have the policy written in legislation, but then is a task of making that translation from legislation into code. So, we're off. We're now ready to tackle even the toughest of policy challenges. And everyone of you could develop a model. The biggest challenge for government about this is releasing policy models and finding a balance between transparency and privacy. How to protect citizens' privacy and autonomy? Providing the output to a closed-source model does have its risks. So, if I provided out the source model and I provided the output to the model, could, in theory, someone reverse engineer that to figure out what the closed data was? An example would be your tax return. If people filed their individual tax returns, and I created this model that read it all in, and I provided you with the model and the output of what was the model, could, in theory, someone then reconstruct what your tax return information is going to look like. And that's the real challenge here. The other thing is that government isn't just in isolation. It is also responsible for the integrity of our policy environment and our national security. So, unlike a private company, it does have its other responsibilities. For example, if government were to release models on tax compliance activities or national defense, could that pose considerable threats? And it may not actually be in the public interest. So, making that distinction is not really that easy about whether an issue is a threat, national interest or not. So, could gossums be exploited to do more harm than good, I guess is the question. And before releasing any government modeling, it's really important to consider the potential for abuse by individuals and entities such as organized crime. So, yeah, that's a tough one to deal with. And then there's licensing. Software developers and open source software today are really aware of the issues relating to ownership, open source licensing. A government model has a slightly different take on it. And that's partly because government's funded by the taxpayer and they also need to consider liability issues, particularly as government has a duty of care to its citizens. So, if we released a model and something happened, would government be liable for that? So, that's a bit of an issue as well. Hypothetically, I think governments could be responsible for maintaining a code repository and incorporating the beneficial changes from the public into an official version. So that policy would be the ultimate winner. But this is actually open source with a slight difference. Usually when code is developed, it's developed with a specific outcome in mind in terms of its usability and in terms of its functionality and in terms of achieving an end goal. But can Gozums be actually used to distort information? Can it be exploited by lobby groups to sell their own perspective on what things should be like? Will groups try and serve their own personal interests? But the key with open source is that with open source, there's also transparency. So, in theory, you would be able to see the underlying assumptions that these other models, I think new territory where political philosophies are subjective and the scalability of models in credit are numerous. And what you end up with is an information exclusion, which brings us to big data, buzzword. Data science and big data. IBM describes a data scientist as having a foundation in computer science and applications, modeling, statistics and analytics and math. You know, it's a pretty big task. It's a pretty big ask. You kind of need a superhero to be across all those skills. And the thing is that it's actually been really well publicized in the media that over the next few years, we will actually be experiencing a global shortage of data scientists. And as I said, it's because it requires a specific combination of skills. In fact, what's happening today is that private companies are actually employing teams of people with specific skills to sort of try and cover what they expect from a data scientist. But this is where you guys, the open source community really excels. By engaging the open source community, you have access to such a diverse range of skills. I mean, you're not really talking about a team of five. You've got a team of as big as the open source community is and that's just taking it that order of magnitude better. Diversity is the key to innovation. Government has an incredible opportunity of tapping into this valuable resource to come up with unique solutions to even the most complex policy problems. And they're unique because every individual here has a unique way of problem solving. So we need you. Don't you love this pic? I mean, the first time I saw it, it was on the Gov 2.0 blog and I just kind of jumped out of my seat and thought, oh, who else is it looking over my shoulder? But yeah, it caught me off guard and it's because of the problem and it sends a message to each and every one of you that your contribution is actually really very important. Oh, penguins. Policy is dynamic, it's engaging, and it's really important. It allows you to have a shared vision about an issue you feel really about. It allows you to get behind that you really believe in. But this time with open source models, you'll be the one in a position of power. You'll be the one in a position of influence. Each of you has the power to influence public debate in a unique way and play a role by shaping our public policy and shaping our future. So how can we increase citizen participation? It's imperative that these models are open source because they're not only belong to the taxpayer because they're essentially funded by the taxpayer and we need to minimize any barriers to entry, not just for transparency, but so that citizens can become more involved in the development part of public policy issues. And what this means is that the access to technology and Gozums need to be convenient. Now according to the ABS, I think they said approximately 85% of Australian households have access to the internet and that was back in 2012, 13 and it's quite high, but not as high as I would have wanted. If we expect our citizens to contribute to policy modeling, shouldn't government provide them with the necessary tools? And that's not just access to computers, it's also access to education in open source programming skills in schools. And more than that, I think that programming in an open source language knowing that a model is going to be publicly released is actually quite different from programming in a language that's closed and I think it's partly the mentality. When you're writing code for just yourself, you're more likely to cut corners and write in a slightly different style than you would if you knew your models are going to be released or that was your intention. And so from a developer's perspective, I think when you start off writing your code, you're in a complete safe and mindset if you feel like your code needs to be put out as open source. So if we do participate in policy modeling, each and every one of us, what can we expect? For my personal experience, I say be prepared to gain valuable insights into how policies affect your lives and ways that you never expect. And the other big thing is be prepared to see how policies interact with other policies in ways you don't expect other. And it's uncharted territory in a sense. As you explore, you'll discover many, many other secrets about the world that you live in. Now, the next question I guess is, we've talked about how useful it is to the public, but what about the policy itself? From a policy maker's point of view, the possibilities are endless. Having a whole ecosystem of virtual models created by every single individual, I mean, this is thinking very big, how exciting, how exciting. Improved access to computational power has seen the policy development process transform into a dynamic and iterative process at the moment. So because you've got such high computational power, what essentially ends up happening between bureaucrats and politicians is that they go through this iterative process of, well, we've got this policy, can you please figure out how much that's gonna cost us? And because we're able to turn around that information so quickly, there's even more questions and even a larger range of policies. But now imagine if it's not just your bureaucrats, but your entire community doing this, you will just have a lot of information. But more than that, having so many models out there, I mean, an ecosystem will blow it out of the water. Imagine a virtual environment where each policy model or Gozum with its own uniqueness created by each and every one of you, now interacted with the other models that each and every one of you created. And what you essentially end up with is like a survival of the fittest. So you have your own individual hypothetical policies interacting with everyone else's hypothetical policies and you could construct an environment where the most fittest policy survived or the most optimal policies to society survived. That's just awesome, isn't it? You wouldn't have to have one or two individuals sitting there thinking about policy ideas. You just have such a plethora. I even promised myself not to talk about genetic algorithms, but there you have it. Okay. One more thing about government open source models. Have I told you how great they are? Government open source models, Gozums, have the potential to be a public good. Now, when I studied economics and my uni professor, as with most economic professors, would always use a lighthouse as the canonical example of a public good. And you see, a public good has two main definitions. The first one is that it's non-rival. So, light from the lighthouse. If you receive light from the lighthouse, it does not reduce anyone else's capacity to receive light from the lighthouse. And that's why it's non-rival. You're consuming that good isn't affecting anyone else's consumption. And the second is that it's non-excludable. It's not possible to exclude anyone else from using it. So, light out there is not just meant for one ship. You can't exclude ships from using that light. And that's what the definition of a public good is. And I mean, I thought public goods are pretty cool, but the physics nerd inside me got even more excited because with open source models, it's even cooler. And here's why. Because you're able to modify, distribute it, imagine yellow light coming out there and a ship was able to convert that to an orange light and then beam that back out. So you have yellow and orange and another ship comes past and converts the light to blue light. And suddenly you've got this complete amazing light show, but I didn't think my professor, my economics professor actually got the concept of that at the time, but I was in a buzz being the physicist and stuff, but yeah. So, essentially what I'm saying is that you have your traditional public goods, but open source has the potential to make public goods to a completely different dimension. Yeah, so one of my favorite, his favorite, his favorite things in the world is to read to my three-year-old son before he goes to bed. And my story is that I spend most of my life studying math, physics, doing a lot of 4-turn programming. And it's where I felt really comfortable. 4-turn programming for those who all the love to know what is. You probably don't wanna go there right now. And I had almost no interest in public policy whatsoever. I was lucky if I, and this is really embarrassing, but I was really lucky finding who the prime minister was. I was that buried in what I was doing. And then one day I thought, how cool would it be if I could apply my quantitative skills to something, to some less esoteric application, yeah, of policy. And what I discovered completely blew me away. I learned that public policy was a new land waiting to be explored by nerds. See, I get emotional about this stuff. And government open source models has the potential to be a legacy that we leave to future generations. And that's what I believe is the spirit of democracy. Thanks. Before we open up for questions, I just wanna dedicate this talk to my three-year-old boy, who's my inspiration. And a special thanks to all my friends in the public service who have enjoyed long and arduous discussions about a lot of this content. So thanks to you guys. Questions? Thank you. Yeah, if anyone have any questions, just ask for them. Firstly, it was a great talk and I love the passion with which you spoke for policy. It's not usually think people get so massively passionate, but it was very engaging. My question is, you mentioned a shortage of data scientists and I know a heck of a lot of people I thought had the skills for data scientists. So two questions. What are the skills for a data scientist and who out there funds data scientists to do data science? I know a lot of people who do something completely different because there's no money. Data science is actually a lot of different types of skills. I think the predominant ones are statistics, information visualization, and your hardcore programmers really. So someone who's across all that stuff. And people have different strengths and weaknesses, but in terms of employability, there's a lot of companies, and this is a trend starting in the US at the moment where private organizations are going to end up with really huge amounts of data because we're now collecting all sorts of data from your mobile phones and not just data in terms of numbers, but metadata. And what we're up with is this huge amount of data that people need to be able to be clue enough to figure out what it actually means. So companies are starting to look at things, and you're starting to look at how do you appeal to different clients based on their usage of certain internet sites or what their clients actually are interested in. So it's a huge amount of information and the skills, and yeah, the skills shortage is a bit of an issue. And so what's actually happening now is that universities in the US are running specific data science degrees, but they can't turn them out fast enough. But I think the open source community has a distinct advantage in that space because I mean, this is what we do best. We problem solve because of the size and innovation that we're able to tap into. It's not restricted to just five individuals hired for specific skills. It's open to everybody. Hi, so I like the idea of this ecosystem where a lot of policies that are trying to have the same goal can exist. The one thing I don't know how to address with that is what about when people actually have different opinions as to what the outcome of the policy should be, I what figures they should be trying to maximize out of that and end up with different types of policies that might be the best in what they do, but actually have different end results. Yeah, that's a really good question. I guess with Gozums, the key thing about that is transparency and it's about people understanding where the numbers come from. So what my vision is is to have the debates about what policies should be used as being the setback of the policy discussion. We should have lots of using numbers as a way to hide behind, using numbers to hide behind what we actually believe in. So our discussion should be about what we want to want the end result to be. And then once we determine what policy we're actually interested in, then determining what the optimal way, a pathway is to get to that using the ecosystem approach. But I completely get your point because if different people have different views, then how do you determine the survival of the fittest and which one wins? So I think that kind of discussion, I mean, that's for each and every one of us based on our political philosophies to discuss. But when you elect your government representative and a decision is made, then achieving that outcome and the optimal way to achieve the outcome in the sense of satisfying the most number of citizens who are having the least amount of tax burden, I guess. That's where the creativity comes in. Does that? Yeah. So anyone else? Yeah, and it kind of really follows on from that. And I sort of, I'll talk a little bit about the algorithm, perhaps this is a chance to talk more. I mean, instead of people developing new models, could they be developed automatically? Could you write algorithms to develop these models and have them optimised? And what does that mean for bureaucrats? What does that mean for politicians? What does it mean for political philosophies? What does it mean for democracy? I say bring it on. And this is the beauty of that open source, right? Like you've got the freedom, the freedom to do that, if that's what you choose. And from my personal perspective, I think the more the better because we're sort of covering, I mean, the more species you have in your ecosystem, I think makes for a more exciting world. Is there a risk of basically the machines deciding what's best for us? Yes and no. I'll tell you this. I think that there is a danger about having too much information. For example, the other day I went out to buy some toothpaste and I'm usually used to my one brand and they didn't have it. And there were so many brands. And it confused me. It really did. And they have done studies in economics, in behavioral economics that sometimes when you give people too much of a choice, it's actually harder for them to make a decision than when they're actually given less choice. And that's part of how a neurology works. So in terms of decision making, whether we should be getting humans to decide it or machines to decide it, I guess I go back to my previous answer in that I want to empower us to make our own decisions based on our political philosophies and then use the algorithms as finding the most efficient means to get to that point. We need to make decisions not based on because this policy is gonna actually get us into a surplus or policy B is gonna get us into a deficit because that's not the underlying way that I believe policy should be come up with. Policy ought to be based on your philosophy and your beliefs as opposed to I'm deciding this policy because it costs that much. And that in a sense is sort of where we tend to in our media discussions. So making the decisions by humans but getting the computers to figure out the pathways to those decisions. Sadly run out of time for more questions but I'm sure we have an opportunity to ask them on the whole of my track. Yeah, sure. Thanks very much. So I thank you for your very interesting talk about representation of that. Thanks. Thank you very much. On behalf of LCA. Yeah, we'll continue in about 10 minutes. Thank you. Thank you.