 a lovely welcome back to the Huck stage this evening here to present you a talk about models in science opportunities and mechanisms limitations by a Marcos Belter. He will be talking about scientific models, what they do, how they work, what are they and they are quite in the news right now because of the coronavirus and climate change and with models you can do a lot you can model things but for the public it's not always quite apparent how they work and what they do so we have our own streaming site with an IRC and no JavaScript if you don't like that go to live.hack.media there you can also ask questions in the IRC or via Twitter with the hashtag RC3Huck and apart from that I think we're good to go so I'm giving over to Marcos. All right so talk it's scheduled for 90 minutes I might take a little bit longer but it should be interesting all right so I want to talk about models I'll cover topics like the difference between analytical numerical parametric models forecasting versus explanation abstraction and simplification statistics forecasting the past sounds stupid but it's useful sensitivity analysis optimization forecasting versus scenarios fitting of data fake numerical precision chaos emergence and unknown unknown that's quite a lot of stuff so I'll run rather fast as you'll notice the examples will be taken from cranes pendulums weather and climate flight simulators sheep and ants gravitational waves fusion the LHC the event horizon telescope cellular automata attribute turkeys and you might ask yourself why the hell is this guy talking about you have some background I'm a physics engineer and a computer scientist but more importantly over the last 12 actually years I ran a science podcast called Omega Tau scientists and engineers for about 600 hours and that's basically where all of the material of this talk comes from the podcast also gave me an hour in the backseat of an f-16 which of course was the highlight all right so let's get started let's get started means you probably have all seen these large steel monster and if you look closely at these little traffic lights kind of thing the cabin highlighted here it signals to outside personnel the load of the crane not the weight that is on the hook but the actual stress on the crane the load on the crane and of course obviously the mean what you'd expect okay be careful and at the limit and so what is the load of the crane well one is the bending of the boom right the boom is a long stick that is of course bent from the weight that's at its tip on the hook and of course also the stability the balance of the lifted mass and the counter mass I should say counterweight so how's the crane know all of these things well there's a model that calculates this and drives the traffic lights and let's investigate what goes into that model well one of course there's the weight that is currently being lifted the more weight the higher the bending moment on the boom the length of the boom is relevant of course these booms are telescopic and if they're longer you have for the same lifted weight you have or mass you have more bending moment the angle of the boom obviously you can lower it and also you know make the angles deeper and the lower the angle of the boom again the higher the bending moment because the radius is larger there are additional wires and poles you can use to stiffen up the whole construction and if you have those installed then your bending moment is less for a given lifted weight and finally wind because if the wind moves your lifted mass sideways this leads to sideways bending of the boom which lowers the overall stability so that's for the bending for the balance obviously the relationship between the lifted weight and the counterweight and their respective radii is relevant so that is something that's taken into account and the distance slash radius of the stanchions the further out you have moved them the bigger is the if you will footprint of the crane and the more stable it stands so these two aspects of overall load are calculated separately and whenever one of them reaches you know critical level that is that drives the traffic light so how is this done well for the stability for the balance we simply use the the lever principle right so you basically calculate the two moments the the actual lifted weight is measured from the pressure in the hydraulic cylinders the counterweight is configured and then you know the two radii you can calculate those as well from the angle and the length of the boom and then you can just simply figure out whether the crane is stable the balance is in balance for the bending it's a little bit more interesting because here what you basically do is you run an FEM analysis finite element analysis where you basically simulate every little discrete piece of boom material and run a numerical simulation a numerical model that figures out where the load is high and when it reaches a limit at any of these discrete small finite volumes volumes and of course you can't run this in real time in the crane which is why the results of this analysis are abstracted into basically lookup tables right this is called a parametric model we'll get back to these model categories later but that's basically how it works and so there is an interesting thing that happened this year or maybe it was last year this is a 750 ton crane the LTM 1750.9.1 manufactured by Lipo and again it can lift 750 tons well it was able to do 750 until earlier this or last year because now it can do 800 only because of a software update nothing changed in the actual construction of the machine in fact existing cranes can be updated retrospectively so how does this work well there's a something translated from the Lipo website it says we completely recalculated the crane more and more precise FEM models more and more precise FEM models I should say on faster computing hardware permit a less conservative approach the latest regulations regarding the load bearing capacity in compliance with all the applicable standards so blah blah blah so basically they they have newer modeling tools both in terms of algorithms and software and machinery computers and now they can be a little bit less pessimistic conservative about what the crane can lift and so they've the software updated update basically installed new lookup tables and now the same machine can lift 50 tons more regarding the balance I talked to a bunch of crane operators this year obviously and one told me you know today it's basically impossible to to knock over a crane to to bring it out of balance so that it actually falls over oops this is from 2013 where one of these large Leapel tracked cranes fell over in Brazil during the construction of their Olympic stadium and the problem there was surface stability right so that the ground gave in these cranes produce a significant pressure on the ground and if the ground isn't suitably prepared then the whole thing falls over and obviously the model that you know takes that considers as we have seen the levers of of the weight in the counterweight can't know about this so this is a problem here's another example of something that went wrong a large ship board crane in rostock germany completely crashed and so again is this a modeling problem well no not really it was very likely a material fall the production mistake in the hook itself the hook broke and then the whole boom basically snapped over backwards completely destroying the crane millions of millions of you know money lost and several people hurt big big drama so what can we learn about this we have to be really careful about what goes in a model the model sounds trivial but the model doesn't make any statement about whatever hasn't been put into it like for example the status of the ground or the material quality of the hook and these assumptions and constraints must be well defined that they must be communicated to the users so is this a useful model it's a lego actually yeah it's a lego model of one of these ltm 1750 cranes it's not useful for understanding the load capacity because obviously the lego has no you know relationship to the real thing in terms of stability but you can explain in principle how the crane works you know with with extensions and telescopic booms and stuff so it can serve as a model for illustrating that it might serve as a model for work site planning you know if you have the work site in the same scale then why not right so this model also has its purposes but it's not the same purpose as these mathematical models that we've talked about before so again the user of a model has got to know what's in the model what the purpose is and the modeler has to decide what is important what impacts the results so let's define the notion of model it's an abstract representation of something that exists in real life in the real world expressed in a suitable language like chemistry physics math lego should be an o not a for the purpose of understanding prediction optimization proof or production of that real world thing the proof and production part will ignore in this talk we're going to look at understanding prediction optimization so when you build a model you have to think about what part of the reality you want to represent in your model what is relevant to represent that part faithfully in the model how to represent and compute right i mean you have to somehow make it quote runnable mathematically calculatable or in other ways executable you have to think about how do you observe parts of the reality that serve as input to the model and then what is the relevance and trust of the output again if you don't input something about the ground then you're okay if you can only measure that unreliably then the relevance of the output and the trust level of the output is low we'll get back to that so let's revisit analytical numeric and parametric models here's a very simple analytic model it's a differential equation of the movement of a pendulum right it basically says that the angular acceleration which is what the left term says basically depends on the angle itself on the sinus on the sign of the angle and then you multiply it basically by the fraction of the earth acceleration and the length of the pendulum and so an analytical model is basically where you use precise or approximate equations as we'll see this one as approximate and then for dynamical systems these are often differential equations which relate a quantity to its first or second or whatever a derivative again here it relates the angle of the angle phi to the second derivative of that same angle and so again a numerical model sorry an analytical model is what we all know from school as formulas equations now a numerical model is more interesting because in practice plays a more relevant role for what we you know hear about models today what you'd use to discretize your world in the finite element analysis we saw in the context of cranes before you basically discretized the overall boom into finite small volumes of steel and climate and weather models you discretize the atmosphere into volumes of air if you do epidemiology you have sets of subsets of the population and then you reassemble the overall system by basically joining all these little volumes and computing a whole bunch of properties for each of these i call these properties e for eigenshaft sorry change that the properties and you then again from from the calculated properties of each of these small volumes you reassemble the overall story so and then you also typically if this is a system that evolves over time you iterate over time right so again for the weather you have small volumes of air you calculate various properties for each of them and then you run this thing over time there's a bunch of constraints one is that you have to make sure that the changes are plausible both in time and space what i mean by that is if you have a weather forecast and it says you know at time t equals one the temperature at some location is 20 degrees and 10 minutes later it's 10 degrees there's probably something wrong same is for spatial plausibility you if you have 10 degrees in schuttgart and 20 degrees in eslingen city nearby it's probably strange so you have to make sure that either by design the model doesn't jump both in time and space or whatever other discretization dimension you've used or if if this is an outcome then your model is wrong if things jump so basically you still calculate with equations in numerical models but these are discretized and often spatially and temporally these are the most obvious discretizations but of course others might be used as well if you use smaller boxes smaller discrete volumes you get higher resolution same if you use smaller time steps so this gives you increased precision in some cases it gives you correctness for example in control theory if you have two large time steps you might not get a control algorithm that is stable because it because of its large time steps it it can't see oscillations and control those so that the algorithm isn't just imprecise it doesn't work if you don't use a fine grained high resolution time same thing in some cases for space if you want to forecast thermals you know the stuff that propels gliders up warm air rising above mountain peaks and you have a coarse grained resolution of your terrain then these peaks are averaged out and you will not be able to produce correct predictions of how and when thermals develop above those peaks it's something glider pilots know very well when they fly in in mountainous areas that the forecasts are often wrong and then of course the drawback of a higher resolution is that you need more computational power and even today with faster and faster computers it's still quite plausible I mean scientists still have to make lots of compromises in various ways for example in resolution to make it feasible to run on existing machines and then of course you can compose these for example the ECMWF the European center for medium-range weather forecasting runs a worldwide weather model they use nine by nine kilometers horizontal resolution and 500 meters vertical resolution up to 80 kilometers by the way and then the german weather service the dvd they have a higher resolution model but only for germany and basically they use the the more coarse grained model from the ECMWF as a as a boundary condition for what the and and also a means of initialization in initializing data for the final grant model and by the way weather models calculate around 50 of these properties so lots of stuff going on there and so you can see easily why even large computers can be utilized completely with this approach so let's look at parametric models again we look at cloud formation you can see a beautiful picture of clouds here and the question is you know what kind of cloud forms how much of the sky is covered by clouds what is their lower and upper limit called base and ceiling and there's a lot of parameters to go in right the position of the sun basically time of day how much high clouds you have that limit the energy imparted by the sun then of course humidity pressure temperature properties of the ground the elevation of the ground and so you want to if you want to compute this algorithmically there's lots of chemical and physical processes going on some of them are theoretically not even known lots of small things happening lots of complex interactions so basically what you do in principle again is you you discretize and you associate all these interesting properties with each of these discretized air volumes but you can't really run this as a numerical model because again some things aren't known interactions are complex it would take too long so what do you do you basically put all these things into a series of lookup tables right so for example you know properties e1 and e2 from let's say observations and from those you calculate some property x and then this property x together with properties 3 and 4 you put that into a simple equation which gives you a property y which you then use in another lookup table together with property e5 and you get the result so this is basically a very simple lookup table and or set of equations and of course those are as we've seen with the crane filled by numerical models analytical models and experiments right experiments even if you don't know how something works you can easily just take the experiment results and package them into a parametric model which you then run to make a forecast another example of a very simple parametric model comes from medicine this is actually the only software example I think in my whole talk I built an application in the healthcare domain not too long ago where for example from the systolic and diastolic blood pressure depending on the ranges there in you calculate a risk factor which you then use for example together with the age and the weight of the patient to make further decisions regarding treatment at combining these different kinds of models this is a picture of a aircraft which many of you probably know from the movie top gun although this is not actually a picture of a real aircraft this is a screenshot from the dcs here's another one looks beautifully realistic so there would be another example of models is how do you model these surfaces and like it but we're going to talk about something else and that is the modeling of the aeromechanical properties of so let's say we are a maverick and you know you know we're flying around in top gun with goose on the goose that he's gonna turn right so he'll smash the stick to the right what happens next right how does the simulator simulate reacts when the pilot moves the stick to the right so we have the stick movement which basically is translated simply you know stick angle becomes a percentage of deflection and that's the input to the hydromechanical flight control system and now you have to think about to what degree the arrow surfaces in this case it's elevons and spoilers on the wing to what degree they are deflected and interestingly they actually calculate that in the simulator and they calculate that based on the current hydraulic pressure in the simulated hydraulic system because if week 28 shot away half of it you know remember top gun um then you have less pressure on your hydro in your hydraulic system and the deflections are slower then you get the you you find out what your flap settings are you find out what the wings we angle is as you remember from top gun the wing the f14 has sweepable wings right to adapt the wing to different speed table what is the coefficient of lift for that deflection for this flap setting and this wing sweep and this is a parametric model which happily NASA did in the 70s they put one of these aircraft into the wind tunnel and measured all of these things and those guys somehow got hold of this wind tunnel data and now that we have the coefficient of lift we can basically calculate the actual lift force and we do that by multiplying the coefficient with a bunch of properties of the aircraft but also specifically with the speed of the aircraft which they get from the simulation environment so there's another calculation and then they look at again the wings speak as wing sweep angle which gives them an inertia and they know how much fuel is in the wing in the simulator because they know how much engine fuel the engine has burned since they took off and they take this information to actually calculate the inertial moment that acts against the movement from the differential lift of the two sides so you know and this then gives you a role role movement so the point is that i'm trying to illustrate here how this overall model uses analytical meaning calculations and parametric models jointly in one if you will big calculation relying on previously calculated numerical data in this case stuff that nasa has figured out in the wind tunnel now models aren't just relevant for controlling simulated airplane on a pc there's also actually a model very prominently used in the flight control system of an airbus a320 and of course of all subsequent airbus airplanes so the way you actually fly an airbus is that you move the stick and what this does literally is that the stick flies an idealized airplane a model of an idealized airplane in the computer then the various attitude sensors and you know air data computers they measure what is the state of the physical aircraft things like you know angle of attack roll angle stuff like that and then there is obviously a difference between the model that the pilot flies the idealized airplane and the real one and then you have control algorithms that decide how best to get the state of the real aircraft in sync with what the pilot wants the aircraft to fly like based on its input into the idealized aircraft model and it's interesting because if for example one of the flight computers has failed and you cannot control your ailerons then you still use the same flight inputs but of course the system then figures out how to use for example the spoilers to produce the roll rate in order to bring the state of the real aircraft in line with the model so to wrap up that part of the talk we have analytical numerical and parametric models analytical models are basically functions if you will equations that take a bunch of inputs and you can calculate the output you're interested in directly for every point in time for every kind of combination of inputs the quantities you calculate with are usually continuous and the equations are physics or whatever else chemistry biology but it's the real science in a numerical model you discretize basically the analytical world you iterate then and typically the iteration well almost always your iteration will depend on the on the previous state of you know in the last time step for example and this means that you can't just say hey model give me the state at t equals 5000 seconds because in order to know that the model has to calculate you know for all 5000 previous time steps and that makes these kinds of numerical models much less flexible on the other hand they're much simpler because solving complex you know systems of equations numerically is much easier than analytically the equations are still basically the real science but the quantities are discrete in a parametric model you can again have a direct look up it's not expensive but the quantities are usually much more coarse grained and the equations are not necessarily somehow resembling the real science they might be completely opaque right they might be completely just numbers you figured out somehow and you know they kind of work and you use them but you they don't tell you anything about how the real world works they make a prediction but they don't explain so let's elaborate on that difference let's say you have the real world changing somehow as shown in this lower dark gray box and the you know the reality changes basically as a straight line this wobbly change is sensed by your model as a step function the model calculates and outputs some kind of reaction okay that's fine that is a prediction so the model expresses or gives you information about how reality will change as a consequence of whatever impulse you you know are interested in like how does the weather change if temperature increases how does the virus spread rate change if 60% of people will wear ffp2 masks right stuff like that but there's no explanation going on which raises the question what actually is an explanation and i struggled with this quite a bit because what is the difference between the how and the why isn't the why also also kind of how you know how do you make the difference and i interviewed somebody from from cern and he gave me a very very interesting definition of what why means in this context he says that if a model produces its prediction through applying a more general theory to this particular case right if you don't have if you will a theory specifically for this problem but if you have a more general theory and the model if you will configures that theory in order to produce a correct prediction then if you look at the configuration of that model you have the why you have the explanation examples for such for such general theories are of course newton's laws or quantum chromodynamics or darwin's evolution or some kind of reaction kinetics in chemistry right and so this really is a better model because it just doesn't just predict it also explains why it makes that prediction it explains the underlying mechanisms which is again more useful now here is an example a category of models that don't do that right you have probably all know what this resembles here this is a neural network which has a bunch of inputs and a whole bunch of weights and an output so the neural network makes a prediction right it's the ultimate form of a parametric model because each of the weights here in that neural network and there are millions in real networks is a parameter and it is not at all transparent what each of these millions of parameters means and what the model has learned you can test and that's what people do but if you forget to test certain things you have no idea how the model will behave and there's this funny example where people were trying to build a machine learning model neural network to recognize sheep right the idea was to oh should be idea the idea was to make the model learn the shape of sheep but of course what that particular model learned was you know the green of the grass on which shapes or sheeps are usually photographed and so um you know when you showed the thing just green pastures it also you know detected sheep and so basically um such a opaque pure parametric model will produce wrong results with some probability for new situations and the problem is that you don't necessarily know what a new situation is because you have no clue you know what the thing has learned obviously you test but testing as we all know from programming but not sure if we all know but as programmers know testing can only prove the presence of the buck never the absence of bucks and you have that problem here so this really doesn't matter for advertising it's annoying if they try to sell me the same shoes that I just bought right or tried to sell me I don't know you know orange hats which I will never buy but it's not really a problem it is more of a problem if people aren't given credit because some stupid machine learning algorithm has basically you know incorporated some discriminatory um decision based on stupid training data and it's really a problem it's potentially fatal in autonomous driving so there's a whole thing about you know uh trustworthy machine learning and you know understanding what a model learned but that's in its in its early stages so let's look at abstraction and simplification again back to our simple pendulum and we can ask how quickly will this thing pendule back and forth and yes I know pendule is not an English word but I thought it's funny so what's the period of this pendulum and if you look at the wikipedia article for this you get a very long formula and you can see the three dots so it continues I did actually a screenshot of wikipedia because it was too lazy to type this up in a formula editor myself because it's so long the point is that you can see that this period depends on the angle so if you um you know move the pendulum further out it you know oscillates with a different rate than if you move it out only a little bit and you can make a simplification for very small angles where the sign of the angle is basically the numbered angle itself sine x equals x then you can simplify for small angles that the period is 2 pi square root of l over g and that is probably the formula you learned in school which is completely fine it gives you the right result if you stay within that range for which the simplification is valid right and it gives you wrong results outside of that validity range so this is another one of these things if a model has been built for a certain limited parameter range if you will or range of inputs and you use it outside the forecast is wrong the obvious example here also is newton's mechanics because newton's mechanics are wrong right because einstein showed that things are quite a bit more complicated with a special and general relativity but we also know that for our everyday world because we are at low speeds and relatively low gravitational forces newton is good enough but you have to be aware of this if you try to calculate the path and trajectories of spaceships with newton you will not get the results at the required position that you expect so another question will this thing oscillate forever and if you look at this formula again there is nothing that seems to slow this thing down so it's actually this formula is actually given with no damping right so this thing will just continue forever and we know experientially that that's not the case you know stuff will stop oscillating at some point because of aerodynamic drag of the of the weight and also because of internal friction within the the wire that holds the weight so actually it looks like this right so it will become less and at some point it will stop now again importantly for your pendulum clock right you'll i think it's called grandfather clock in in english right i'm not sure um it doesn't really matter because you'll you know add additional energy to the clock every evening you know right around target show anyway and so doesn't really matter but if you try to work with this kind of problem then it is a concern not sure if you recognize this picture it basically sorry represents the how a gravitational wave looks like in 2016 the LIGO gravitational wave detectors have heard gravitational waves for the first time and again here is here's this thing again this shows time on the x axis and the frequency of the wave on the y axis and the darker the shape the more intense the louder the stronger the wave so they call this a chirp because it basically sounds like right because it the higher frequency increases it becomes louder like a bird's chirp so how do those guys detect gravitational waves well they have a interferometer which basically sorry which means that they send in a laser pulse along one arm the laser is reflected passes through a semi-transparent mirror along another arm that is orthogonal to the first one reflected again and then back to a place where the two laser beams that went through either or both arms are interfered and if the two beams arrive in phase there will be a constructive interference whereas if they arrive exactly out of phase then the output will become dark so any difference in length between the two arms will change the interference pattern at the output that's the idea and so if the gravitational wave passes by and makes one arm longer and the other one shorter i hope you can see this on my great wonderful animation here then you can see this thing basically vibrate the output vibrates visually well i mean lots of detectors going on but principally of course this only works if these mirrors here on either end don't vibrate just so right they told me the story when i visited the geo 600 detector whenever the post you know the package delivery guy drives too fast along the access way to their to their side there they can see the the mirrors vibrate and they're telling the you know the post guys to go slow and whenever a new post guy shows up destroys the measurement so what they do is they attach these mirrors to two level pendulums because pendulums actually when they have a high mass have a very good damping about 1000 and so with this triple pendulum here they can get a damping a passive damping of i think 1000 to the power of three that would be 10 to the power of nine right and then they do some active damping as well they precisely understand the behavior of the laser they even optimize and understand and measure and change and you know try to limit shot noise and so my point here is you don't have to understand the damping of a pendulum for your stupid clock but you do have to understand it very precisely if you want to distinguish the natural pendulum behavior from you know environment postal guys and little earthquakes if you want to distinguish that from whatever a gravitational wave does so all models are simplifications even the one for the gravitational gravitational gravitational wave detector but how and how far depends on a model's purpose let's look at another example i visited the bendelstein 7x stellar rate of fusion experiment in greifswald and i asked them so how do you guys model the plasma in the reactor and they told me well it's not so easy they have a one model that models it as a fluid it's called magneto hydrodynamics the other one models it as a mix of two gases right the the electron gas and the nuclear gas basically they can also model it as small particles that are disc-like and they can model it as point-like particles that move on this disc as on the circle around on this disc because of the magnetic field configuration and they use all of them right depending on what they want to do they use all of them obviously the mhd model on the left doesn't give you detailed information about a particular particle but if you want to characterize the overall behavior of the fluid as sorry of the plasma as a body that is inside the reactor then that's good enough and obviously each of these models have to make the same prediction when you get to a boundary case from which on you use the other representation another kind of physical constraint let's look at this example not sure if people recognize this is the configuration of the various accelerators at CERN the big ring is the large hadron collider and the smaller rings are earlier colliders that are now used as accelerators as pre accelerators before the hadrons enter the large storage ring and these the the LHC has these huge experiments you see the person standing there at the bottom these huge experiments which capture what happens when particles from these beams collide and again here is another picture here and again consider the people for reference the way these detectors work is basically that you have a whole bunch of different detecting elements arranged around the center collision point and when particles fly out then they hit these sub detectors as they're called and produces some kind of electronic signal and that is then used quick comment to the technical people here I get a lot of noise on my mumble channel not sure who should micro mute their their microphone there all right so here is a an illustration of one of these collision events you can see how the particles stream in from well can't see the particles but you can see the beam pipe at the center through which the two particles that collide travel and then you can see how the various collision products stream out and how they light up the various detector surfaces and what this shows here is the collision of two protons which forms a Higgs boson which then decays to two bottom quarks and to a W boson which then also decays how do we know right how the heck do they know what's happening what's going on here so let's first understand a little bit how these detectors work fundamentally what they do is they have a very strong magnetic field and if a particle that has some electromagnetic charge and most of them have otherwise it's hard to detect them actually then you know again a little bit of school physics laurence force and stuff like that the particle will be bent around a circular track by the magnetic field and then they have different kinds of detectors and for example if you look at a electron here that's the second example from the top it moves it leaves a trace in what's called the silicon tracker and then is it decays in the electromagnetic calorimeter and so if you look at the various different particles each of them leaves traces in a different combination of detecting elements so this gives you the type of particle and then through the bending you can measure its momentum and basically through type and momentum you get a unique characterization of what's going on you can even you know calculate the overall energy balance and stuff like that and the moment momentum balance it's very sophisticated really interesting stuff so and so in this case in the example I gave we have a Higgs boson that decays in various ways I've just called these particles a through f here doesn't really matter what they are and these decays they are multi-step so what the detector actually sees is what they call the final state which is these decay products what the scientists are interested in is what actually happens in the collision and so the challenges after observing the final state the decay products with the detector you want to calculate back what actually happened in the collision was there a Higgs or wasn't there a Higgs and what is the mass of that Higgs boson right that was the big question back then and I think 2013 when they discovered it by the way Einstein has nothing to do with this I just used the picture so in German we say there are Haken right it's not so easy there are there are problems and so I used this picture of a hook it doesn't really work in English but so what are some of the challenges they have to overcome well we have seen that the Higgs decays in multiple ways and each of these decays happens with a certain probability so they have to somehow untangle that the decays are non-unique as you can see in this example this hypothetical e particle can either arise from h to b to e or from h to a to d to e so just because you see an e you don't really know you know what happened in between then these collisions are not unique depending on how exactly you know how well the the protons align in their collision you either get hard collisions or soft collisions or any intermediate state and so the created particles the kind and their energy depends on whether it was a soft or a hard collision and all kinds of other parameters so that is also a source of complexity and then finally this detector I mean these detectors are marvels of engineering they're also terribly complex each of these components has their own failure modes and you know failure models and these are complex systems in themselves so again how do they know what they're observing and how do they know that with a you know with a with five sigma five sigma is a measure of the basically a measure of the trust in the result five sigma is the threshold in physics for a discovery and five sigma means that that there is a probability of zero point zero zero zero five seven something percent that the observation happened by chance and not and is not the effect that they assume in terms of physics and this is also quite related to the p-value that's used in statistics it's inversely proportional to the amount of data the more data you have the higher the trust and of course also inversely proportional to the size of the effect the bigger the effect the higher the trust by the way it's interesting to compare these five sigmas to the p-values they use in in medical trials right they are happy if they get five percent not zero point zero zero zero zero zero five seven it's a quite a big of big difference so how the hell do they do that well statistics right they simulate billions of events to understand the distribution of what will happen so how does this work so they have what's called an event generator it's a software model that simulates the physics of decay after a collision right so you use basically specify collision parameters is it a hard or a soft collision and then this thing gives you a distribution of particle vectors of basically what's streaming out of these of this collision site then they have what's called an experiment model it's called g and four it is a complete well obviously it's a simplification but it's a representation of the actual detector and of the physics in the detectors themselves because i mean the detectors work by basically running certain physical processes when the particles interact with the detector material that is simulated in this model this gives you then simulated detector hits then they have basically an electronic simulator that has a noise model for all of the electronics of these detectors and this gives you simulated raw data and they run billions of these simulated collisions in order to get what they call the background which is how what what would they see in this detector statistically if there wasn't any hicks right so they run these simulations without picks just the other nuclear processes and then they take the measurements and they compare and then this is the screenshot of what atlas saw and you can see that at 125 giga electron volts there is some kind of peak and that is the Higgs boson and they basically find this out through statistical calculations based on huge amounts of data right this is why they have some of the largest computing centers they run millions and billions lines of code in these various simulators developed over decades of course a good old Fortran is still the backbone they also use python for a lot and they have i think it's terabytes of data there's a whole worldwide distribution and scheduling infrastructure for you know calculating and running all these simulations there's a huge infrastructure behind this in order to get enough data to find out with some degree of trust that in this case they discovered the Higgs so how can we build trust in models well one way we can run lots of experiments with lots of data and then do statistics now experiments are nice because they give you controlled conditions they allow you to isolate various influencing factors by controlling for them right and so you can if you see stuff correlated it's relatively easy to derive causality so you know instead of just saying you know this happens and that happens and whenever this happens that happens you can also say because this happens that happens that is that is causality but sometimes you can't do experiments in climate or epidemiology or economics or many societally relevant questions you can't really run an experiment I mean you know in some sense we run an experiment with our atmosphere but but you can't build another planet and and run you know a climate experiment in that sense so what you have to do then is you have to fall back on just observation where you do not have controlled conditions you cannot isolate influencing factors at least usually you can't and so you really just get correlation and then you have to make an attempt at deriving causality but that's much harder sometimes you're lucky right you have natural experiments the economics profession you know tries to use that when they say in this country when they did this and that it gave them inflation and in this other country whether it was the following difference there was no inflation and so then because of that following difference they claim this is what produces inflation but of course again you you never know what else has changed right it's harder it's maybe a bit easier in epidemiology these days you know the factors aren't that different so maybe maybe looking at other countries that do stronger lockdowns is a good idea different story so what can we do again if we don't really have a way of running experiments in that sense let's say we want to run some kind of forecast and we know that this forecast depends on some kind of parameter p is a parameter that we have to tune in our model for the forecast to be correct but again we can't run we can't validate easily so we have no data for comparison because we want to you know forecast the future weather forecast is a good example well what we can do is we can forecast the past right so we can move back in time and run the forecast from a point in the past where we do know what actually happened in the real world and then we can tune our parameter p in this case p2 some value p2 is the best setting for p in order to fit our forecast best with the actual you know data that we know because they happened in the past and then we can continue using this parameter for the future this actually happens in weather forecasting there it's called re forecasting and these weather models that are continuously run the various weather for services they continuously basically go back and forth between re forecasting for a parameter tuning and also for you know filling in inputs that they don't know because you know there is not a way to find the temperature everywhere above the ocean only where there are ships right and a bunch of weather buoys so that ships can just go back and forth to do this kind of stuff and so this is really the backbone of weather forecasting so another way of building trust in models is re forecasting the past now all models are simplifications we've talked about that right and so the question is if we simplify which of potentially many parameters can we ignore which are important right so here is another model m it has three parameters p1 p2 and p3 i realize i used p2 as a value on the previous example here it's the name for a different parameter not a good decision i'm sorry so we have three parameters p1 through three and we don't know which of them is important well what we can do is we can scan through the ranges of all of these parameters and see which which which scan has the largest output or the i should say the largest consequence on the output of the model in this case we can see delta m for p3 is the largest one right so we know p3 has the largest influence by the way there might be an even larger influence when we combine the scanning of p1 p3 so we have a combinatorics problem i don't know i forget which scientists i talked with when i asked about that and they said well we don't we don't do these combinations it's too complicated so that's another simplification they treat each of these parameters as uh uncorrelated with others which might or might not be correct anyway so we find out that this model here is sensitive to the value of p3 and so what does this mean well it means we have to invest effort or computational power into determining p3 precisely or if we can do that again because we might not have a way of finding the value because we don't have you know temperature sensors all over the ocean maybe satellites can help but different topic then we have to always vary p3 and whenever we make a forecast make it for the whole range of p3 and see what this gives us and this is well more or less this is what ensemble forecasting is called in meteorology where they run different models different parameterizations of the same models and also different values of the input conditions well where they are not known in order to basically get a probability distribution of outputs right so they might vary the temperature at a given point run the forecast and they get a probability distribution of the output and then they either communicate the probability distribution to their users or they just give you the result that is most probable there is a weather forecasting service called meteor blue and i like them because if you look at this screenshot from the left you can see in german of course the trefsicherheit der wetter prognose is middle the reliability of this prognosis is sorry is medium and now you can click on a button and you now get what they call the multi model this is essentially a if you will sensitivity analysis it shows you what the different models forecast and you can nicely see if you actually go to the page how they diverge more over time they even give you the verification they they basically tell you how good this model was in the last three days so they give you a lot of information in actually if you want to invest the time in analyzing which how reliable the model is and this particular one is used by glider pilots i used the example from gup it's a favorite gliding site in southern france and people spend half the morning over this and other forecast to figure out what the weather will be now in epidemiology things are really simple right it's exponential growth meaning the infection rate will grow exponentially with some factor this b here is kind of what we know as r it's not exactly the same but for now it's good enough right but we also know it's not really like that right if there are no more people left that can be infected or more generally if there are no more resources resources which an exponential growth can if you will grow into then the growth has to stop it's called the logistic growth model as opposed to an exponential growth model and we know everybody knows now that if you will the the herd immunity is the step or is the amount of infection where there are no more people left to be infected or where at least the infection rate becomes quote inefficient from the perspective of the virus and the growth becomes slower and at some point increased infection stops so what else could there be right simple well there is this website called covid sim by a company called explosives and they have this covid simulator and actually there's a whole bunch of differential equations they use like the number of susceptible individuals um you know the individuals in various periods of infection the number of recovered individuals and people of number of people who have died so it's quite a bit more complicated actually it's way more complicated there's lots of different dynamical equations there are intervention effects you might want to model and there's a whole long range of parameters that you can set and that you have to set right and so there's an interesting effect that I figured out when I talked to one of these guys the the ratio at which the population reaches herd immunity depends on how they break down the age groups now I'm not talking about the actual age distribution in the population obviously if you have more old people more people will die we know that right that's our experience what I'm talking about here is whether you split you know your population into three groups 0 to 30 30 to 60 62 infinity or whether you split that same age pyramid into 10 or 15 subgroups the model makes different predictions who'd have thunk right so this is an example of a sensitivity analysis where you have to somehow figure out what is a good way for this age structuring in your model there's another nice example for sensitivity analysis in fusion so one thing they might want to optimize in their reactor is how do you maximize the distance between the plasma and the reactor wall because as you perhaps know that the plasma is very hot it's stabilized through a magnetic field and if the reactor sorry if the plasma hits the wall well it might damage the wall but more importantly the plasma will be damaged and might collapse so you you want to keep the plasma away from the wall and again that's the point of the magnetic field so one parameter you can tune is you can increase or decrease the field you can increase or decrease the gas density you can increase or decrease the number of neutral particles that are injected in or as a way of injecting energy basically you can change the temperature of the plasma point is that it's 10 to the 100 combinations and how do you do that right so how do you make this optimization well one way is you have to use the simplest of these various representations of the plasma I mentioned before so this is something they do with the mhd the fluid representation because it's the simplest it's the most efficient but also again even then you know we only have 10 to the power of 10 to the 85th atoms in the universe so 10 to the power of 100 combinations won't work so this is where in these numerical models computer science comes in right where you have various different optimization algorithms numerical solvers you know things like hill climbing simulated annealing not something I want to cover in this talk but I thought it was important to point out that it's not just about finding the right model and the right abstractions and the right sense you know deciding on the right parameters there's a lot of cleverness going on here in the programming and how you perhaps then also run this on a parallel computer by the way little anecdote I talked to a meteorologist a few days ago actually for a podcast interview and he said it's going to be really painful when in the next or next next generation of supercomputers everything will be computed on graphics cards because we can basically throw away all our code and have to rewrite everything so that's going to be painful so how can we build trust in models we can vary parameters do a sensitivity analysis and analyze the impact now I want to re-emphasize something because somebody recently on Twitter told me that this modeling for covid seems to be shit because the forecasts were wrong and this person understood something misunderstood something completely and that is forecasting versus scenarios so let's say you have a model and you want to make the model correct you know make the right predictions but you have but you can't take into account all of these millions of different parameters you have to find out which ones are the most relevant so that's when you as we just learned you run a sensitivity analysis right for example you want to find out the cloud formation depending on temperature and humidity the radiated energy by the earth depending on serious cloud coverage or the amount of precipitation depending on layering of the atmosphere right once you figure that out you put that into your model and then you can actually measure various data in the world and make forecasts actual forecasts that the output of the model claims that this is how the weather will behave okay that's what we do in forecasting now let's look at another example let's say we want to figure out or we're going to understand how the one-to-one infection rate are depends on how much virus people shed when they breathe um how the virus shedding depends on the kind of mask you wear and how the one-to-many infection rate depends on the rate of interpersonal contacts in this case you can then also make a sensitivity analysis run it on simulated data and then what you get isn't the forecast because you don't know how people will behave but what you get is scenarios you can say that if you guys behave in the following way then you know the infection rate will increase so there's no point in saying the covid forecast was wrong because it didn't make a forecast it gave us scenarios that um you know give us hints about how we should behave and of course it's a bit frustrating right we have a healthcare model that says that early and hard lockdown works best right because it's easier to slow down exponential growth when the absolute numbers are low an economical model told us that early and hard lockdowns work best because shorter lockdowns for the reasons above means less total cost there's also psychological analysis that says well early and hard lockdown works best because if you become harder over time people are already fatigued they have lockdown fatigue so they won't you know comply so germany still started a lockdown light and did it too late so it's quite a bit useless if you have all these models that make these useful and i guess rather reliable scenario forecasts if you will but then we don't care okay but that's just a side note it frustrates me a great deal so i had to include that into the talk all right so again going back to building trust in models we were at this parameter variation story but there is another way how we can build trust in models and that is if we do have an explaining model one that doesn't just tell us the how but also the why then we can inspect that model and can and see if this makes sense right is this explanation plausible does it make sense for example if your plasma optimization thingy comes up with a great solution for maximizing the distance of the plasma from the reactor wall but it predicts a pressure that is twice as high than what your reactor vessel can bear it's not a good forecast it's not a good result right but you can only do this because you can inspect the solution the model created for sensibility if you will so you can check explanations if they are there so let's look at how you get from data to the model you probably all remember this picture right it's the first ever image of a black hole well it's sort of an image or i should rather say this is an image but what the EHT the event horizon telescope imaged wasn't really an image right because of how the EHT works let's let's see how this works so the EHT is a radio informator interferometer which means you computationally combine several telescopes and there's lots of computing going on so in this case they combined a bunch of telescopes all over the world in something that's called very large baseline interferometry the very large baseline comes from the distance between the telescopes and that is relevant because the resolution the the whole computational thing can resolve depends on the length of these baselines so the longer you the further you can distance these telescopes the higher resolution is the picture well picture this thing is going to take here is a picture of one of these telescopes okay this particular one ethics bulk is not used in the EHT because it can't observe at the perspective wavelength but it's these kinds of telescopes smaller some of them all over the world now how does this work so these different telescopes they look at the same spot in the sky right but because they are some distance apart the the distance from the telescope to that point in the sky is different right so they see the same wave the light wave that arrives you know with a certain phase difference because there is this difference d in a distance offset right and of course because light speed is constant this is proportional to some time difference as well now as the earth rotates and also as you look at different points in the sky this difference also sorry this distance also changes because the angle of the object in the sky is different from each of the telescopes so if you take these two observations or realizations and then apply a whole bunch of mathematics which i'm not going to go into i probably also couldn't in detail you get the following equation which again not important in detail but what this this equation shows is a Fourier transform of the sky so the EHT actually observes a Fourier transformation a spatial it observes the spatial frequency distribution of the radiation across the sky okay and then the actual data point is basically there's a we can we can we can basically observe this amount of radiation with this frequency and by that i mean spatial frequency at the following orientation and then they combine all of this data and the more different lengths of baselines they have and the more different orientations of baselines they have the better but still in any case you know they only have like whatever seven telescopes so they have 10 different baselines and the orientation of these baselines well there's a only a limited number of these orientations sure the earth rotates giving them a bit more but still it will be incomplete data so what they'll have to do is they have to do an inverse Fourier transformation of incomplete data the question they're answering is which is which mathematical description of the object in the actual space domain represents or approximates best the data that has been seen by the telescope in the frequency domain right and so let's look at a simple example of this kind of problem on the left side you can see a bunch of measurements right some data and on the right side you see a a mathematical model in this case a horizontal line a simple equation y equals y0 no dependency on x and the total error of that model relative to the observation is you can imagine that as the sum of all the red bars right the the difference between what the model predicts what was actually observed so there could be different mathematical models right that approximate that same data question is which one is the right one right so again in the eht case what visual shape actually fits best to the observed observed incomplete data in the frequency domain and this example here on the right is best right because it has a total error of zero but actually what this is is an overfit if you put enough terms into your polynomial you can fit every data with some kind of polynomial but while it reproduces the data perfectly it makes no no useful prediction because there's no way you can tell what the next like if you extrapolate to the right would be and it's no it's no abstraction it's not a useful model it's just a encoding if you will of the measurements as a formula so by the way this is what's called overlearning in neural networks just as a side note so how do you decide between the three remaining ones well you decide based on knowledge about the system at hand right you know something about how your observed system should look like because you understand the physics you have a simple analytical intuitive model about what's going on so in this case the model of the black hole tells us that between the center of the black hole and a distance of 2.6 times the so-called schwarzschild radius you will see black because in the middle you look at the black front side of the black hole a little bit further out the your light rays or the light rays coming from the black hole are actually bent by the strong gravitation so that you can see the back side of the black hole which is also black so we know that you know up to 2.6 rs it'll be dark and outside there we will see a glow from radiation that is accelerated around the black hole from the gravitational force so there must be something glowing and there must be a black thing in the middle and we even know roughly how big this is because we can calculate the schwarzschild radius through other means or at least a guesstimated and so we kind of know what it will look like well it kind of look like this donut and so knowing that we can use this knowledge and basically help interpreting the incomplete data now of course the problem is how is that not self-fulfilling how do we avoid observing exactly what we want to observe you know then we don't have to observe it because we know what will look like anyway it will fake it so what they did here is they used different three different deconvolution algorithms to do this kind of inverse Fourier transformation with three different software packages for independent teams who weren't allowed to see everybody else's data they also worked with synthetic data they used the same magneto hydrodynamic hydrodynamic magneto hydrodynamic simulations of black holes and of other shapes right they they they produced such data of i don't know rectangles and and you know if the if the if the software still produced a black donut as a result then something fishy is going on right and of course they still minimized the error of the fitting so there's a lot of work going into this reverse analysis of the of the data it's fascinating to read there's a nice book by high no faike one of the leaders of the project i think it's called light in the dark or something that talks about that and other things so again does the explanation make sense right we know how the physics work so we can say something about what what we expect so how far can we trust models here's a statement i've read if we had started the lockdown one week earlier 36,372 fewer people would have died this is what i call pointless precision because the precision of whatever your model predicts must be proportional somehow to the precision of the input and the parameterization and obviously we there's a lot of things we don't know precisely for this virus and how society reacts right so this leads us to this observation that there are models that are qualitative in nature it only tells us tendencies right if input a increases output x will also increase there are relative models which give you comparative influences like input a influences the output x twice as much as input b useful and then there are quantitative models which actually give you numbers right if you increase by 10 percent the input a then output b decreases by 33.3 percent now formula doesn't mean numerical sorry formula doesn't mean quantitative there is this drake equation which tells you how many civilizations there are in our galaxy right and it's this equation it has all kinds of factors like the average rate of star formation the fraction of planets that could support life fraction of civilizations that send detectable signals blah blah blah right it's a formula has lots of factors so it's numeric right it gives you quantitative outputs no it doesn't this is really just a simple parametric model because none of these parameters has really known quantities we have rough boundaries or or you know boxings for each of those but not really so using an just because you have an equation doesn't mean that it's a quantitative model right it might just be it basically just says which things influence this thing and whether it's proportional or antiproportional so let's look at fundamental limitations of this whole modeling business so first is chaos which means that a small change in the initial conditions leads to huge effects over time so you've probably all seen this chaos pendulum right it's a pendulum with two hinges is that the word i don't know and if you take the pendulum and put it into a certain you know position and let it go it will oscillate in some way and if you then take the pendulum and try to put it into the same position again after a few seconds it will move completely differently because the behavior is chaotic and you're not able to uh put the pendulum in exactly the same initial condition also you don't know exactly what other factors influence it you know there might be some wind going on or whatever so the other example that everybody talks about in the context of chaos is like if the butterfly leaves you know starts flying in brazil somewhere this leads to thunderstorms in europe i did ask uh various meteorologists here whether they actually think that's true and i mean nobody disagrees that the weather is chaotic but i think they said that this very small effect will very likely not lead to a thunderstorm although in theory it could so the point is that both of these systems are deterministic right there is no randomness right there's no randomness in the physics of the weather of the of the atmosphere the quantum effects really aren't relevant on that level there's also no uh randomness in this in this pendulum but we don't know the interactions for example the friction in these two hinges and the initial conditions precisely enough if we if we if you knew them priced precisely enough then we could make a perfect prediction right so or stated less absolutely more input gives you a better prediction more weather stations more satellites more ships more airplanes with sensors gives you a better prediction so this chaotic behavior of the weather really is the fundamental limit for weather forecasts because right for any given data quality and computational power right so that's why i think what they told me was that roughly every 10 years the useful range of weather forecasts becomes one more one day more right and so that is not just because the computers get faster it is also just because they have more inputs and of course because they understand the mechanism better so if we can't predict the weather you know for more than a few days how then will the climate skeptics say how then can we predict climate right this is all fake they can't even predict the weather and of course this is bullshit because climate is a statistical prediction it's not relevant if it will rain on 5th march 2038 in stuttgart right that's not what climate models do they make statistical predictions and just to draw home this difference again going back to the fusion example we really cannot because of chaotic behavior we cannot predict the position of each molecule or element atom in this mix of two gases kinetic model here but what we can do is we can calculate the averages in this case the averages of these movements is called temperature and pressure right and so we can calculate that precisely but you know despite not being able to forecast every molecule precisely so there is an interesting thing in climate science it's called attribution science and so what these guys ask is is this extreme weather event that happened there is this cost by climate change right really what they're asking is with which probability is this kind of weather event caused by climate change at this location on the earth at this time of year right how do they do that well they take the current atmosphere they modify the initial conditions like temperature humidity pressure stuff like that they scan through these and then they run the you know climate models or weather models well this is the same models anyway it's a question of parameterization these days and they figure out with how many of these modified of these initial conditions will we get this kind of extreme weather event then they take the atmosphere without the human impact they remove the co2 that we've introduced over the last whatever 50 years and the methane because I mean that is relatively well known how much this is they do the same modification of initials run the weather forecasts and they get another ratio of extreme weather still happening versus not happening so basically they get different degrees of robustness of that extreme weather phenomenon depending on whether they run the real atmosphere or the one without human impact and then they basically take the difference between the two and this gives you the probability of this weather event being caused by climate change I thought that's a very ingenious use by climate models of course it also really takes a lot of computational power because you have to run these models lots and lots and lots of times now you could say well this whole chaos stuff it's not really fundamental it's just because we can't measure precisely enough really there isn't any chaos it's just because our engineering is too bad well let's look at cellular automata a cellular automaton is a mathematical abstraction that has a row of cells a cell can be either alive or dead and then we observe how the cells evolve over time or generations so the generations or time is the y-axis and then we have a whole bunch of cells on the x-axis and of course generation i depends on the previous generation and each value of a cell depends on the value of the two neighbors in the previous generation so there's an so we can see this here in this in this diagram we have generation i minus one and the value of the cell with a question mark in generation i depends on whether you know the three other gray boxes cells whether they were alive or dead and then we have an example rule the one here that says you know if the if all three predecessors were alive then next time in the next next generation we're dead if it's you know in this case it's only if the three predecessors were all dead then the next one in the next generation is going to be alive so some kind of birth from nothing point is if we plot this rule one there is a way of classifying these rules and giving them names if we run this rule one over 20 or 100 or 1000s or whatever how many steps you can see that a regular pattern occurs right and so we can give a formula for this rule in this case it's very simple because only if well only in one case do we basically we basically flip the values of the of the previous cells except for these two cells in the middle next and right to the center they always stay zero so point is there is an iterative formula right and it takes into account the value of a previous generation that's why it's iterative but we can also give a closed formula one that does not rely on the previous generation right we can distinguish between even and odd generations and just state which ones are alive and which ones are dead so that's fine that's deterministic behavior we can predict it for any number of generations and we don't have to iterate now here is another rule rule 13 here is how this one looks after 20 steps and here is how it looks after 100 steps and in these two other illustrations top right you can see how a simple change in the initial conditions how it spreads through the whole basically the whole row so this is actually chaotic behavior and it's chaotic behavior in pure math there is nothing going on with measurement it's a property of the math so we cannot predict the value of a cell in the i-th generation we have to run the iteration this is also called computationally irreducible in fact it's a very good secure random generator because the behavior is random it has been proven to be random it's all work by Stephen Wolfram who has written this book a new kind of science it's interesting not everybody takes it too seriously but these this work on cellular automata is is not disputed so it's it's it's cool stuff so chaos is a fundamental property of our universe and we have to somehow you know if it occurs in our systems because our systems are complex as we'll see this limits the range to which we can say something useful with models another example is emergence you've all heard about how ant colonies exhibit these very sophisticated group behaviors even though presumably the brains of these ants are rather small and simple so what's going on here well we have what's called agents each is an agent it has simple rules it behaves based on for example based on pheromones and smells and stuff like that but then we have lots of interacting of these agents they interact in different ways and then they produce some kind of complex ultimate outcome and the point is that we cannot predict the outcome just by observing and understanding the rules we have to actually run also iteratively the system and see what can potentially come out and then the thing is that in lots of systems we care about they work exactly this way right politics sets rules and then we all as society behave in some way we interact based on rules maybe not based on rules people know there's lots of game theoretical personal optimization going on right and so we really can't predict right so I did an interview with somebody from to delft Igor Nikolich about socio-technical systems he investigates how society interacts with technical systems and what resulting behaviors could come out and again he can't really predict because of this emergence but what he can do is he can run these agent models and then look at the scenarios that could potentially happen and then discuss this with stakeholders potentially go back to change the rules and run again and maybe iterate to a better set of rules there's no proof there right there's no guaranteed you know understanding but it can help he has lots of interesting cases there there's also a very nice counter example from from politics in Germany recently in Berlin they were setting an upper limit for rents right so the idea was to therefore have lower rents and so less affluent people will find apartments what actually happened at least initially I haven't checked in the last few weeks is that 70 37 percent more apartments were say were sold instead of being on the market for rent because you know this just became less interesting for people for the renters 28 percent fewer apartments were on the market 200 percent more inquiries per apartment and there was a shadow rent for the case when the law wasn't complied with the German basic law so the point is this failed right and it failed because you know one tried to set simple rules in an obviously complex system with lots of emergent behaviors now I do think that one maybe could have anticipated that even without running complex agent-based simulations but still it's an example of how rulemaking in politics really is challenging because you just don't know how the system will react once you put some kind of stimulus on it by changing regulations there is also an interesting example from cellular automata game of life you've probably all heard it right it's basically a 2d cellular automaton where the question whether the cell marked with x in the next generation depends on not just you know it depends on what happens around it so this phenomena like you know dying from loneliness dying from overpopulation healthy environment reproduction the rules don't really matter so here is something so this runs a game of life simulation for I don't know how many oscillations here or how many runs and you can see there are things oscillating back and forth there are so-called spaceships there are glider guns these are the technical terms for these seemingly coordinated behaviors right so who would have thought that from this relatively simple set of very local rules seemingly coordinated phenomena can happen almost looks like ants walking around a track right so again pure math can produce complex or coordinated seeming behavior from simple rules that do not somehow presuppose that kind of coordinate behavior so what can we do here right we we just said we cannot predict behavior of the whole system by examining the rules but we can what we can do is we can set a new we can define a new set of rules that specify that that are valid for the system as a whole like whatever if you have more than a hundred ants and the temperatures over 30 degrees they'll come out of their bunk or whatever it's called and will start rummaging around you know stuff like that you can observe or create higher level theories in fact this happens all the time right because the behavior of ants might be based on biology but biology is just applied chemistry and chemistry as we all know is just the physics of the of the of the electrons right it's a joke among physicists so but the point is we couldn't predict the behavior even of a single ant if we try to do it do that with physics even though it is physics that drives the ant everything is driven by physics right we could try with chemistry but that's probably also a bit too low level we can maybe be more successful with biology or medicine by doing an MRI or an fMRI of the brain of an ant right i mean i'm not a biologist as you can probably tell but the point is what we're doing all the time we we're stacking abstractions on top of another of one another we're building models of aggregated subsystems in order to understand those there is a very nice quote by etzger dykstra a famous computer scientist who says the purpose of abstraction is not to be vague but to create and but to create a new semantic level on which one can be absolutely precise right this also relies to this averaging stuff in the in the fusion reactor so very very cool kind of framework that that humanity has has built for themselves there another example of stuff that can go wrong in modeling path dependence right so there is a complex structure of decisions we've made over time i don't know just a cool screenshot right but we are somewhere out there on the outer edges of this path how did we get here and the answer is we got here because there were all these random and context and resource dependent and cultural and political decisions how we that we've made over time for lots of phenomena if you ask so why the hell is something like the way it is in our society there really isn't a good answer it could just as well be different but because of all these random things over the decades and centuries we've just ended up there there isn't a good reason it just happened right and the only way to predict if you will is by going the path you cannot simplify you can't run an analysis and say well obviously you know uh i don't know the society has to do things this way it's not like that even the obvious example is evolution right evolution is not a deterministic process it just you know because of all kinds of constraints and randomness and you know a meteor impacted the earth stuff happened all of these things are examples of complexity there are other things like hidden links feedback cycles tipping points power loss if you listen closely you have heard all of those in the discussion about climate right which just drives home the point that our climate is a complex system which makes it hard to control it in a deterministic way and by control I don't necessarily mean just geoengineering right it's really hard to understand and and and decide what to do I mean obviously we have to reduce co2 I'm not discussing that but in detail I guess maybe the point is that geoengineering is risky so last point before we wrap up this talk unknown unknowns models almost always represent somehow model knowledge and experience there's no completely unknown stuff in models because nobody could have put it in there right so there might be also biases and prejudices and we won't ever run a sensitivity analysis for unknown unknowns because we don't know them we were not aware that they're unknowns so we're not testing the model for what it would mean if that unknown parameter would have a different value we don't know it right so the really nice example uh I forget where I have it from it's very well known I actually know where I have it from but I forget that whatever it doesn't matter so you know turkey turkey gets fed by the farmer turkey likes the farmer because you know it's the guy who brings food so based on Bayesian reasoning updating its own perception the turkey builds a model of the farmer in their brain and you know becomes more and more you know on the good side but then on day 100 thanksgiving the farmer comes and kills the turkey of course the model could not have predicted that right because there was nothing in the experiential world of the turkey there was no indication based on which the turkey could have updated the model in their brain to maybe not be so optimistic because you know one day thanksgiving comes around so again so this means that when we think about uh what we will all die from we as a as human kind right we can do some probabilistic modeling that estimates the likelihood of dying from volcanic super volcanic eruptions or meteors hitting or natural pandemics because we have experience from the past but we cannot use this approach to predict well how likely it is we're gonna die from our own nuclear wars or ai eating us or a man-made pandemic I did an interview with toby or to wrote a book called the precipice where he discusses these likelihoods as well interesting story so was this crane accident here consequence of an unknown unknown I don't think so because they knew how important it is that the hook doesn't break right so they put a safety factor of three they told me into the hook design but if the process in manufacturing fails well what do you do right you can't your model won't predict so all right models they're not perfect but what else there is a nice example that maybe uh uh you know gives a bit of a counterpoint to detailed modeling there is something called a fast and frugal tree which is a very simple decision tree basically to decide whether a patient who shows up in the emergency room should be moved to the intensive care unit or to a regular care unit and as you can see there is only basically very few three yes no answers we don't have to discuss what these mean in detail this thing performs better than a machine learning model that takes dozens of factors into account so the doctors would say something like how much I'm always like macht which is a german for well we've always known that we don't need this stupid machine learning stuff right but the point is more data more detailed modeling doesn't always mean better outcome right sometimes common sense and just experience of humans is something that is relevant especially also when interpreting models and of course models change over time right that's science as we get new insights we update how our models work from Newton to Einstein to a grand unified theory maybe at some point so models are everywhere they use the use of models allows us to forecast developments and quantity quantify uncertainties that's cool models make influencing variables and their effects explicit right the explanatory models can even help understand something models are the basis for constructive discussion this is really important when you have a model you can constructively disagree whether some additional parameter should be taken into account or whatever you cannot do that if you just run around and claim shit right as some people do in this pandemic but of course boundary conditions and limits of modeling must be considered and in the end you know in case of doubt crap in crap out is always true here as well all right this brings me to the end of the talk interestingly I took two minutes longer than in my in my trial sorry for running a little bit over time but I'm done now there is a book that you might want to check out and of course there are the podcast episodes listed here that are most relevant to the stuff we talked about obviously 330 more but these are the ones that have direct influence on this talk second with the weather path dependence isn't just a lack of proper accounting of state or if the person asking missed something there sure I mean it's similar to this problem with chaos right if you could if you could track everything to arbitrary detail and could keep track of everything then maybe path dependence would not be a thing but we know that's not possible in practice I think that's it from my standpoint right now there are on any more questions I am aware of so chat has any more there is something more in the IRC doesn't look like it yeah I think we can end here