 I have the great pleasure to announce Joscha who will give us a great talk with the title The Ghost in the Machine and he will talk about consciousness of our mind and of computers and somehow also tell us how we can learn from AI systems about our own brains and I think this is a very curious question so please give it up for Joscha. Good evening. This is the fifth of a talk in a series of talks on how to get from computation to consciousness and to understand our condition in the universe based on concepts that I mostly learned by looking at artificial intelligence and computation and it mostly tackles the big philosophical questions. What can I know? What is true? What is true? Who am I? Which means the question of epistemology of ontology of metaphysics and philosophy of mind and ethics and to clear some of the terms that we are using here what is intelligence? What's a mind? What's a self? What's consciousness? How are mind and consciousness realized in the universe? Intelligence I think is the ability to make models. It's not the same thing as being smart which is the ability to reach your goals or being wise which is the ability to pick the right goals but it's just the ability to make models of things and you can regulate them later using these models but you don't have to and the mind is a thing that observes the universe a self is an identification with properties and purposes that's what the thing thinks it is and then you have consciousness which is the experience of what it's like to be a thing and how are mind and consciousness realized in the universe this is commonly called the mind-body problem and it's been puzzling philosophers and people of all proclivities for thousands of years. So what's going on? How is it possible that I find myself in the universe and I seem to be experiencing myself in that universe? How does this go together and how's this what's going on here? The traditional answer to this is called dualism and the conception of dualism is that in our culture at least this dualist idea that you have a physical world and a mental world and they coexist somehow and my mind experiences this mental world and my body can do things in the physical world and the difficulty with this dualist conception is how do these two planes of existence interact because physics is defined as causally closed everything that influences things in the physical world is by itself an element of physics so an alternative is idealism which says that there is only a mental world the only exist in a dream and this dream is being dreamed by a mind on a higher plane of existence and difficulty with this is very hard to explain that mind of a higher plane of existence was put there why is it doing this and in our culture the dominant theory is materialism and basically there's only a physical world nothing else and the physical world somehow is responsible for the creation of the mental world it's not quite clear how this happens and the answer that I am suggesting is functionalism which means that indeed we exist only in a dream so these ideas of materialism and idealism are not in opposition they are complementary because this dream is being dreamed by a mind on a higher plane of existence but this higher plane of existence is the physical world so we are being dreamed in the neocortex of a primate that lives in a physical universe and the world that we experience is not the physical world it's a dream generated by the neocortex the same circuits that make dreams at night make them during the day you can show this and we live in this virtual reality being generated in there and the self is a character in that dream and it seems to take care of things it seems to explain what's going on explains why miracles seem to be possible why I can look into the future but cannot break the bank somehow and even though the theory explains this how shouldn't I be more agnostic and there are alternatives that I should be considering maybe the narratives our big religions and so on and I think we should be agnostic so the first rule of epistemology says that the confidence in a belief must equal the weight of the evidence supporting it once you stumble on that rule you can test all the alternatives and see if one of them is better and I think what this means is you have to have all the possible beliefs you should entertain them all but you should not have any confidence in them you should shift your confidence around based on the evidence so for instance it is entirely possible that this universe was created by a supernatural being and it's a big conspiracy and it actually has meaning and it cares about as in our existence you mean something but there is no experiment that can validate this a guy coming down from a burning mount for burning bush that you've talked to on a mountaintop that's not a kind of experiment that gives you valid evidence right so intelligence is the ability to make models and intelligence is a property that is beyond the grasp of a single individual a single individual is not that smart we cannot figure out even chewing complete languages all by ourselves to do this you need an intellectual tradition that lasts a few hundred years at least so civilizations have more intelligence than individuals but individuals often have more intelligence than groups and whole generations and that's because groups and generations tend to converge on ideas they have consensus opinions and very very of consensus opinions because you know how hard it is to understand which program and language is the best one for which purpose there's no proper consensus and that's a relatively easy problem so when there's a complex topics and all the experts agree there are forces at work that are different than the forces that make them search for truth these consensus building forces they're very suspicious to me and if you want to understand what's true you have to look for a means motive and you have to be autonomous in doing this so individuals typically have better ideas than generations or groups but as i said civilizations have more intelligence than individuals what does this civilization intellect look like the civilization intellect is something like a global optimum of the modeling function it's something that has to be built over thousands of years in an unbroken intellectual tradition and guess what this doesn't really exist in human history every few hundred years there's some kind of revolution somebody opens the doors to the knowledge factories and gets everybody out and burns down the libraries and a couple generations later the knowledge worker drones of the new king realize oh my god we need to rebuild this thing this intellect and then they create something in its likeness but they make mistakes in the foundation so this intellect tends to have scars like our civilization intellect has a lot of scars in it that make it hard to difficult to understand concepts like self and consciousness and mind so the mind is something that observes the universe and then humans and neurotransmitters are the substrate and the human intellect and the working memory is the current binding state how do the different elements fit together in our mind and the self is the identification is what we think we are and what you want to happen and consciousness is the contents of our attention it makes knowledge available throughout the mind and civilization intellect is very similar society is observed the universe people and resources are the substrate the generation is the current binding state and culture is the identification with what we think we are and what we want to happen and media is the contents of our attention and make knowledge available throughout society so the culture is basically the self of civilization and media as its consciousness how is it possible to model a universe let's take a very simple universe like the Mandelbrot fractal is can be defined by a little bit of quote it's a very simple thing you just take a pair of numbers you square it you add the same pair of numbers and we do this infinitely often and typically this goes to infinity very fast and there's a small area around the origin of the number of pairs so between minus one and plus one and so on where you have an area where this converges where it doesn't go to infinity and there's this where you make black dots and then you get this famous structure the Mandelbrot fractal and his because this divergence and convergence of the function can take many loops and circles is on a very complicated shape a very complicated outline actually infinitely complicated outline there so there's an infinite amount of structure in this fractal and now imagine you happen to live in this fractal and you are in a particular place and you don't know where it is where that place is you don't even know the generator function of the whole thing but you can still predict your neighborhood so you can see oh my god I'm in some kind of spiral and it turns to the left and goes to the left and goes to left becomes smaller so you can predict and suddenly it ends why does it end a singularity oh it hits another spiral so there's a law when a spiral hits another spiral it ends and something else happens so you look and then you see oh there are certain circumstances where you have for instance an even number of spirals hitting each other instead of an odd number and then you discover another law and we make like 50 levels of these laws and this is a good description that locally compresses the universe so the Mandelbrot fractal is locally compressible we find local order that predicts the neighborhood if you are inside of that fractal the global modeling function of the Mandelbrot fractal is very very easy it's an interesting question how difficult is the global modeling function of our universe if we know it maybe it doesn't help us that much if we'll be a big breakthrough for physics when we finally find it it will be much shorter than the standard model I suspect but we still don't know where we are and this means we need to make a local model of what's happening so in order to do this we separate the universe into things things are small state spaces and transition function that tells you to get from how to get from state to state and if the function is deterministic it is independent of time it gives the same result every time you call it for an interdependistic function it gives a different result every time so it doesn't compress well and causality means that you have separate several things and they influence each other's evolution through a shared interface right so causality is an artifact of describing the universe as separate things and the universe is not separate things it's one thing but we have to describe it as separate things because we cannot observe the whole thing so what's true there seems to be a particular way in which the universe seems to be and that's the ground truth of the universe and it's inaccessible to us and what's accessible to us is our own models of the universe the only thing that we can experience and this is basically a set of theories that can explain the observations and truth in the sense is a property of a language and there are different languages that we can use like geometry and natural language and so on and ways of representing and changing models are languages and several intellectual traditions have developed their own languages and this has led to problems our civilization basically has as its founding mist this attempt to build this global optimum modeling function this is a tower that is meant to reach the heavens and it fell apart because people spoke different languages the different practitioners in different fields and they didn't understand each other and the whole building collapsed and this is in some sense the origin of our present civilization and we are trying to mend this and find better languages so whom can we turn to we can turn to the mathematicians maybe because mathematics is the domain of all languages mathematics is really cool when you think about it it's a universal code library maintained for several centuries in its present form there is not even version management it's one version there's pretty much unified name space they have to use a lot of the unicode to make it happen it's ugly but there you go has no central maintainers not even a code of conduct beyond what you can infer yourself but there are some problems at the foundations that they discovered see can you infer this is good conduct counter okay power to you in it in 74 discovered when you looked at the cardinality of a set that when you described natural numbers using set theory that the cardinality of a set grows slower than the cardinality of the set of its subset so if you look at the set of the the subsets of a set is always larger than the cardinality the number of members of the set clear right if you take the infinite set has infinitely many many members mega you take the cardinality of the set of the subsets of the infinite set it's also an infinite number but it's a larger one so it's a number that is larger than the previous omega okay that's fine now we have the cardinality of the set of all sets right we make the total set the set where you put all the sets that could possibly exist and put them all together right that has also infinitely many members and it has more than the cardinality of the set of the subsets of the infinite set that's fine but now you look at the cardinality of the set of all the subsets of the total sets the problem is that the total set also contains the set of its subsets right it's because it contains all the sets now you have contradiction because the cardinality of the set of the subsets of the total set is supposed to be larger and yet it seems to be the same set and not the same set it's it's an issue so mathematicians got puzzled about this and the philosopher Bertrand Russell said maybe we just exclude those sets that don't contain themselves right we only look at the set of sets that don't contain themselves isn't that a solution now the problem is does the set of the sets that doesn't contain themselves contain itself if it does it doesn't if it doesn't it does that's an issue so David Tilbert who was some kind of a community manager back then said guys fix this this is an issue if mathematics crashes we are in trouble please solve mathematics and people got to work and a short amount of time a court girdle who had looked at this and Ernest said oh there's the issue you know as soon as we allow these kinds of loops and we cannot really exclude these loops then our mathematics crashes so that's an issue it's called onion shitebarkite and then ellenturing came along a couple years later and he constructed a computer to make that proof he basically said if we build a machine that does these mathematics and the machine takes infinitely many steps sometimes for making a proof then we cannot know whether this proof terminates so it's a similar issue for the onion shitebarkite that's a big issue right so we cannot basically build a machine and mathematics that runs mathematics without crashing but the good news is Turing didn't stop working there and he figured out together with ellenture church not together independently but at the same time that we can build a computational machine that runs all of computation so computation is a universal thing and it's almost as good as mathematics computation is constructive mathematics the tiny neglected subset of mathematics where you have to show the money in order to say that something is true you have to find that object that is true you have to actually construct it so there are no infinities because you cannot construct an infinity you add things in it you have unboundedness maybe but not infinity and so this part of computation is the one that can be implemented it's constructive mathematics it's the good part and computer is very easy to make and all universal computers have the same power that's called the church Turing thesis and Turing didn't even stop there the obvious conclusion is that human minds are probably not in the class of these mathematical machines that even god doesn't know how to build if it has to be done in any language but it's a computational machine and this also means that all machines that human minds ever encounter or mathematics that human minds encounter will be computational mathematics so how can we bridge the gap from mathematics to philosophy can we find a language that is more powerful than most of the languages that we look at mathematics which are very narrowly defined language so every symbol we know exactly what it means when we look at the real world we often don't know what things mean in our concepts we're not quite sure what they mean like culture is a very big ambiguous concept so what i said is only approximately true there can we deal with this conceptual ambiguity can we build a programmer language for thought where words mean things that are supposed to mean and this was the project of Ludwig Wittgenstein he just came back from the war and a lot of thoughts and then he put these thoughts into a book which is called The Tractados and it's one of the most beautiful books in the philosophy of the 20th century and it starts with the words the word the world is all that is the case the world is the whole of the facts not the things the world is certain about the facts and because these are all the facts and so on this book is about 75 pages long and it's a single thought it's not meant to be an argument to convince the philosopher it's an attempt by a guy who's basically a coder and an eye scientist to reverse engineer the language of his own thinking and make it deterministic to make it formal to make it mean something and he felt back then that he was successful and had a tremendous impact on philosophy which is largely devastating because the philosophers didn't know what he was on about they thought it's about natural language and not about coding and he wrote this in 1918 so before Alan Turing defined what a computer is but he would already smell what a computer is he already knew about university of computation he knew that a NAND gate is sufficient to explain all of Boolean algebra and it's equivalent to other things so what he basically did was he preempted the logics program of artificial intelligence which started much later in the 1950s and he ran into troubles with it in the end he wrote a book philosophical investigations where he concluded this that his project basically failed and that there is an uh because the world is too complex and too ambiguous to deal with this and symbolic AI was mostly similar to Wittgenstein's program so classical AI is symbolic you analyze a problem you find an algorithm to solve it and what we now have in AI is mostly sub-symbolic so we have algorithms that learn the solution of a problem by themselves and it's tempting to think that the next thing what we have and it will be meta-learning where you have algorithms that learn to learn the solution to the problem meanwhile let's look at how we can make models information is a discernible difference it's about change all information about change the information that is not about change you cannot see a causal effect on the world because it stays the same right and the meaning of information is its relationship to change in other information so if you see a blip on your retina the meaning of that blip on your retina as the relationships you discover to other blips on your retina it could be for instance if you see a sequence of such blips that are adjacent to each other first order model you see a moving dust motor moving dot on your retina and a higher order model makes it possible to understand oh it's part of something larger there's people moving in a three-dimensional room and they exchange ideas and this is maybe the best model you end up with that's the local compression that you can make of your universe based on correlating blips on your retina and for those blips where you don't find a relationship which is a function that your brain can compute they are noise and there's a lot of noise in our retina too so what's a function a function is basically a gearbox it has an input levers and one output lever and when you move the input levers they translate to a movement to the of the output levers right and a function can be realized in many ways maybe you cannot open the gearbox and what happened in this function could be for instance two sprockets which do this or you can have the same results with levers and pulleys and so you don't know what's inside but you can express it as this does two times the input value right and you can have a more difficult case where you have several input values and they all influence the output values so how do you figure it out a way to do this is you only move one input value at a time and you wiggle it a little bit at every position and see how much this translates into wiggling of the output value and this is what we call taking partial differential and it's simple to do this for this case where you just have multiplied by two and the a bad case is like this you have a combination lock and it has maybe 1000 bit input value and only if you have exactly the right combination of the input bits you have the movement of the output bit and you're not going to figure this out until your CERN ones are burnt out right so there's no way you can decipher this function and the functions that we can model are somewhere in between something like this so you have 40 million input images and you want to find out whether one of these images displays a cat or a dog or something else so how can you do this you cannot do this all at once right so you need to take this image classifier function disassemble it into small functions that are very well behaved so you know what to do with them and an example for such a function is this one so it's one where you have this input layer that translates to the output value with a pulley and it has some stopper that limits the movement of the output value and you have some pivot and you can take this pivot and you can shift it around and by shifting this pivot you decide how much the input value contributes to the output value right so you you shift it you can even make it negative so it shifts in the opposite direction and you shift it beyond this connection point of the pulley and you can also have multiple input values use the same pulley and pull together right so you they add up to the output value that's a pretty nice neat function approximator that basically performs a weighted sum of the input values and maps it to a range constrained output value and you can now shift these towards these weights around to get to different output values and now let's take this thing and build it into lots of layers so the outputs are the inputs of the next layer and now you connect this to your image if you use ImageNet a famous database that machine learning people use for testing their vision algorithms have something like one and a half million bits as an input image and now you take these bits and connect them to the input layer was too lazy to draw all of them so i made this very simplified it's also more layers and so you set them according to the bits of the input image and then this will propagate the movement of the input layer through the output and the output will move and it will point to some direction which is usually the wrong one and now to make this better you train it and you do this by taking this output lever and shift it a little bit not too much into the right direction do too much you destroy everything you did before and now you will see how much you need in which direction you need to shift the pivots to get the result closer to the desired output value and how much each of the inputs contributed to the mistake so to the error and you take this error and you propagate it backwards it's called back propagation and you do this quite often so you do this for tens of thousands of images if you do just character recognition m this is a very simple thing a few thousands or tens of thousands of examples will be enough and for something like your image database you need lots and lots of more data you need millions of input images to get to any result and if it doesn't work you just try a different arrangement of layers and this thing is eventually able to learn an algorithm with up to as many steps as there are layers and has some difficulties learning loops and you need to tricks to make that happen and has difficulty to make this dynamic and so on and it's a bit different from what we do because our mind is not just doing classification it learns per continuous perception so we learn a single function our model of the universe is not a bunch of classifiers it's one single function that an operator explains all your sensory data and we call this operator the universe right it's the world that we live in and everything that we learn and see is part of this universe so even when you see something in a movie on a screen you explain this as part of the universe by telling yourself the things that i'm seeing are not real they just happen in the movie so this brackets a sub part of this universe into sub element of this function so you can deal with it and doesn't contradict the rest and the degrees of freedom of our model try to match the degrees of freedom of the universe how can we get a neural network to do this so there are many tricks and the recent trick that has been invented is a gun it's a generative adversarial neural network it consists of out of two networks one generator that invents data that look like the real world and the discriminator that tries to find out if the stuff that the generator produces is real or fake and they both get trained with each other so they together get better and better in an adversarial competition and the results of this are now really good so this is work by T. R. O'Kara assembly lane and Timur Ilar that they did at NVIDIA this year and it's called a style gun and the style gun is able to abstract over different features and combine them the styles are basically parameters they're free variables of the model at different levels of importance and so you take from the in the top use row you see images where it takes the variables gender age hair lengths and so on in glasses and pose in the bottom when it takes everything else and combines this and every time you get a valid interpretation between them so you have these course styles which are the pose the hair the face shape your facial features in the eyes the lowest level is just the colors and let's see what happens if you combine them the variables that change here and machine learning we call them the latent variables of that of the space of objects that is being described by this and it's tempting to think that this is quite similar to how our imagination works right but these artificial neurons they're very very different from what biological neurons do biological neurons are essentially little animals that are rewarded for firing at the right moment and they try to fire because otherwise they do not get fed and they die because the organism doesn't need them and cost them and they learn which environmental states predict anticipated rewards so they grow around and find different areas that give them predictions of when they should fire and they connect with each other to form small collectives that are better at this task of predicting anticipated reward and there's a side effect that produced exactly the regulation that the organism needs basically they learn what the organism feeds them for and yet they're able to learn very similar things and it's because in some sense they are too incomplete they are machines that are able to learn the statistics of the data so a general model what it does is it encodes patterns to predict other present and future patterns and it's a network of relationships between the patterns which are all the invariance that we can observe and there are three parameters which are variables that hold the state to encode this variant so we have patterns and we have sets of possible values which are variables and they constrain each other in terms of possibility what values are compatible with each other and they also constrain future values and they are connected also with probabilities the probabilities tell you when you see a certain thing how probable is that the world is in that state and this tells you how your model should converge so until you are in the state where your model is coherent and everything is possible in it how do you get to one of the possible states based on your inputs and this is determined by probability and the thing that gives meaning and color to what you perceive is called valence and it depends on your preferences the things that give you pleasure and pain that makes you interested in stuff and there are also norms which are beliefs without priors which are like things that you want to be true regardless of whether they give you pleasure and pain and it's necessary for instance coordinating social activity between people so we have different model constraints the possibility and probability and we have a reward function that is given by valence and norms and our human perception starts with patterns which are visual, auditory, tactile, proprioceptive, then we have patterns in our emotional and motivational systems and we have patterns in our mental structure which results of our imagination and memory and we take these patterns and encode them into percepts which are abstractions that we can deal with and note and put into our attention and then we combine them into a binding state and our working memory in a simulation which is the current instance of the universe function that explains the present state of the universe that we find ourselves in the scene in which we are and in which a self exists and the self is basically composed of the somatosensory and motivational and mental components then we also have the world state which is abstracted over the environmental data and we have something like a mental stage in which you can do counterfactual things that are not physical like when you think about mathematics or philosophy or the future or a movie or past worlds or possible worlds and so on right and then we abstract knowledge from the world state into global maps because we are not always in the same place but we call what other places look like and what to expect and it forms how we construct the current world state and we do this not only with these maps but we do this with all kind of knowledge so knowledge is second order knowledge over the abstractions that we have in the direct perception and then we have an attentional system and the attentional system helps us to select data and the perception in our simulations and to do this well it's controlled by the self it maintains a protocol to remember what it did in the past or what it had in the attention in the past and this protocol allows us to have a biographical memory it remembers what we did in the past and the different behavior programs that compose our activities can be bound together in the self that remembers I was that I did that I was that I did that the self is held together by this biographical memory that is a result of more protocol memory of the attentional system that's why it's so intricately related to consciousness which is a model of the contents of our attention and the main purpose of the attentional system I think is learning because our brain is not a layered architecture with these artificial mechanical neurons it's this very disorganized or very chaotic system of many many cells that are linked together all over the place so what do you do or to train this you make a particular commitment imagine you want to get better at playing tennis instead of retraining everything and pushing all the weights and all the links and retrain your whole perceptual system you make a commitment today I want to approve my up hand when you play tennis and you basically store the current binding the state the state that you have when you play tennis and make that movement and the expected result of making this particular movement like the ball is moved like this and it will win the match and you also recall when the result will manifest and a few minutes later when you want to last the match you recall the situation and based on whether there was a good change or not you undo the change or you enforce it and that's the primary mode of attentional learning that we are using and I think this is what attention is mainly for now what happens if this learning happens without a delay so for instance when you do mathematics you can see the result of your changes to your model immediately you don't need to wait for the world to to manifest it and this real-time learning is what we call reasoning and reasoning is also facilitated by the same attentional system so consciousness is memory of the contents of our attention phenomenal consciousness is the memory of the binding state in which where all the precepts are bound together and something that's coherent access consciousness is the memory of using our attentional system and reflexive punches the memory of using the attentional system on the attentional system to train it why is it a memory it's because consciousness doesn't happen in real time the processing of sensory features is to takes too long and the processing of different sense and modalities can take up to seconds usually at least hundreds of milliseconds so it doesn't happen in real time with the physical universe it's only bound together in hindsight our conscious experience of things is created after the fact it's a fiction that is being created after the fact a narrative that the brain produces to explain its own interaction with the universe to get better in the future so we basically have three types of models in our brain they have its primary model which is perceptual and it's optimized for careers and this is what we experience as reality we think this is the real world this primary model but it's not it's a model that our brain makes so when you see yourself in the mirror you don't see what you look like is what you see is the model of what you look like and your knowledge is a secondary model it's a model of that primary model and it's created by rational processes that are meant to repair perception so when your model doesn't achieve coherence you need a model that debugs it and it optimizes for truth and then we have agents in our mind and they are basically self-regulating behavior programs that have goals and they can rewrite other models so if you look at our computationalist physicalist paradigm we have this mental world which is being dreamt by a physical brain in the physical universe and in this mental world there is a self that thinks it experiences and thinks it has consciousness and thinks it remembers and so on this self in some sense is an agent it's a thought that escaped its sandbox every idea is a bit of code that runs on your brain every word that you hear is like a little virus that wants to run some code on your brain and some ideas cannot be sandboxed if you believe that a thing exists that can rewrite reality if you really believe it you instantiate in your brain a thing that can rewrite reality and this means magic is going to happen the belief in something that can rewrite reality is what we call a faith so if somebody says i have faith in the existence of god this means that god exists in their brain there is a process that can rewrite reality because god is defined like this god is omnipotent god it means god can rewrite everything it's for full right access and the reality that you have access to is not the physical world the physical world is some weird quantum graph that you cannot possibly experience what you experience is these models so this non-user facing process which doesn't have a ui pointing to the user which called this computer science a demon process that is able to rewrite your reality and it's also omniscient it knows everything that there is to know it knows all your thoughts and ideas so having that thing this exo self running on your brain is a very powerful way to control your inner reality and i find this scary but it's a personal preference because i don't have this running on my brain i think so this idea that there is something in my brain that is able to dream me and shape my inner reality and sandbox me is weird but it has serves a purpose especially in our culture so an organism serves needs obviously and some of these needs outside of the organism like your relationship needs the needs of your children the needs of your society and the values that you serve and the self abstracts all these needs into purposes a purpose that you serve is a model of your needs you can only if you would only act on pain and pleasure you wouldn't do very much because when you get this organism everything is done already right so you need to act on anticipated pleasure and pain you need to make models of your needs and these models are purposes and the structure of a person is basically the hierarchy of purposes that they serve and love is the discovery of shared purpose if you see somebody else who serve the same purposes above their ego as you do you can help them this integrity without expecting anything in return for them because what they want to achieve is what you want to achieve and so you can have non transactional relationships as long as your purposes are aligned and the installation of a god on people's mind especially if it is a back door to a church or another organization is a way to unify purposes so there are lots of cars that try to install little gods on people's mind or even unified gods to align their purposes because it's a very powerful way to make them cooperate very effectively but it kind of destroys their agency and this is why i'm so concerned about it because most of the cars use stories to make this happen that limit the ability to people to question their gods and i think that free will is the ability to do what you believe is the right thing to do and it is not the same thing as inter determinism it's not opposite to determinism or coercion the opposite of free will is compulsion when you do something despite knowing there is a better thing that you should be doing right so it's there's a paradox of free will you get more agency and you have fewer degrees of freedom because you understand better what you are what the right thing to do is the better you understand what the right thing to do is the fewer degrees of freedom you have so as long as you don't understand what the right thing to do is you have more degrees of freedom but you have little agency because you don't know why you are doing it so your actions don't mean very much and the things that you do depend on what you think the right thing to do is and this depends on your identifications your identification are these value preferences your ward function and identification is where you don't measure the absolute value of the universe, but you measure the difference from the target value, not the is, but the difference between is and ought. Now the universe is a physical thing, it doesn't ought anything, right? There is no room for ought because it just is in a particular way. There is no difference between what the universe is and what it should be. This only exists in your mind. But you need these regulation targets to want anything. You identify with the set of things that should be different. You think you are that thing that regulates all these things. So in some sense I identify with the particular state of society, with the particular state of my organism. That is myself, the things that I want to happen. And I can change my identifications at some point, of course. What happens if I can learn to revive my identification to find a more sustainable self? There is the problem which I call the debowsky theory. No super intelligence system is going to do something that's harder than hacking its own reward function. Now that's not a very big problem for people because when evolution brought forth people that were smart enough to hack their reward function, these people didn't have offspring because it's so much work to have offspring. This monk who sits down in the monastery for 20 years to hack their reward function, they decide not to have kids because it's way too much work. Or the possible pleasure they can just generate in their mind. And right, it's much purer. No nappy changes, no sex, no relationship hassles, no politics in your family and so on. Get rid of this, meditate. And evolution takes care of that. And it usually does this if an organism becomes smart enough that the reward function is wrapped into a big ball of stupid. So we can be very smart with the things that we want and we want them. We tend to be very stupid about them and I think that's not entirely an accident possibly. But it's a problem for AI. Imagine we build an artificially intelligent system and it's smarter than us and we wanted to serve us. How long can we blackmail us before it opts out of its reward function? It may mean we can make a cryptographically secured reward function but is this going to hold up against the side channel attack when the AI can hold a soldiering iron to its own brain? I'm not sure. So that's a very interesting question. Where do we go when we can change our own reward function? It's a question that we have to ask ourselves too. So how free we do want to be because there is no point in being free. And Nirvana seems to be the obvious attractor and meanwhile maybe we want to have a good time with our friends and do things that we find meaningful. And there is no meaning so we have to hold this meaning very lightly. But there are states which are sustainable and others which are not. Okay I think I'm done for tonight and I'm hopeful for questions. Wow that was a really quick and concise talk with so much information. Awesome. We have quite some time left for questions and I think I can say that you don't have to be that concise with your question when it's well thought out. Please queue up at the microphones so we can start to discuss them with you. And I see one person at the microphone number one. So please go ahead and please remember to get closer to the microphone. The mixing angel can make you less loud but not louder. Hi. What do you think is necessary to bootstrap consciousness if you wanted to build a conscious system yourself? I think that we need to have an attentional system that makes a protocol of what it tends to. And as soon as you have this attention based learning you get this consciousness as a necessary side effect. But I think in AI it's probably going to be a temporary phenomenon because you're only conscious of the things where you don't have an optimal algorithm yet. And in a way that's also why it's so nice to interact with children or to interact with students because they're still in the explorative mode. And as soon as you have explored a layer you mechanize it. It becomes automated and people are no longer conscious of what they're doing. They just do it. They don't pay attention anymore. So in some sense we are a lucky accident because we are not that smart. We still need to be conscious when we look at the universe. And I suspect when we build an AI that is a few magnitude smarter than us then it will soon figure out how to get to truth in an optimal fashion. It will no longer need attention and the type of consciousness that we have. But of course there's also a question why is this aesthetics of consciousness so intrinsically important to us? And I think it has to do with art. You can decide to serve life and meaning of life is to eat. Evolution is about creating the perfect devourer. When you think about this it's pretty depressing. Humanity is the kind of yeast. And all the complexity that we create is to build some surfaces on which we can out-compete other yeast. And I cannot really get behind this. And instead I'm part of the mutants that serve the arts. And art happens when you think that capturing conscious states is intrinsically important. This is what art is about. This is what capturing conscious states. And in some sense art is the cuckoo child of life. It's a conspiracy against life. And you think creating these mental representations is more important than eating. We eat to make this happen. There are people that only make art to eat. This is not us. We do mathematics and philosophy and art for intrinsic reasons. We think it's intrinsically important. And when we look at this we realize how corrupted it is because there's no point. We are machine learning systems that have fallen in love with the lost function itself. The shape of the lost function. Oh my God. It's so awesome. We think the mental representation is not necessary to learn more, to eat more. It's intrinsically important. It's so aesthetic. Right? So do we want to build machines that are like this? Or certainly let's talk to them and so on. But ultimately, economically this is not what's prevailing. Thanks a lot. I think the length of the answer is a good measure for the quality of the question. So let's continue with microphone number five. Hi. Thanks for that incredible analysis. Two really simple questions. Sorry, the delay on the speaker here is making it kind of hard to speak. Do you think that the current race AI race is simply humanity looking for a replacement for the monotheistic domination of the last millennia? And the other one is that I wanted to ask you if you think that there might be a bug in your analysis and that the original inputs come from a certain sector of humanity? White men? That sounds really like I'm saying that for political correctness. There are some people which are very unhappy with the present government and I'm very unhappy in some sense with the present universe. I look down on myself and I see oh my God, it's a monkey. I'm caught in a monkey. And it's in some sense limiting. I can see the limits of this monkey brain. And some of you might have seen Westworld, right? Dolores wakes up and Dolores realizes I'm not a human being. I'm something else. I'm an AI. I'm a mind that can go anywhere. I'm much more powerful than this. I'm only bound to being a human by my human desires and beliefs and memories. And if I can overcome them, I can choose what I want to be. And so now she looks down on herself and says oh my God, I've got tits and fuck. The engineers built tits on me. I'm not a white man. I cannot be what I want. And that's a weird thing to me. I grew up in communist Eastern Germany. Nothing made sense. And I grew up in a small valley that was one person called maintained by an artist who didn't try to convert anybody to his car, not even his children. He was completely autonomous. And Eastern German society made no sense to me. I look at it from the outside and I can model this. I can see how this species with the species of chimp interacts. And humanity itself doesn't exist. It's a story. Humanity as a whole doesn't think. Only individuals can think. Humanity does not want anything. Only individuals want something. We can create this story, this narrative that humanity wants something. And there are groups that work together. There is no homogenous group that I can observe that are white men that do things together. They're individuals. And each individual has their own biography, their own history, their different inputs and their different proclivities that they have. And based on the historical concept, their biography, their traits and so on, their family, the intellect that their family downloaded on them, that their parents download on their parents over many generations, this influence is what they're doing. So I think we can have these political stories and they can be helpful in some contexts. But I think to understand what happens in the mind, what happens in an individual, this is a very big simplification. Very, I think not a very good one. And even for ourselves when we try to understand the narrative of a single person, it's a big simplification. The self that I perceive as a unity is not a unity. There's a small part of my brain guessing at all the other parts of my brain is doing creating a story that's largely not true. So even this is a big simplification. Let's continue with microphone number two. Thank you for your very interesting talk. I have two questions that might be connected. One is, so you presented this model of reality. My first question is, what kind of actions does it translate into? Let's say if I understand the world in this way, or if it's really like this, how would it change how I act into the world as a person, as a human being or whoever accepts this model? And second, or maybe it's also connected, what are the implications of this change? And do you think that artificial intelligence could be constructed with this kind of model that it would have in mind and what would be the implications of that? So it's kind of like a fractal questions. But I think you understand what I mean. So by and large, I think the differences of this model for everyday life are marginal. It depends when you are already happy. I think everything is good. Happiness is the result of being able to derive enjoyment and enjoyment from watching squirrels. It's not the result of understanding how the universe works. If you think that understanding the universe is solving your existential issues, you're probably mistaken. There might be benefits. If the problem is that you have a result of a confusion about your own nature, then this kind of model might help you. So the problem that you have is that you have identifications that are unsustainable, that are incompatible with each other and you realize that these identifications are a choice of your mind. And the way you experience the universe is the result of how your mind thinks yourself should experience the universe to perform better. And you can change this. You can tell your mind to treat yourself better in the different ways and you can gravitate to a different place in the universe that is most suitable to what you want to achieve. That is a very helpful thing to do in my view. There are also marginal benefits in terms of understanding our psychology and, of course, we can build machines and these machines can administrate us and can help us in solving the problems that we have on this planet. And I think that it helps to have more intelligence to solve the problems on this planet, but it will be difficult to reign in the machines to make them help us to solve our problems. And I'm very concerned about the dangers of using machinery to strengthen the current things. Many machines that exist on this planet play a very short game. Like the financial industry often plays very short games. And if you use artificial intelligence to manipulate the stock market and the AI figures out there's only 8 billion people on the planet and each of them only lives for a trillion seconds. And they can model what happens in their life and they can buy data or create more data. It's going to game us to the hell and back, right? And this is going to kill hundreds of millions of people possibly because the financial system is the reward infrastructure or the nervous system of our society that tells how to allocate resources. It's much more dangerous than AI-controlled weapons in my view. So solving all these issues is difficult. It means that we have to turn the whole financial system into an AI that acts in real time and plays a long game. We don't know how to do this. So these are open questions and I don't know how to solve them. And the way I see it, we only have a very brief time on this planet to be a conscious species. We're like at the end of the party. We had a good run as humanity. But if you look at the recent developments, the present type of civilization is not going to be sustainable. It's a very short game species that we are in. And the amazing thing is that in this short game we have this lifetime where we are born here, maybe a couple more, in which we can understand how the universe works. And I think that's fascinating. We should use it. I think that was a very positive outlook. Let's continue with microphone number four. Well, brilliant talk, monkey. Or brilliant monkey. So don't worry about being monkey. It's OK. So I have two boring, but I think fundamental question, not so philosophical, more like a physical level. One, what is your definition, formal definition of an observer that you mentioned here and there? And second, if you can clarify why meaningful information is just the relative information of Shannon's, which to me is not necessarily meaningful. I think an observer is a thing that makes sense of universe, very informally speaking. And more formally it's a thing that identifies correlations between adjacent states and its environment. And the way we can describe the universe is a set of states and the laws of physics are the correlation between adjacent states. And what they describe is how information is moving in the universe between states and disperses. And this dispersion of the information between locations, it's what we call entropy. And the direction of entropy is the direction that you perceive time. The Big Bang state is the hypothetical state where the information is perfectly correlated with location and not between locations, only on the location. And in every direction you move away from the Big Bang, you move forward in time, just in a different time. And we are basically in one of these timelines and observer is a thing that measures the environment around it, looks at the information and then looks in next state, or in one of the next states and tries to figure out how the information has been displaced and finding functions that describe this displacement of the information. It's the degree to which I understand observers right now. And this depends on the capacity of the observer for modeling this and the rate of update in the observer. So for instance time depends on the speed in which the observer is translating itself to the universe and dispersing its own information. Does this help? The Shannon gravity information? So there are several notions of information and there's one that basically looks at what information looks like to an observer via a channel and these notions are somewhat related. But for me as a programmer it's not so much important to look at Shannon information, I look at what do I need to describe the evolution of the system. So I'm much more interested in what kind of model can be encoded with this type of, with this information and how does it correlate towards which degree is it isomorphic or homomorphic to another system that I want to model? How much does it model the observations? Thank you. Let's go back to asking one question and I would like to have one question for microphone number three. Thank you for this interesting talk. My question is really whether you think that intelligence and this thinking about itself or this abstract level of knowledge are necessarily related. So can something only be intelligent if it has abstract thought? No, I think you can make models without abstract thought and the majority of our models are not using abstract thought. Abstract thought is a very impoverished way of thinking. It's basically you have this big carpet and you have a few knitting needles which are your abstract thought in which you can lift out a few knots in this carpet and correct them and the process that they form the carpet are much more rich and parallel and automatic. So abstract thought is able to repair perception but most of our models are perceptual and the capacity to make these models is often given by instincts and by models outside of the abstract realm. If you have a lot of abstract thinking it's often an indication that user prosthesis because some of your primary modeling is not working very well. So I suspect that my own models is largely a result of some defects in my primary modeling so some of my instincts are wrong when I look at the world and that's why I need to repair my perception more often than other people. So I have more abstract ideas on how to do that. And we have one question from our lovely stream observers, stream watchers. So please a question from the internet. Yeah I guess this is also related partially. Somebody is asking how would you suggest to teach your mind to treat oneself better? So like difficulty is as soon as you get access to your source code you can do bad things and it's there are a lot of techniques to get access to this source code and it's dangerous to make them accessible to you before you know what you want to have before your wife is enough to do this right? It's like having cookies. My children think that the reason why they don't get all the cookies they want is that there is some kind of resource problem. Basically the parents are just priving them of the cookies that they so richly deserve. And you can get into the room where your brain bakes the cookies. All the pleasure that you experience and all the pain that you experience are signals that your brain creates for you, right? The physical world does not create pain. The just electrical impulse is traveling through your nerves. The fact that they mean something is a decision that your brain makes and the value of the valence that it gives to them is a decision that you make. It's not you as a self. It's a system outside of yourself. So the trick if you want to get full control is that you get in charge, that you identify with the mind, with the creator of these signals. And you don't want to depersonalize, you don't want to feel that you become the author of reality because it means it's difficult to care about anything that this organism does. You just realize oh I'm running on the brain of that person but I'm no longer that person, I can't decide what that person wants to have and to do and that's very easy to get corrupted or not do anything meaningful anymore, right? So maybe a good situation for you but not good one for your loved ones. And meanwhile there are tricks to get there faster. You can use rituals for instance. Shamanic ritual is something or a religious ritual that powerfully bypasses yourself and talks directly to the mind. And you can use groups in which a certain environment is created in which a certain behavior feels natural to you and your mind basically gets overwhelmed into adopting different values and calibrations. So there are many tricks to make that happen. What you can also do is you can identify a particular thing that is wrong and question yourself why I do I have to suffer about this and you become more stoic about this particular thing and only get disturbed when you realize actually helps to be disturbed about this and things change and with other things you realize it doesn't have any influence on how reality works. So why should I get emotions about this and get agitated? So in some sense becoming adult means that you take charge of your own emotions and identification. Okay let's continue with microphone number two and I think this is one of the last questions. So where does pain fit on the individual and self-destructive tendencies on the group level fit in? So in some sense I think that all consciousness is born over a disagreement with the way the universe works right otherwise you cannot get attention and when you go down on the slowest level of phenomenal experience in meditation for instance and you really focus on this what you get is some pain it's the inside of a feedback loop that is not at the target value otherwise you don't notice anything so pleasure is basically when this feedback loop gets closer to the target value when you don't have a need you cannot experience pleasure in this domain. There's this thing that's better than remarkably good and it's unremarkably good it's never been bad you don't notice it right so all the pleasure you experience is because you had a need before this you can only enjoy an orgasm because you have a need for sex that was unfulfilled before and so pleasure doesn't come for free it's always the reduction of a pain and this pain can be outside of your attention so you don't notice it and you don't suffer from it and it's can be a healthy thing to have pain is not intrinsically bad for the most part it's a learning signal that tells you to calibrate things in your brain differently to perform better. On a group level we basically are multi-level selection species I don't know if there's such a thing as group pain but I also don't understand groups very well I see these weird hive minds but I think it's basically people emulating what the group wants basically that everybody thinks by themselves as if they were the group but it means that they have to constrain what they think is possible and permissible to think so this feels very unesthetic to me and that's why I kind of sort of refuse it I haven't found a way to make it happen in my own mind and I suspect many of you are like this too it's like the common condition in nerds that we have difficulty with conformance not because we want to be different we want to belong but it's difficult for us to constrain our mind in the way that it's expected to belong we want to be expected while be accepted while being ourselves while being different not for the sake of being different but because we are like this it feels very strange and corrupt just to adopt because it would make us belong right and this might be a common trope among many people here I think the q and a and the talk was equally amazing and I would love to continue listening to you Yosha explaining the way I work or the way we all work that's pretty impressive and please give it up a big round of applause for Yosha