 What happens if you mix Shannon's information theory and biological systems? A dish better served hot. Please welcome our computational systems biology chef, who will guide you through investigating the information flow in living systems. Please welcome, with a very warm round of applause, Jürgen Parle. Thanks a lot and thanks for having me. It's great that so many of you are interested in that topic, which is not about technical systems, but actually biological cells. I'm leading a group in Heidelberg at the university there, and we are mostly interested in how information is processed, sensed, stored, communicated between biological cells. And we are interested in that because it's not obvious that they actually managed to do that in a reliable fashion. They don't have transistors. They only can use their molecules, mostly proteins, big molecules that are little engines or little motors in the cell that allow them to fulfill their biological functions. If information processing fails in cells, you get diseases like epilepsy, cancer and, of course, others. Now cellular signalling pathways have been studied in some detail, mostly single pathways, more and more also networks of pathways, but surprisingly little conceptual work has been done on them. So we know the molecules that are involved, we know how they react, how they combine to build these pathways, but we don't know how actually information is transferred or communicated across these pathways. And we intend to fill that gap in our group, and, of course, first we have to model these networks, we have to model these biochemical pathways, and this is how we proceed. So you have a cell, you can't see that here, but on the upper left corner you have that scheme of a cell with all the different components. You have volumes in this cell where chemical reactions happen. So chemical reactions take biochemical species, ions, proteins, what have you, and they convert them into other chemical species, and these reactions happen in the different compartments. Now it's very important to assign speeds or velocities to these reactions because these speeds determine how fast the reactions happen and how the dynamic behavior then results. And once you have done that, you can translate all of that into a mathematical model like the one shown here on the right. This is an ordinary differential equation system, I don't want to go into detail, I only have two or three formulas that might be interesting for you. So this is just any mathematical model you have of these systems and then you can start analyzing them. You can ask questions like how does the system change over time, that's simulation, which parts influence the behavior most, what are the stable states, do you have oscillations, do you have a steady state and so on. Now you don't have to do that by hand because we are actually also developing software, that's just another thing. I guess you know that all models are wrong, we try to build useful ones. So I said you don't have to do this by hand because we are also into method development and we are building scientific software. One of the software we build is called Copasi Complex Pathway Simulator, it's free and open source, you can all go to that website, download it, play around with it if you want. Because we also use more demanding computations which we send to compute clusters, we also developed a scripting interface for Copasi which is called Cork, the Copasi R connector and this allows you to use the Copasi backend with all the different tools that are in Copasi from your programming environment and then you can build workflows and send them to compute cluster. If you think it's easy to use, if you play around with it, you get stuck, then just let me know. So this is software you can use, you can play around with, where do we get the models? Well, there is a model database that is called biomodels.net, also free to use, you can go there, download models. At the moment they have almost 800 different manually curated models and almost 10 times of that that are built automatically. You can just download them in the so-called SBML format which is the systems biology markup language that then import it into Copasi or other software and play around with them. Okay, so coming back to biology, one of our favorite systems is calcium signaling. And calcium signaling works roughly like this, you have these little, I mean the oval thing is a cell, then you have these red cones that are hormones and other substances that you have in your bloodstream or somewhere outside the cell. They bind to these black things which are receptors on the cell membrane and then the cascade of processes happen that in the end lead to an in-stream of calcium ions, these blue balls from the ER, which is not emergency room but endoplasmatic radiculum, which is one of the compartments in the cell into the main compartment, the cytosol of the cell and also calcium streams into the cell from outside the cell. And this leads to a sharp increase of the concentration of calcium until it's pumped out again. There are pumps that take calcium ions and remove them from the cytosol and pump them out of the cell and back into the ER. This is very important because calcium is a very versatile second messenger, that's what they call it. It regulates a number of very important cellular processes. If you move, your muscle contraction is regulated by calcium, learning, secretion of neurotransmitters in your brain, fertilization, a lot of different things are regulated by calcium and if you simulate the dynamic processes you get behavior like that. Here you can see it oscillates, it shows these regular spikes so this is the calcium concentration over time. Now if you actually measure this in real cells and this is data measured by collaboration partners of mine in England, you see it's not that smooth. You get these differences in amplitude of the peaks, you get secondary spikes, you get fluctuations around the basal level and this is because you have random fluctuations in your system. Intrinsic random fluctuations that are just due to random fluctuations in the timings of single reactive events, single reactions, biochemical reactions that happen. And in order to capture this behavior because this behavior is important, that can hamper reliable information transfer. We have to resort to special simulation algorithms, for example the so-called Gillespie algorithm. And if you do that and apply it to the calcium system you can actually capture the secondary peaks and all the different other fluctuations you have in there. Now this is just a Monte Carlo simulation. I say just, it's really time consuming and demanding because you have to calculate each and every single reactive event in the cell and that takes a lot of time, that's why we do that on a compute cluster. I told you already that calcium is a very versatile second messenger so you have very many different triggers of calcium response in the cell, things that lead to a certain calcium dynamics and on the other hand downstream, calcium regulates many different things. So you have these hour class or bow tie structure and that's why people have speculated about the calcium code. How can it be that the proteins should go back that actually do all these cellular functions, these green cylinders that bind calcium and are then activated or inhibited by it? How can it be that they know which stimulus or which hormone is outside of the cell? They don't see them because there is a cell membrane around the cell, around the cytosol. So people have speculated is there an information encoded in the specific calcium waveform? Is there a calcium code? And how can it be that the proteins actually decode that code? It's fairly established that calcium has or shows amplitude modulation so the higher the amplitude of calcium the more active get some proteins. It also shows frequency modulation meaning the higher the frequency of the calcium oscillations the more active get some proteins but maybe there are other information carrying features in the waveform like duration, waveform, timing and so on. Now a doctoral student in my group has looked into frequency modulation and he actually showed that there are proteins in that case N-FAT which is the nuclear factor of activated T cells which are important in your immune system. They only react to calcium oscillations of a certain frequency so they get activated in a very narrow frequency band and that's why we call it band pass activation. Okay, so I guess you all know signaling speeds of technical systems they're fairly fast by now. One of our results because we quantify actually information transfer is that calcium signaling operates at roughly 0.4 bit per second. If you compare that to technical systems that seems very low but maybe that's enough for all the functions that the cell has to fulfill. So how did we arrive at this result? Well, we used information theory, classical information theory pioneered by people like Claude Shannon in the 40s also by Hartley, Tucky and a few other people. So they looked at technical systems and they have this prototypical communication system where there is an information source on the left side then this information is somehow encoded it's transmitted over a noisy channel where the message is scrambled then it's received by a receiver decoded and then hopefully you get the same message at the destination that was chosen at the information source. In our case we look at calcium as an information source and we study how much information is actually transferred to downstream proteins. How do you do that? Well, information theory 101, information theory primer. In statistical information theory of the Shannon type you look at random variables. You look at events that have a certain probability of happening. So let's say you have an event that has a probability of happening and then Shannon said that the information content of this event should be the negative logarithm which is shown here the curve on the right hand side should be the negative logarithm of the probability meaning that if an event happens all the time and I will show you an example later there is no information content. The information content is zero. There is no surprise if that event happens because it happens all the time. It's like a sunny day somewhere in the desert. However, if you go to lower probabilities then the surprise becomes bigger and the information content rises. Now in a system you have several events that are possible and if you take the average uncertainty of all possible events you get something that Shannon called entropy. This is still not information because information is a difference in entropy. So you have to calculate the entropy for system and then you calculate the entropy that is remaining after an observation say and this difference is the information gained by the observation. Now coming to a simple example let's say we have a very simple weather system where you can only have rainy and sunny days and let's say they are equally likely. So you have a probability of one-half for each of them the average of the negative logarithm is one so you gain when you observe the weather in this system you gain one bit per day. You can also think of bits as the information you need or a cell needs to answer or decide on one yes or no question. Now if it's always sunny and no rain then you get zero information content or uncertainty the average is zero you don't get any information if you observe the weather in the desert say 80-20 you get a certain bit number per day in that case 0.64 bit per day and you can do that for Leipzig in that case Leipzig has 99 rainy days per year according to the Deutsche Wetterdienst and this gives you an information of 0.84 bit per day you can do it in general way so let's say you have one event with a probability of P and another event with a probability of one minus P and then you get this curve which shows you that the information content is actually maximal if you have maximal uncertainty if you have equally likely events if you have more possible events in that case four different ones sunny, cloudy, rainy and thunderstorm you get two bit and this is because of the logarithm so if you have double the amount of events and they're equally likely you get one bit more hope I didn't lose anyone okay now we are always looking at processes dynamic things things that change over time and if we look at processes we have to look at transition probabilities so we have to change probabilities to transition probabilities so we can summarize them in a matrix so let's say if we have a sunny day today it's more likely that it's also sunny tomorrow and less likely that it's raining maybe only 25% and if it's rainy today you can't tell it's equally likely and these processes are also called Markov process Markov was a Russian mathematician and you have them everywhere these Markovian processes are used in your cell phones in your hard drives they're used for error correction the page rank algorithm of Google is one big Markov process so you're using them all the time nothing technological would work nowadays without them now because we have knowledge about today's weather the uncertainty about tomorrow's weather decreases so now we have an entropy rate instead of an entropy and the difference is again the information you gain by today's weather so you can do the maths in our example the entropy would be 0.92 bit per day and the entropy rate given that you know today's weather yes it's 0.87 bit per day now to complicate things a bit more maybe we also look at a second process in that case air pressure and you can measure air pressure with these little devices the parameters and maybe if it's sunny today and the air pressure is high in 90% you get a sunny day tomorrow and only in 10% of the cases you get a rainy day and so on you can go through the table in our case I looked it up yesterday we had high air pressure and it was raining so in our little model system it would mean that it's sunny today now I told you information is a decrease in uncertainty now how much information do we get by the barometer by knowing the air pressure and this is the difference in uncertainty without barometer and with the barometer in our case we have to assume that the probability of high and low air pressure is the same and we get 0.39 bit per day that we gain by looking at the air pressure now what does that have to do with biological systems well we have two processes we have a calcium process that shows some dynamics and we have the process of an activated protein that does something in the cell so we can look at both of these and then calculate how much information is actually transferred from the calcium to the protein how much uncertainty do we lose about the protein dynamics if we know the calcium dynamics and this is mathematically exactly what we are doing and this is called transfer entropy it's an information theoretical measure developed by Thomas Schreiber in 2000 there are some practical complications that we are working on and this is what we are using actually for the calculation so in our case we have data from experiments or we use models of calcium oscillations and then we couple a model of protein to these calcium dynamics this gives us time causes both of calcium and protein stochastic time causes including the random fluctuations and then we use the information theoretic machinery to study them and some of our results I want to show you for example if you increase the system size if you increase the particle numbers if you make the cell bigger then the information that you can transfer is higher meaning if the cell invests more energy and produces more proteins it can actually achieve a more reliable information transfer which comes of course with costs for the cell also it seems that if you use more complicated dynamics meaning not only spiking but maybe bursting behavior where you have secondary spikes then you can transmit more information because the input signal carries more information or can carry more information in its different features another result is that proteins a very interesting result I think is that proteins can actually be tuned to certain characteristics of the calcium input meaning with all the different calcium sensitive proteins in the cell they are tuned to a specific signal so they only get activated or these pathways only allow information transmission if a certain signal is observed in the cell by these proteins so in a way the 3D structure of the protein defines how it behaves dynamically how quickly it binds and so on how many binding sites it has and then this dynamic behavior determines to what input signals that protein is actually sensitive and on the right hand side you can see some calculations we did the peaks actually show where this specific protein which is a cell module in like protein you don't have to memorize that it's a very important calcium sensitive protein where these differently parameterized models actually get activated and allow information transfer and this allows differential regulation because you have all the different proteins you have only one calcium concentration and only the proteins that are sensitive to a specific input get activated or do their things in the cell now if you look at more complicated proteins so Calmodulin the one I just showed you was only activated by calcium more complicated proteins like protein kinase C for example they are both activated and inhibited so they show biophasic behavior where in an intermediate range of calcium concentration they get activated with very high or very low concentrations they are inactivated and you can actually see that these more complicated proteins allow a higher information transfer and again producing these more complicated proteins might be more costly for the cell but it can be valuable because they allow more information to be transferred and this you can see in this plot where we actually scanned over the activation and the inhibition constant of these model proteins and you can see that you have these sweet spots where you get a very high information transfer so color coded is transfer entropy now coming to a different system just quickly we also looked at other systems of course calcium signaling is just one of our favorite one we also looked at bacteria and this is E. coli a very famous model system for biologists these are cells that can actually move around because they have little propellers at the end and so they want to they want to find sources of nutrients for example to get food so they swim into a direction and then they decide whether to swim whether to keep swimming in that direction or whether to tumble, reorient randomly and swim in some other direction and the problem for them is they are too small they can't detect a concentration gradient of nutrients of food between their front and the back of the cell so they have to swim in one direction and then they have to remember some nutrient concentration of some time back and then they have to compare is the nutrient concentration actually increasing then I should continue swimming if it's decreasing I should reorient and swim in some other direction and this allows them to on average swim towards sources of food now in order to compare over time the nutrient concentrations they have to memorize they have to know how much nutrients were there some time ago and for that they have a little memory and the memory is actually in the you can see on the left hand side the receptor that actually senses these nutrients they can be modified these receptors we call that methylated so they get a methylation group attached and they have different states of methylation five different ones in that model we are looking at and this builds a memory and we looked into that we quantified that with information theory this is a measure this is called mutual information it's not transfer entropy it's another measure of in that case a statical information you can see this is the amount of information that is actually stored about the nutrient concentration that is outside of the cell this is in nuts it's not in bits it's just a different you can translate them it's just a different unit for information and you can also see how the different methylation states so these are the colored curves how they go through how they are active with different nutrient concentrations and this is ongoing research so maybe next time hopefully next time I can show you much more just to finish this we also look at time scales because the time scales have to be right the system adapts so if you keep that cell in a certain nutrient concentration it adapts to that nutrient concentration and goes back to its normal operating level now if you increase the nutrient concentration again it shows some swimming behavior so it adapts but it also has to decide it also has to compare the different nutrients at different positions and that's how they have to manage the different time scales of decision making and memory or adaptation and we are looking into that as well coming to the conclusions I hope I could convince you that information theory can be applied to biology that it's a very interesting topic it's a fascinating area and we are just at the beginning to do that I also showed you that it's such that in signaling pathways the components can be tuned to their input which allows differential regulation so even though you don't have wires you can still specifically activate different proteins with one signal or multiplex if you want we are of course in the process of studying what features of the input signal actually information carrying so we are looking into things like waveform and timing and we want to look into how these things change in the deceased case so if you have things like cancer where certain signaling pathways are perturbed or fail we want to exactly find out what does that do to the information processing capabilities of the cell we also found out that estimating these information theoretical quantities it can be a very tricky business another project we are doing at the moment is actually only on how to interpret these in a reliable manner how to estimate this from sparse and noisy data so that's also ongoing work I would like to thank some of my collaborators of course my own group also some others in particular the kubasi team that is spread all over the world and with that I would like to thank you for your attention and I would be happy to answer any question you might have thank you if you have questions there are two microphones, microphone number one microphone number two and please speak loudly into the microphone and I think the first one is microphone number two your question please has there been any work done on computational modeling of G-protein coupled receptors and the second messenger cascade is there can you repeat that sorry has there been any work done on computational modeling of G-protein coupled receptors G-protein, yeah oh yes I mean we are doing that because calcium is actually, I mean the calcium signal is actually triggered by a cascade that includes the G-protein or most of these receptors are actually coupled or G-protein coupled receptors so that's what we are doing thank you microphone number two again first of all thanks for the talk and I wanted to ask you talk a little bit about how different proteins get activated by different signals and could you go a bit into detail about what kind of signal qualities the proteins can detect so are they like are they triggered by specific frequencies or specific decays like which characteristics of the signals can be picked up by the different proteins well that's actually what we study I mean we have another package that is linked here is the last one the oscillator generator this is a package in R that allows you to create artificial inputs where you have complete control of all the parameters like amplitude duration of the peak duration of the secondary peaks frequencies of the primary peaks of the secondary peaks refraction period and so on you have complete control and we do at the moment we also running scans and want to find out what proteins are actually sensitive to what parameters in the input signal what we know from Calcium is that for example Calcium Calmodulin kinase 2 also a very important protein in the nervous system that shows frequency modulation and this has also been shown experimentally where they put that protein on a surface they immobilized it on a surface and then they superfused it with Calcium concentrations or with solutions of different Calcium concentration in a pulsed manner and they measured the activity of that protein and they showed that with increasing frequency the activation gets bigger at the same time it also shows amplitude modulation it's also sensitive to the amplitude meaning the absolute height of the concentration of Calcium thank you and again, number 2 please so you talked about a lot of on and off kinetics and I wonder if you think about neurons which not only having on and off but also many amplitudes that take a big role in development of cells and synapses how do you measure that so how do you measure like baseline sporadic activity of Calcium well in our case there are different ways of measuring Calcium that's not what we are doing not really measuring, sorry but more like how do you integrate it in your system because it's not really an on and off reaction more like sporadic miniature yeah I mean in the case of Calcium you have these time courses and we look at the complete time course so we have the Calcium concentration sampled at every second or half second in the cell by different methods so our collaboration partners they use different dyes that show fluorescence say when they bind Calcium some others show bioluminescence and then we use these time courses in the Neural system it's a bit different there you also get the analogue mode where neurons are directly connected and they exchange substances but most of the case you have action potentials and they didn't go into Neural systems at all because things they are totally different you get these action potentials they are uniform mostly so they all have the same duration they all have the same amplitude and then people in neuroscience or computational neuroscience mostly they boil the information down to just the timings of these peaks and they use this information and mathematically this is a point process and you can use different mathematical tools to study that we are not really looking into neurons we are mostly interested in non-excitable cells like liver cells, pancreatic cells and so on cells that are not activated that don't show massive depolarization like in neurons thank you thank you and obviously again, number two hi so you mentioned Camkinazis too and I got you don't work on neuroscience specifically but I'm pretty sure you had a quite extensive knowledge in the subject what do you think about this I would say hypothesis that were quite popular a few years ago I think in the US mainly about the fact that the cytoskeletron of neurons can actually encode and decode through kinases in the cytoskeletron memories like bits in a hard drive what's your feeling I'm not going to speculate on that specific hypothesis because I'm not really into that but I know that many people are also looking into spatially effects which I didn't mention here the model I showed you is a spatially homogeneous model we don't look at concentration gradients within the cell our cells are homogeneous at the moment so if people do that and of course then you can look into things for example like a new topic is morphological computation meaning that spatially you can also perform computations but if you're interested in that I mean we can talk offline do you buy into this theory or I can give you some pointers there do you have a good feeling about those theories or you think they are clueless well I think that the spatial effect is a very important thing and that's also something we should look at I mean to me random fluctuations are very important intrinsic fluctuations because you can't separate them from the dynamics of the system they're always there at least some of the fluctuations and also the spatial effects are very important because you have these you not only have these different compartments where the reactions happen but you also have concentration gradients across the cell so with calcium people have looked into calcium puffs and calcium waves because when you have a channel that allows calcium to enter of course directly at that channel you get a much higher calcium concentration and then in some cases you get waves that are traveling across the cell and to me it sounds plausible that this also has a major impact on the information processing thank you thank you ladies, Jürgen, thank you for your talk and please give a very warm applause to him thank you