 Hello. Hi. Everybody here. I guess that's enough. Okay. So my name is Anu and this talk is called the mutable state monster and how to defeat it. I am a Google developer expert for Android, which means I'm very, very smart and you should all listen to me. So I'm going to tell you a story today and like all good stories, you need a prologue, right? So what is state? Anybody shout it out quick. What is twist? Not like Karnataka. I mean like in a program. Nobody. Okay. Let's ask Wikipedia. Wikipedia says a computer program stores data and variables, which represents storage locations in the computer's memory. The contents of these memory locations at any given point in the program's execution is called the program state. That's a lot of words. So this is my definition. State is the current value of data at any point of execution of your program. Okay. Chapter one, the wizard. So a long time ago, there was a grand wizard. His name was John McCarthy. So he pioneered artificial intelligence, right? So this guy spent basically his whole life trying to figure out how to get computers to think like humans. So in 1958, he came up with a language called Lisp, which you might have heard of. This is what it looks like. So Lisp was the first language to have automatic memory management using a garbage collector, right? Which we have in Java. We all know and we all love our GCs. It was also one of the first language to have atoms, this concept of an atom, right? So there's basically data that could be divided any further. So it's kind of like the atom in physics, the smallest unit of data. But why? Why? So this is 1958. This is just when computers are getting started. They're thinking about, you know, they don't even know what the limits of computation are. Maybe they do because Valentine's Day was there before. But anyway, so this is just the very early days and, you know, the concept of atoms and data that could not be modified is in this language. Chapter two, the monster. So I'm going to show you some programs and let's see if you guys can figure out if they work correctly when they run on multiple threads, right? So let's start the first one. That's big enough. Cool. Okay. So this is a simple check and put kind of operation. You check a map, you put something in it. Let's look at what it does. So there's a map. When you pass it a value, I check if it contains a given key. And if it does not contain the key, then I put it into the map, right? That's how maps work. Okay, cool. Simple enough. So let's say the first thread is doing this, right? I run this function on two threads at the same time. The first thread is checking if the map contains the key. Now thread two comes in and removes the key from the map. Maybe I have different functions. Thread one happily goes and puts it in. Or maybe, you know, whatever the condition is now wrong, because whatever is checked is not what was the state when the other thread performed this operation and then this execution is passed back to this thread, right? And so this happens. So this is known as the check, then act problem. Okay. And this is a well-known problem in computer science, multi-threaded environments, blah, blah. But we can fix it because we're smart. So it throws synchronized block around it. And only one thread can write to it at a time now. And we're happy. Cool. So that was done. Let's look at another program. So this is a simple counter kind of a thing, right? So I have an increment method, which is going to increase the count of whatever variable this is. Simple enough. So here's, so they should not be a problem. This is a very simple program. If I run it on a multi-threaded environment, like multiple threads, it should not be a problem. Here's what happens. So thread one is, let's say here, right? And this plus plus count looks like a single operation, but in reality it's three operations. Okay. So first you have to read, then you have to modify, then you have to update the data. So let's say thread one reads, right? Thread two comes in and modifies the value of count, whatever that one. Thread one also modifies the value of count because it only knows about the previous part. And then thread one writes it back. And our counter is wrong. So this is known as the read, modify, and write problem. Another problem, first I'm going to tell you all the bad things and then we'll get to the good things in time. But we can fix it. Good old synchronized lock around the code. Happiness. Next program. So I have a class called run Frodo run and Frodo is going to run. So I have a Boolean stopped. It should be false in the beginning. Somebody calls run. While stopped is, you know, not true. You keep running, right? Then somebody calls stop. We said stop to true. And you know, it should ideally stop the program. So, okay, thread one comes in calls run. And it started running. Thread two calls stop. Okay, a different thread. Now, they should work. Right? It's pretty simple. Now thread one is checking it's stopped and we expected to stop. But here's what can happen. We expect stopped the value of stopped to be read from main memory. Right? There's also another memory called thread local memory, which is like a cache for a particular thread. So if instead of reading from main memory, it reads from thread local memory, it will not see the updated value of stopped. It can also read from another memory called L1, L2 processor caches. Right? And even in that case, so the CPU also has a cache. Even in that case, it will not see it. And that's a problem. This is known as the visibility problem. Okay? But we can fix this. We've done this before. So we make the boolean molotile. And what that basically does is tell the JVM not to cache it in the processor or the thread local cache. And basically anytime you update it, everybody should see this at all times is what we're saying. Okay? So now you finally know what the volatile keyword does. And we're happy. Okay, last example I'll bore you with. So we have the class called orc humorals. We have killed some orcs yesterday and we've killed some today. We have an init function that initializes them to one and two. Right? Simple. Then we're going to query how many are dead and we expect a proper result. So let's see what happens. So we've called init function thread one comes in and has initialized this part that to call comes in and calls get dead orcs, right? Trying to get the count. Okay, what should it be? So this is one. Right? And the initialization, they're both zero in the beginning. And thread ones here, and then thread two calls this value should be one plus zero and one, right? Right? Okay. So it should be one. But here's the thing, the JVM can reorder incorrectly synchronized operation. There's a thing called happens before relationship, which I don't want to get into today. But basically that's how you tell the JVM that two operations are related and they have to be executed in the same order. Otherwise sometimes it can make optimizations and reorder operations. So this is what we have now. That's operation one and that's operation two. JVM can go ahead and do this. It's like such a betrayal, you know, we expected our programs will run in the order that we write them, but JVM doesn't care. So the same example, sometimes thread one can do this operation before the other operation, right? Because the order can be flipped. So this can sometimes get us one, right? Or it can sometimes get us two. Our program is now non-deterministic and nobody likes non-deterministic programs. This is bad. And this is known as the ordering problem, but we can fix it. We fix everything. So we make them both volatile and some black magic happens and then we are happy. So, but we fixed it, right? Like, I mean, you know, we know the problems and we fix them. So that means we have no problem. So let's play. Let's do a little thought experiment. So we have, let's say you have a real world application and you have a user model, okay? By model, I mean whatever you want to call it, database or a class or network, whatever, some place where user is told. So then we have orders also, right? Because we're an e-commerce app and we have payment, let's say, right? Pretty standard stuff. Then we have an activity which is going to access all this stuff. Okay? Sounds simple? No problem. So then we have another activity because PM wants a new activity. Okay? That also needs this stuff. Now we have to synchronize between these two, right? If this starts, this starts some threads, that starts some threads, I don't know, right? Something can happen. You can imagine instead of the activities then being threads or anything else, presenters, whatever you want. Some place in our app we are going to eventually start accessing more and more data and we are going to start accessing common data, shared data, right? Then we have another activity then because chat bots are a new thing, we have a chat, then we have another activity. That depends on some of the stuff. Eventually your system looks like this, right? Then you have to synchronize across all of this, right? Then it's a deadlock sometime. Then you go like this. You don't want to hear the word synchronize again. So concurrency is hard and our brains are small. I'm a very stupid person and generally most of us are. So we just cannot hold all this stuff in our head. It's not possible. If I see more than 20 lines of code at a time, I forget the other 20. So when you mix shared mutable state, right? Which is what we saw. We have state and you can change it. It's mutable, right? And it's shared across your application or whatever your system. And then you mix that with complexity. If it's small, it's okay. We can still deal with it. But when it gets big and it will get big. Think of like a user class. You know, you have one activity and log in and it's fine. And then eventually you have 500 and user is everywhere. God knows what all you were has. It has a database inside it. So when you do that, you make shared mutable state and complexity. You get mutable state monster. It is out to get you and murder you. Chapter three, the profit. So there was once a profit called Gordon Moore. He co-founded a semiconductor company called Intel. You might have heard of it. So in 1965, he made a prophecy. He said the number of transistors in a dense integrated circuit doubles approximately every two years. So if you have a chip, you can stuff that's a X number of transistors into it. He said every two years, now you can stuff two X number of chips into it. And next year, double of that, four X, 16 X, exponential curve, right? So computers get faster. You know that it happened during our lifetime, right? And the prophecy was true. So this is a graph of CPU transistor counts from 1971 to 2008 and Moore's law. Look at all the, can you see the tiny, tiny names of things? Maybe you recognize some of them. There's a Pentium in there. Go to 8086, anyone? CS programmers back in college, fiddling with dark times. So yeah, so we had those two. And as you can see, the graph just goes up. I don't know how to read graphs, but if they go up, it proves my point, I guess. So as you can see, at the top of the graph, it's kind of slowing down. It's not as big a jump as it used to be because basically in the past decade, the prophecy hasn't held on so well. Our computers won't become significantly faster every year. They're becoming faster, they're like 10%, 15%. It will keep slowing down because we cannot stuff more transistors into processors anymore. We are hitting the physical limits of what can be done. I don't understand physics. That's what the scientists say and I believe them. So the only choice is to put more cores into our computers. If you can't make one core more powerful, you have to put more cores in them so that you can get more done. When you put more cores, that means more threads. That's the only way you can do work on multiple cores. And if we are going to make our computers use more cores, we have to make them parallelizable. We have to write programs that can effectively do this in a safe way and not destroy everything. Our tools suck. Locks, New Texas, right? They... For most application programming, they seem too low level, at least to me. Most of the time. It's not like we don't have any use, obviously they do. But most of the time, for the kind of applications you write, create, read, update, delete, the kind of concern we deal with, they seem really bad. They seem to add much more complexity. And you have to read books this thick to understand what the hell they even do. We need to do better because our processors aren't getting any faster. We're just going to have to deal with it. Chapter four, the Fellowship of the Functional Programmers. This is a file photo. There's John McCarthy, the guy we discussed earlier. Alonzo Church invented a thing called Lambda Calculus, which is kind of one of the foundations of functional programming. I don't know what it is, and neither do you have to. Haskell Curry, anybody heard of Haskell? Language nobody understands. And it has to do with Curry. Rich Hickey, founder of Clojure. Nice language, I like his talks. Okay, so what these guys were doing is they were thinking of an alternative way of representing computation. Okay? So here's the question they were trying to answer. How many represent programs as a series of data transformations instead of a series of sequential data mutations? That's like a lot of words. So we're going to see an example of what that means. Okay? So this is, we're going to do a very simple imperative program. Look at that, nice and easy. So we have an array of numbers, right? You want to sum them totally equal to zero. You've done this a million times. You traverse through the array. Keep adding to the total, and you return the total. Result is 30. Okay? That's the sum of 5, 10 and 15, if you're wondering. Now let's do a functional program. This will blow your mind, okay? So, hang on. So, I have a sum function that has an x. It takes an x and a y to integer. Returns x plus y. Then I have a compute function that just calls this. So, sum of 5, 10 and 15 is also the sum of zero and five, and the result of that with 10, and the result of that with 15. Right? The result is still 30. Congratulations. Now you know functional programming. That is basically it. Functional programming is about modeling our programs in terms of expressions rather than step-by-step instructions. Right? That's what we did, right? In the first thing we said, okay, store this variable here. Run through this loop. Keep adding to that. In the second one, we said here's a sum, and the sum of all this is the sum of these subparts. You've already done this. You know recursion, right? You know SQL, right? Sort of. So when you say select star from whatever, do you write for i equal to zero, go through all the tables, put records into things, give me the... You don't do that, right? You just say, here's what I want. You figure it out. So the idea is to be as declarative as possible and let the system worry about low level. You want to make a table faster, you put an index on it. You don't go and like write an algorithm to fetch it yourself. So, we don't need to know everything about functional programming. We're just going to see some good ideas. Okay? So let's go. So first idea, referential transparency. So referential transparency means there is no difference between the value and the reference to a piece of data. Okay? So let's see this. So I have a book. Simple jar class. This makes this book. I say set title. I change the title. I say set colors. I change the colors. I get a new book. Right? But the thing is the same book object can have different values at different points of time. So it's not referential transparent. When I say a thing is a thing, it should just be that thing. It should not be a different thing later on sometime. Okay? Do not change my thing. It's the point of referential transparency. So to achieve referential transparency, we need to steal another idea. This idea is called immutability. So immutability means data once created cannot be modified. Okay? So I have the same book as before. I say set title. It's not going to happen. Can't change it. That's immutability. Once you create a thing, you can't change it. Okay? But here's the thing. We do want new things. But instead of modifying, we can make a whole new object with the same properties that we want. And if we screw up, we can go back to the old one. The sort of the idea behind it. So let's look at a more, you know, codey example. So we have, let's see, we have a bookstore class. Okay? And this one's the mutable version of it. So we have a books list. Right? It's just a bunch of strings. We have a bookstore. It's the constructor. We can add books. Okay? And then we just add all to the existing collections. Then when you want the books, we return the books. Simple. Something you do all the time. Here's the mutation part. Right? When I add books, I'm actually modifying the books that exist. Right? So how do we make this immutable? So this is an immutable bookstore. So this is what it looks like. The first thing we do is make the books final because we don't want anybody changing that. Okay? Then when we do a get books, we make a copy, deep copy of the existing books. So let's say, so first we make a new array. We go through the existing books. We add it all to the copy. And then we return the deep copy of the array. Okay? Why deep copy? Because if you just return the old one, you're still giving away the reference. Right? If you give away the reference, the list itself might not be modifiable, but the elements inside it can be. In our case, we have strings which are immutable by default. That's why I chose this example. But yeah, like imagine some other class in there. And if you give away the reference to list, which is final, the list might not change, but the elements inside it can be changed if they are notable. So we have ad books. When you want to add a bunch of books, we use the get books method that we just created. So it returns a deep copy. And so we add the new books to this existing copy. We made a copy of what we had. And then we return a whole new bookstore. Okay? So we have the store. Want to add some books. We get a different store. The original bookstore is always the original bookstore. That's referential transparency. And it's beautiful. So the important part, how does this stuff make concurrency safer for us? Okay, so we have a bookstore. The same one we just made. We have a app, beautiful app that shows books. Right? Material design and everything. So we are observing this thing on the UI thread. And basically whatever, that is our source of truth. Imagine it's a database or whatever you want. Just like let's say a class. And we are observing that, right? And anytime the thing gets updated, we want to be able to update the view for the user as well. So we're going to do some RX magic. If you don't know RX, it's fine. It's simple. So we'll see. So we're going to make an observable out of this book. The original bookstore. First operation we want to do is we want to add some books to it. Okay? Then we want to sort the books according to some logic. Right? Then we subscribe on schedulers. Observe on main thread basically. You're saying do it in the background. Give me the result on the main thread so I can update my view. Okay? Simple enough. Then we subscribe to the whole thing and then when we update, when the new books are coming. You've done stuff like this a lot of times. So let's say we have a background thread and we are at the map part. Okay. We made the observable. We are going to add some books. Right? So when that happens, we get a new bookstore. Okay? Because our bookstore is now immutable. You can't modify it. So we get a new bookstore. And the advantage is. You can't have in completely modified data. Your operation has to complete because you're not updating the UI until then. You're not updating the other object until then. Okay? So if this were to fail through an exception, whatever, I added some books. I was halfway through adding some books. Something happened and then crashed. Right? Or it just failed for whatever reason. What happens to my original bookstore now? It's got the original data and then some half-ass version of the new data. Right? Because in the middle of that processing, it like exploded. I've corrupted my data. Right? Here you can't corrupt your data. You can't modify your data. How will you corrupt it? So you get a new bookstore. And then you do what you need to do with it. And only when it's completely done will we change the reference. So let's look at it. So let's say another thing is concurrency, right? That's why we're here. Meanwhile, another background thread writes to our bookstore. It gets its own copy. You don't care. It can't modify our bookstore. Let's say another thread comes in. It wants to do some analytics. It wants to say, okay, what is the current bookstore? It wants to like get the count of the books and pass it to Firebase or something. I don't know. Right? So it wants to do something. Maybe it wants to modify it. If it tries to modify it, it gets a whole new copy. Our original operation is not affected in any way because we are not working on the same data. We get a new copy. No lock-based synchronization necessary. This is the whole point of lock-based synchronization, right? We want our data to be consistent. So, okay, then we sort the books. We get another bookstore. Finally, we change the reference to the new bookstore that we want. And we just update it. So how do we change the reference depends on how your architecture is. Maybe you pass it to the recycler view or put it into a database and send out an event, whatever. But the point is, while you are doing your background processing, your data cannot be messed with. And if anybody is trying to access the same piece of data, he gets a new data. So you don't need lock-based synchronization. Basically, this helps people go home on Friday. And Friday is beautiful. But my poor garbage collector is one thing I always hear and cold mechanists will kill me for saying this. But correctness is often more important than raw performance. This is what I think, right? I'd rather have a correct program than a fast incorrect program. Isn't that mostly what all of us want if that was the choice? So I'm not saying that change everything to immutable today and never use, you know, objects or variables again. What I'm saying is you can use immutability only where you expect concurrency, right? You know where your app has concurrency. You know there are some models or something you're going to be processing in the background. You're going to do some operations on them. You know that. It's your app, right? Those things you can make immutable. And as always measure. And if there is a problem, we work back to the old ways and add complexity, right? Add locks. Add what you have to add. But don't go for the hardest thing by default. Let's do the easy thing. And if the easy thing does not work, then we can go to the hard stuff. Chapter five, a new age. The functional programming is getting popular every day. This is some random Google Trends thing I have pulled up and like all data, this is also bullshit, but it shows my point. Front end people are loving the functional stuff. We have React, Closure.js, Elm, Cycle.js. JavaScript people do not want to write anything in JavaScript. They make languages. Provided. Then we have backend languages. Erlang. We have Elixir. Closure and Scala run on the JVM. They do all this functional weirdo stuff and they seem to work fine and run huge applications. So maybe this works. What about good old Android though? So we have Java 8 on Android. Not really. They have streams, lambdas and method references. Streams I think you can only use above API 24. Is that correct? Yeah. So which means maybe 10 years. We have Rx though and it has all the filter stream map and all the nice operators you can do to do this stuff. So one thing about Rx is it's kind of meant to operate in this paradigm. So if you start mutating data inside your operator, it's not going to work out well for you. There's a reason there's a side effect operator section in the wiki. So when you mutate data, when you change the state of the world, it's called a side effect. And so if you use immutable data, Rx becomes easy and you can do that. You have Kotlin. Kotlin has, I wrote first class, but it has like nice support for immutable data. Much better than Java anyway. So if you guys are using Kotlin, you can try this out. Awesome. You don't have to do it all at once. Okay. So the next time you have to do something in the background, just think about it for a minute. See if you can use this. If it applies, that's good. It's always nice to have another tool in your toolkit. And don't blame me when everything explodes. Because I told you so. And you don't have to do it alone. This is functional programming is not as weird as it sounds. There's like a lot of books, tutorials, blogs, everything online. This Google-like functional programming in Java and you get things. And you can do that in Android as well for the most part. Right? And you can learn more about it. The future is bright as long as you listen to me. That's my talk. Thank you. Hey, so regarding that bookstore class example, you didn't make it final, I believe. So this means someone can just subclass it and add some mutators. That's true. That's valid. I should probably change that in the slides. But don't copy that example word for word. Have to forget it. Thank you for the great talk. So you said that this RxJava or like this observable thing. Okay, so as you have shown that, you know, there were so many books. Okay. And then you pull some, so if you pull some books from the database when they change, okay, continuously, can we use it for something like an infinite scroll and something like in WhatsApp, you know, that you continuously scroll and the data is fetched on the back end continuously. It is observing the database. So is it a good use case or it's not the best way to do again things like that. It's hard to say exactly. You've got to try it out. If you're doing something in the background, mutability, like, you know, if you have wherever you have concurrency, mutability is a good idea to try out at least. Right? And then you have to measure the performance impact because again, it's always going to be a little heavy on the memory, depending on what kind of objects you are creating. So because Java on Android does not have like an efficient implementation for things like this. We got to be careful, but I guess so. No, so if the data is not changing by different threads. Okay. The data is not not changing by different threads. Then you have no problem to deal with. Then you have, if you have no concurrency, then shouldn't be what should be the way then this should not. This is not the best case in that case. I don't think so. Again, I'll have to see your code. But like the point is to kind of make concurrency. As you mentioned for concurrency, you return a new object every time so that it doesn't mutate the original data. Doesn't that cause a bloated object? A bloated, you know, lots of it might duplicate the data. It might, we have some mechanisms to make sure that it is not exactly the same data or every time I request a new object, it gives me a new object. Right. So I'm saying if I'm using, you know, if I'm compositing an object, an object inside another object, and that is building up like this. So will that not cause a bloated object? I mean, like if you have a, I don't know what you mean by bloated object. Bloated just means, you know, a lot of duplication that there are restrictions on memory. I'm guessing right. I mean, like if you create a thousand strings, those are duplicated. Yes. So if you have super deep nesting inside your objects, probably not a good idea to deep copy the whole thing. That's what I'm asking for a very large data set. So is there a different approach for concurrency there? So that depends. Like if you have a very large, very deep nested object, right? You can think about splitting it out into smaller objects. Okay. And then individually then immutable. Yeah. Like then, then it becomes easier. I think. Okay. Good part. Yeah. So, but then again, this is a very case by case kind of a thing. Thank you. So, do you know of any like plugins or something which can help, which can help us create such classes? I don't know. Does the intelligent have something like that to make a thing immutable under studio auto value makes immutable by default? Yeah. All right. Not sure. But I guess it's not that hard. Maybe I know should you can generate that for you. So, my question is about the bite code thing you showed. I have no idea where you are. Okay. Yeah. So my question is about the bite code you showed. Bite code. Not bite code. You should write Java JVM re-orders. Right. Ordering problem. Yeah. Yeah. When it generates byte code. Any idea when it re-orders like that? So, there is a concept called a happens before relationship which needs to be established. That will take a little more time to explain. So, let's discuss that offline. Sure. This looks very good in theory. But in practical on low end devices, how does it perform? Have you ever tried it in real life experience? I tried it in real life experience. My real life experience has been decently good. But again, so if you're creating a thousand copies of the same object, it's going to cause pressure on the memory. Right. That's inevitable. But do you really have a thousand threads running in your application at any given point of time? Usually most applications will have three, four, maybe five background operations running concurrently at the most. So, maybe you create and then let's say two operations, two threads are working on one operation. So, you have ten threads. So, you might have five duplicate copies or ten duplicate copies or twenty. You've done far worse than twenty, which is normal programming. Again, try it out and you know, maybe it's useful, maybe it's not. It's very hard to like for me to stand up here and say, this is the way to do everything from now on because that doesn't work for everyone. So, somebody asked me a question. I can't hear. That's it. More questions? No, we're running a little late. So, I'm afraid we have to close questions. If you have any questions for him, please meet him outside and for all other speakers. We're going to take a couple of minutes to set up the next speaker. Meanwhile, please fill out your feedback form. Okay. Thanks guys. Bye.