 Okay. Good morning, everybody. So, um... Recording in progress. I wonder who that person is who says recording in progress. It's like telephones a couple of decades ago. If you were to get a busy number or a number that doesn't exist, there'd be somebody, some voice that would say, this number is no longer in service. Please make a note of it. Who is that person? But in any case, so just to reiterate, the rest of this lecture series of lectures, it's now going to be computer science and a little bit tomorrow on the thermodynamics of computer science. So before making that transition, you guys have been basically a shotgun with a whole bunch of material coming at you from very many different directions. The way that we're going to actually schedule the remaining part is Guilge will be presenting some material in what are called deterministic fine automata this morning. So we're just waiting for her to be able to come down. And then I'll be presenting touring machines this afternoon, touring machines tomorrow morning, and then some of the thermodynamics of computational machines modeled as I was showing yesterday in terms of finite paths. But before going to all of that, as I was saying, you guys have had a whole bunch of stuff. It's still okay to say you guys in a generic sense, you people, you human beings, I've had a whole bunch of stuff thrown at you from very many different directions. So are there any questions? Are there any, especially questions about the foundations, the fundamentals, the way that this extends all conventional statistical physics, equilibrium statistical physics? Sorry, did I hear from over that side? Okay, everybody. Is everything trivial? Did you learn anything? Mark Masomenoff. Hey, don't laugh at my accent. Okay, so a little bit then of the idea of a deterministic fine automata up to touring machines. Who here has heard about the Chomsky hierarchy? Just a little bit. Sorry? I have heard the name. Heard the name, okay, anybody else? Okay, so Noam Chomsky, he's actually still alive. MIT professor. Quite an interesting guy. If you look at his Wikipedia entry right now, you'll see a lot of stuff. He basically revolutionized the entire field of linguistics of our, he was instrumental in our current understanding of how human beings actually generate language. Based upon that, his consideration of how human beings generate language, generate sentences, he actually built a bunch of abstract models of what it means to actually generate a sentence, generate a string. These different machines are part of what's called the Chomsky hierarchy. So the simplest one, which you'll be hearing about first this morning, are called deterministic finite automata. And then in increasing sophistication, there are other ones, there's what's called push down automata and several others. And then you end up with touring machines. These have greater computational power. The way that you actually formalize computational power with these different machines in the Chomsky hierarchy is the following way. All of them take strings as input, strings of bits, for example. That's the input coming into the system. Everything okay? Okay. So are you gonna be ready in just a second to present the go for it, go for it. So the way that my computational power is measured with all these systems is that you've got the machine, which usually has a finite number of internal states, the states of the computational machine. It gets a string, there's a string of inputs as well, a tape, which has zeros and ones. And the machine basically examines sequentially positions on the tape. Up at the level of touring machines, it can actually write to the tape. Back here, you're not allowed to write to the tape. Touring machines can also go backwards on the tape. Deterministic finite automata cannot go backwards on the tape. And there's this notion of a language that is recognized by the machine. So for example, deterministic automata. Here's the automata. It has N internal states. One of them is the initial state. And then there's a set of them that are called accepting states. And what happens, it's quite simple. There's a sequence of bits that come in one after the other. So you have it's zero, bit one, bit two, dot, dot, dot. Every time a bit comes in based on the current state of the machine, you have an update function which says taking in a bit, I'll call it W sub Z, plus the current state of the machine, that just changes to the new state of the machine. And that's all that it does. This is a deterministic finite automata as Gilger will be elaborating. What happens is you send in this particular string of bits and then it ends, the string ends at some point. And then you see, is the state one of the quote accepting states? If the answer is yes, you say that you accept that particular string, that word, if the answer is no, it's not an accepting state, then you don't. The language accepted by a fixed deterministic finite automata is the set of strings that it will accept. Okay, here's one of the very, the fundamental concepts of computer science. For deterministic finite automata, it's very easy to const, well, it's not too difficult, let's say, to construct languages that are not accepted by any deterministic finite automata, no matter how many states, no matter how many accepting states, and no matter what the update function is. Just to clarify, you could have all states be accepting states, but the language that's accepted by the automata, to have a deterministic automata or recognize a particular language means that it will accept words in that language, but will not accept words that are not in that language. So you can't just make a system that accepts all states, that accepts all strings by simply having every state be an accepting state, okay? Notice if we look at this particular string here as a tape, Yacopo? Yes, yep, usually it's an infinite set. So for example, right here, deterministic finite automata, notice that word finite, is a finite number of states. That, combined with the fact that you cannot go backwards and that you cannot write, means that there are many, many languages that cannot be recognized by any deterministic finite automata. For example, we could have a language of palindromes. Palindromes are strings such that you go forward and you get a particular pattern and then the ending of it is the exact same pattern in reverse. For example, that's a palindrome, okay? And you cannot build a finite automaton that can accept all palindromes. The reason is that palindromes can be arbitrary length and the only way that you can actually keep track of whether we are right now recognizing a palindrome or not is based upon which state we're in. And there's a finite number of states, an infinite number of palindromes. So basically, and there's no trick, basically, that allows you to condense the question of am I right now scanning a palindrome or not down to just a single state, so you're screwed. There are other infinite languages that are accepted by deterministic finite automata. These are called the regular languages. Now, in contrast, as I was saying, they make only very, very small changes in a Turing machine named after Alan Turing, which is to say that you can also, in the exact same scenario, but the computational machine can go backwards as well as forwards along the tape in the simplest version of a Turing machine. And most crucially, it can actually change. It can what's called write to the tape. It can overwrite a particular bit. Those two small changes make Turing machines as powerful as every single laptop, and it's actually far more powerful than any single laptop in this room. It's very easy now to recognize palindromes. You folks all have Python interpreters or some such on your laptops. If you just write a short little program to recognize whether an arbitrary length string is a palindrome, you have just programmed a Turing machine that can recognize palindromes. Any of the normal computational languages that we use in our software, those are actually, can all of them be implemented with Turing machines. So all of these, you can actually say, what are the actual thermodynamics of a physical system that implements one of these computational machines from the Chomsky hierarchy, which is what we'll be getting to a little bit tomorrow. But before getting there, one very, very important point, I want to bang on about Turing machines. I should have a soapbox I can climb up to because this is me getting on my soapbox, sort of pontificating. There's something called the church Turing thesis, which people may have heard of, which basically says that any function that is computable by any system, including the human brain, in an ill-defined sense, can be computed by a Turing machine. It basically says that anything that we can do with our brains can, in theory, be done with a Turing machine. For example, a Turing machine that would emulate down to the level of molecules, every single neuron in our brains, and run that forward, it would be able to exactly emulate whatever function our brain is calculating. Now, the crucially important thing, which is very, very tied up, what you may have heard of the notion of chromosomal complexity, is that there are strong limits on what Turing machines can do. You can create functions very easily, well, very easily in hindsight, that are perfectly well-defined, and they are not computable by any Turing machine. So what this means is that modern mathematics, modern computer science, has been able to construct mathematical objects, in fact, an infinity of mathematical objects, which are beyond the capability of human beings to compute. That strikes me as one of the most fundamental philosophical insights ever generated by humanity, and the details of exactly how it's computable or not computable are extraordinarily rich. Kilda? Every time someone mentions church thesis, I wanna still point out, for example, this sentence, for example, that I don't know, the brain functions can be implemented by the Turing machine and so on, so forth, but in theory, Chris Morseburg on how dimensionality actually causes a problem for physical real dynamical systems to implement Turing machines. Can you also point that out, for example? Dimensionality, I'm not sure. It's actually, I would say it's the opposite. For example, I can construct a Turing machine just using three-body motion in the solar system, where the input tape is essentially given by the positions in phase space of those three bodies. Chris Moore, actually, some of his early papers shows how to use the baker's map and so on, and there's lots of people, most recently, Naato Shorishi has actually got a very nice paper in which he constructs a physical system involving spin glasses and shows that whether that property of quantum spin glasses, when you do the quantum mechanical formulation, an important property, depending on the details of the spin system, is whether it's called thermalizers or not. And he constructs a very simple Hamiltonian. He has collaborators and showing that it's not computable in general whether such a spin system will thermalize. Though I would say that actually raises very profound questions about does the universe, in some sense, calculate due computations that we cannot. It's the subject of lots of controversy. People are still fighting over these things and there's no clean resolution. This is all tied up a lot as well with girdles and completeness theorem. As Gilger is emphasizing, the church thesis, church-turing thesis is actually, it's very vague. It's almost explicitly vague. It's actually almost a dictionary definition. It's saying whenever we talk about what is computable by us human beings in general, the thesis is it can always actually be implemented by a turing machine. You can accept that or not accept that. Roger Penrose famously said that no, he thinks that due to quantum gravity, the brain is actually stronger than turing machines. I mean, even, sorry, I'm in trouble, but I mean, before quantum gravity, there are again so many more interesting issues, right? I mean, what does it mean to implement a turing machine that has an infinite tape and a physical system? So what you would normally do in a physical system you have to consider a physically extensible tape is the way that what she's talking about now is the fact that strictly speaking, this tape is actually infinite and in the real universe you don't have infinite tapes, but it suffices to actually say that the tape is finite but it's extensible. That's exactly the way that your computer, for example, runs, can emulate a turing machine is basically, it's only got a finite memory, but you say that well, if it ever is going to run out of memory, it can just go out to the cloud and get other memory and if you just assume that it can go out to the cloud and get other memory for an arbitrary large amount of memory, which of course is not exactly true, but if we were to make that kind of assumption in that kind of a universe, then the computer could emulate everything that we can. So the question then becomes that kind of a universe which does have this difference from us in that the Amazon web services or the Microsoft version of the cloud or whatever is actually infinite in those other universes rather than being finite in our universe, does that have any actual consequences for the fundamental nature of the universe? Anyway, this is just a very, very brief, quick overview of what we'll be getting to in the remaining two days' worth of lectures and as I'm emphasizing, the thermodynamics of these things changes drastically some of the computer science. Rather than looking at things like accepting or not accepting or coming off of complexity, looks at things like the shortest input string that generates a given output. If you instead worry about thermodynamic quantities, you get a whole bunch of what can be viewed as corrections to coming off complexity, for example. And if you do the, instead of you do things in terms of the finite bath formulation of computational machines, which Kluge and I are working on together, then you can actually get things like fluctuation theorems that apply to computers. But that's a very, very quick overview of where we'll be going for these next two days. David, so I have a question, I don't know how silly it is, but so you could think of a code that transforms a string of bits into a sequence of words as deterministic finite automata. Well, the deterministic fact, you have to be careful about the definition. Certainly when you're doing the thermodynamics, it becomes very much a super issue. But this definition of a DFA, there's no outputs. There's only inputs, the strings come in, you jump around the states, you accept or not. You can also define something that's very, very closely related, and that some people with confusing language would also call a DFA, in which for every single state transition, something is emitted. So it depends on the precise detail. You could think of a code as a tree, right? Then you start the state at the top, and then depending on whether you get a zero or a one, you go on the side or on the other, then you have terminal states, which are the accepting states. Yeah, or accepting or not, yeah. Well, yeah, okay, so, but couldn't you think of a code as a special kind? I mean, seen in this way, couldn't you see a code as a special type of deterministic finite type? Oh, in that sense it is, and the bits coming in are what's going down the tree. And so, for example, if the strings were all what's called prefix free, which Gilger described what that is earlier on, then you would actually know it's a binary tree and things like that, and it will be directed tree in that particular case. You can actually even, okay, what you said is okay, something else, but I'm gonna say sort of like complimentary. Even, for example, you can think of it as a device that sort of takes like these binary strings and sort of encodes this in a different way. For example, can I? You wanted to take over? Do you have your laptop up? Yeah, no, not to just, yeah. I don't know, whatever you tell me, you're the boss, but, yeah, so like, for example, this one. For example, when you have some scheme like that, okay, let's do this as well. Let's say that you're starting from this stage. Where am I holding this? I don't know. Okay, let's start that you're starting from here and you're reading this one and then you're reading this one and you're reading this one. Because your machine is deterministic, what you can do is actually also keep track of these states. For example, Q0, Q1, Q2, or like, I don't know, write it in terms of ABC, and sort of like. So those are these states within the computational machine? Yeah, this is actually interestingly, some of the things that people can use the finite automaton for, for example, encoding, like, for example, sliding block codes in information theory, they use it to encode information. Okay, apologies, I had a technical problem with my computational machine, so. Like, this one, not that one, so I guess that's gonna worry. Okay, well, I'm gonna take one minute. Tip, tip, tip. How are you doing? Are you fine? How's it going? Everyone's happy? Everyone thinks that this is the most excited part of the course? Yeah, no? Okay, zoom, almost there. Mateo, what is the pass code? Six, six, nine, three. Okay, thank you. Okay, just before starting, I wanna emphasize something. Recording in progress. Okay, thank you. Yes, now let's also share, I think so. Yeah, yes. Okay, it's always a risky thing to do, but this is not gonna be on, I mean, the details of this kind of things that we're gonna be talking about. These are not gonna be on the exam, at least for this part, like the finite automaton and so on so forth and theorems related. So this actually introduction to DFA isn't something where we are gonna be, for example, like talking about this, oh, the oldest intellectual and just like this interesting aspects of this gold mine that goes by the name of the theory of formal, like this finite automata, sorry, formal theoretical automata, but we're gonna be just touching upon why we should be interested in this, okay? Then David is gonna actually sort of like come back and talk about Turing machines and algorithmic complexity and so on and so forth and we're just gonna give this sort of basic idea and the setup and sort of what I want to do is to sort of inside, I don't know, just fuel your intuition about this topic, okay, so that we can see where we can use physics and stochastic thermodynamics, okay? Okay, so first of all, just this question of just I think clarify some things. Why do we actually see, for example, something as thermodynamics even relevant to computer science, okay? So this is Derek, this person is a clever person. He's one of the titans of physics, I would say, okay, he's not a philosopher, but so like, we are gonna take him seriously and he says that there is no information without representation. So when you read something like that, maybe it just doesn't click instantly but you can actually insert some dynamics inside and try to think about what information processing would mean, okay, if there is no information without representation. So one of the things that we define competition is basically the processing of information, okay? So if this holds, we can basically, I think, think that there is no information processing without the manipulation of representation, okay? This is actually something that you can find, I don't know about this, okay, philosophy of computer science also and so on and so forth but this guy is a physicist, so we care about that. Now there are two different fields of interest to us but also just like mainly super, super, super interesting that actually talks about information processing in terms of different representations, different ways of representations. So in computer science, what you have is a representation of information with more ecological nature, right? This makes sense, it makes total sense. And in physics, this is more of a material nature, okay? And so one of the things that we see observe is actually in these both fields, they actually have this overlap between the way that they approach the information processing and representation in the sense that both fields reason about behaviors, computer science, operational systems and so on and so forth and physics is just dynamical systems. And both exist because constraints do. If you think about computer science at its heart, it's just a field that actually is sort of like developed to understand how do you solve problems, okay? In so many levels, how efficiently you solve them but how can you solve problems and so on and so forth. And when it comes to physics, well things happen because they have constraints. If you have prospects about this, Murray Gellman has some good arguments about that, okay? So what we are gonna be concentrated on is the problem of resources. So in computer science, there's this really interesting field called... I was like, am I tripping? I'm like so. Ahmed, can you please switch off your microphone? No? So maybe, okay, thank you. Okay, so coming back to this one, computer science and physics, both are concerned with resources and most interesting field I think, I think this is a personal opinion of computer science is at least at this point, it's computational complexity, which is asking this main question of okay, we are curious about these problems that we can solve but even beyond that, we're curious if we can solve these problems efficiently if we are provided some computational machine, for example, deterministic finite automata or Turing machines that come with different architectures and different constraints and so on and so forth. And basically computational complexity itself, if you ask a computer scientist, it refers to what is now called the resource complexity, which and the resources come in terms of like this time and memory and space that is given to you when you want to carry out a computational task, okay. So in thermodynamics, while we've been talking about all of these resources, for example, in terms of like this energetic costs to carry out a thermodynamic process or a sustained and un-equilibrium steady state and so on and so forth, so we can have this sort of like this baby little intuition set up to maybe look out for analogies between computer science and thermodynamics, okay. So models of computation to actually ask even, you know, start asking this question of all what we can compute and sort of what is something, what is an architecture that allows us to compute some certain set of problems and so on and so forth, we need to introduce models of computation. This is a model of computation. This is the most basic model of computation that you can consider. This is a model of computation where you can get as universal as possible, okay. So we are gonna be sort of like introducing this, you know, the basic definitions of VFA's and we are gonna be talking about one complexity measure of VFA's that we are gonna be maybe touching upon tomorrow and so on and so forth, but why do we need models of computation? First of all, computers are really complex structures to talk about. If I give you a problem, how are you going to model a computer? What you're taking, I mean, what you're doing is basically taking this into an abstract form and developing these models that we are gonna, again, introduce in one slide or something like that in a more formal manner. So computer scientists started to, again, tackle this question of what can you compute and how can you compute this thing by actually, now what it is called, as a computability theory, okay. So this one, I think, okay, historical, why didn't I write that? Again, if you ask a computer scientist, he or she will say that, okay, computability theories, like a research field that lived between 1930s and 1950s, but actually the roots go even, you know, like just like to lightness. So lightness, okay, interesting person. I think he was living in a pipe dream, like thinking about the best of the possible worlds, but also just basically saying things like, oh, well, we have to calculate and we have to, we have an intellect, okay. So in his writings, he's like a complete philosopher, actually, in this sense. If we have a human intellect and if we are doing mathematics, well, the whole world itself can be actually seen as this product, sort of like this reflection of the human intellect, and we can optimize the whole world and we can optimize mathematics. Okay, so we can optimize our expressions or representations of this physical reality and so on and so forth. So he's one of the first people to our records who told about just sort of like asking this problem of how can we automate our mathematical expressions? Okay, when I say mathematical expressions, I'm meaning, for example, proofs, but of course I'm not talking about proofs because lightness didn't go that far. So lightness failed, you know that, but in the 19th century, there was a very bright mathematician. We all know him, interesting guy, again, all of them are guys, but so yeah, Hilbert, okay. So he was thinking that because this, you know, the formal theory of logic was also sort of, you know, developed in the 19th century, he was thinking that mathematics was a perfect and in his words, a complete structure where you have these formal arguments in mathematics that can be always verified in mathematics itself. Okay, so what Hilbert wanted to do was actually, you know, take this idea of using mathematics to talk about mathematics. Okay, meta in a way. And to basically prove that, oh, everything in mathematics, you don't have to go out, but it can be proven by mathematics. Even though it's an axiomatic formal system where you are basically an axiom by definition, it's an axiom where you come up with definitions you're not proving, but he believed that it's a perfect system. And so he pulls some set of questions and it is awaiting to be solved and so on and so forth. And then there was an even more interesting person. I think one of the most interesting people that lived in the 20th century, Kurt Gödel, he said, no, no, no, no, no, there is something called an incomplete theorem. We call it like that right now. And he showed that mathematics is limited. It's expressive power is limited. So there are some arguments, mathematical expressions in mathematics that you cannot prove in mathematics in your formal system. Because basically because you have axioms and so on. And then, okay, Gödel actually crushed some people's hearts and so on and so forth, made people depressed, but then there was still one fundamental question open that actually sort of, it was a hope inspiring nature, which is basically, okay, we know that we cannot prove everything in mathematics by mathematics, but we can prove some stuff, right? We can verify some mathematical expressions. So the remaining problem was that can we actually develop some, now introducing the words algorithms for everything that we can prove in mathematics? Can we do that? Well, it's a good question, Turing said, no. Okay, so this happened in 1930. So this guy, I think he came around in 1930. And so this is the sort of like marking the start of the computability theory in a formal sense. And this guy actually publishes paper on this universal machines in 1936. And after that, like about 15 years, people produced some results, but then the question changed. So the computability theory was interested in what can you prove, what can you compute, and so on and so forth. But people started to get curious about this question of, like how well you can compute something, how efficiently you can compute something if I provide your computational machine. So this is how where all the beautiful things in life reside in, like p equals mp problems and so on and so forth. So we say that going back to the models of computation and why we need such abstraction, this is a model of computation. And this is the most basic model of computation, but this is a bit boring. You have some states, we will see it, and you jump and so on and so forth. But as, for example, David mentioned, Turing machines are incredibly capable tractors. So, for example, in this class, why are we even interested in deterministic finite automata? So there are actually, I think a way to see this, you know, these kind of deterministic finite automata and so on and so forth, is that it's like a microcosm of computer science, of these models of computational machines, because you can actually ask some questions in a formal manner and get your answers in a formal manner really well. For example, one of these questions is that, does non-determinism result in efficiency? You're gonna be actually talking about the deterministic finite automata and non-deterministic finite automata, and we are gonna be talking about this. So this is some question that we can ask and get some really nice answers to them and computability and complexity. Do they have some relation? Or, for example, do they have some really tight relations? And, for example, we will see that an undeterministic finite automata, a deterministic finite automata, is in this sense that except the same language, they do not differ in the way that they compute, so they sort of like belong to the same computability class, but in the complexity, in the efficiency, in the way that they compute, how they compute, they differ, okay? So this is something that we learn. And then we are gonna be talking, we are gonna be emphasizing that if you want to, for example, let's say that if you're coming up with a set of equivalent finite automata that actually accepts the same one language, you want to, for example, I mean, just like a zoological human being, you want to prefer the minimal one, right? The one that comes with this minimal number of states, the one that doesn't tire you when you run a computation on that machine and so on and so forth, but we're gonna be seeing, even before Timerdyna makes it, comes with some computational cost to get this minimal deterministic finite automata. And, well, it's reverting in the sense that if you're like a multidisciplinary person and you're looking for some interesting stuff out there, it gives you noble prizes. So this is, so the 2011, I think this is something really interesting. That's, I don't know if it's relevant for you, but I think it's really interesting. So one of the things that you can do by finite automata is that you can find patterns in the data sequences and some people actually use that as a mathematical basis to identify patterns in quasicrystals, okay? So I give you, I don't know, like this lattice structure, okay? They took this lattice structure, encoded it in strings, fed it into finite automata. So they take this, basically, you're having this physical structure and encoding it in a finite automata and you're trying to predict the all other possible crystal structures and so on and so forth, okay? Yeah, so that's one thing. And then I think this is the most important thing. We talked about so many things, but this is the most important thing. So does it sound familiar, for example, what David depicted here? You're in a state, something happens, you read a string, you process something, and then you jump to another state. You're basically going through some, for example, trajectory. It almost sounds like, right? So it might be some canonical model to actually start discussing some computational aspects, right? Does it make sense? Oh, I'm failing so hard, okay. So now, formal definition will come in a second, but what we're having is basically a state diagram of the arrows, you don't see the arrows, right? Okay, there are arrows, okay? I want you to imagine that. These guys, they are placed on arrows. Okay, so this is an arrow, this is a transition symbol. Okay, so when we want to represent the finite automata, what we're doing is basically introducing a state diagram. Okay, and in this state diagram, we're assuming that there are some set of states. Okay, now we start to talk about the architecture of this computational machine. Okay, there are some states, and we're mostly assuming that there is a fixed start state. Okay, does that make sense? And then, as David mentioned, there are some except states, okay? Where you basically start jumping between states, and then when you visit, when you come to a state, if you stop processing some, you know, like an input that's given to you, this is an except state. And then you have, of course, some transitions because, I mean, it's almost trivial. If you're not jumping from one state to another, you will not be able to even process some inputs. You're not going to be able to talk about computation. And the, yeah, it will come in the formal definition as well, but there is an alphabet that actually basically allows you to store these strings. It defines which strings that actually can be seen in these state transitions. Okay, so, yeah, input is a finite string. Again, you should remember it from this one. And you're putting this, feeding this finite string into your finite and trauma term, and your output is going to be like an accept or reject. And how do we define computation in such a basic model? Again, it's the simplest model of computation. Your feeling strings, this guy over here, it starts reading, for example, from Q0, it starts reading one symbol at a time, and makes a transition by reading this symbol, okay? That composes a part of a string, a component, it's a component of a string. And basically, when the string is actually sort of like coming to an end, which what this guy does is that if it's in an accept state that is formally defined, previously like initially defined, initially fixed, then it will stop by accept, like an accept message, or it will reject it if it's not in an accept state when you start, when you finish reading the string, okay? Okay, so this is the sort of mathematical formal definition of a deterministic finite automaton, five-tuple list of five components. This is like, I'm just to describe, this is a finite automaton. I think this is like the most regular symbol that we are using. So, Q, finite set of states. And this one, yeah, is just this, right? Okay? And this sigma, it's a finite alphabet. For example, now what we have here is a binary alphabet and this guy here is a transition function. So, what this guy does is that, so you see that Cartesian product of Q and sigma, and it basically tells you that you're in a state and you read a symbol and this maps you to a new state. Okay? So, this is how we define a transition function and Q zero is a start state and K is a set of accepting state. So, this is basically the thing that defines this set up that we call to be this deterministic finite automaton. Okay? Again, yes, as David introduced, we use this term, a language, okay? Oh, before that, okay, maybe I should put, yeah, I should probably put it here. But anyways, so, omega to the north is string and a string is basically a set of string is, for example, it can be written as this. Pop, pop, pop, pop, so this is one string. This is another one and so on and so forth. So, we define a language that is recognized by a DFA to be the set of strings that it accepts or recognizes. We also use the word recognizes. So, accepting again means that you're feeding the string into the DFA and when the string is like over, so you are processing the final component, this final symbol on the string, if you're in an accept state, you accept that. If you're in a reject state, if you're not in an accept state, you're rejecting that. So, this definition is clear, right? Yes, okay, great. Okay, and again, just like as a formal definition of this thing, for the finite automaton, we say that if every language that is recognized by a finite automata, we call it to be a regular language, okay? This is something that you can check for yourself, just like pollen drones, for example, if you want to write down this kind of a language and construct a finite automaton that recognizes this language, you are going to fail because there is no finite automata that recognizes this language because this is not a regular language. Well, this is a regular language. You can try to do that, but it's not really important for our purposes right now. It is more of like, I don't know, formal theory of automata 101 kind of a thing, like if you're interested and you can do that. So, okay, so in these examples that we saw, for example, just like this one, what we had was the following. You're in a state, okay? You're reading a symbol and you're arriving into another state. So, if you know that, for example, you read that symbol most recently and you're ending up in this state, then you know where you came from, right? So it's like, you can reverse this transition function. Okay? What happens if we actually sort of say, oh, let's say that not each state has this unique transition that describes this state jump from one state to another, but there is some nondeterminism involved. So for example, when P goes back to itself, it might go back to itself by reading zero, the processing the symbol zero or processing the symbol one. Okay? So this is what we call, so this kind of structure is like called to be nondeterministic finite automata. So one of the, I think, most important points is that, okay, let's say that this is, okay, determinist, what is going on? Deterministic finite automata and nondeterministic finite automata. One of the things that people tried to see was, for example, again, as we described this slide number two or something like that, if you actually introduce some nondeterminism in your computational to your computational machine, what happens to the languages that you can recognize? So in a sense, for example, this was the problem of computability, right? What happens to the problems that you can solve? We're not talking about how efficient that you can solve this problem, but basically just that language, which problem, okay? So turns out any DFA, of course, it was a bit trivial here. It can be seen as a special kind of a nondeterministic finite automaton where you're basically imposing this condition that transition function takes you from one state to another, like exactly one state. So you know, if you read one symbol, you know where you're going to be ending up, and so on and so forth. So you can trace back where you came from. And conversely, one thing that you can see is that for every NFA, the finite nondeterministic finite automaton, you can find a DFA so that it recognizes the same language so that it accepts the same very identical set of strings, okay? So this suggests that NFA's are not really more like computationally capable than DFA's. But they're like this, you know, this sort of at least speaking on the computability level, they're almost the same. Not one other is more capable than the remaining one. But of course, there are some complexity issues involved. So when we are trying to come up with something interesting in our computer designs, what they're preferring is like the, for example, inclusion of nondeterministic finite automaton, designing nondeterministic finite automaton, and so on and so forth, okay? So this was one of the problems that we pointed out. Again, slide number two, the relations between computability and complexity. Yes? Oh, this is not, there is no probability here. This is just nondeterministic. This is not stochastic or probabilistic. Nondeterministic, just when you got all possible future trajectories, see what happens on any of them. Yeah, of course, there is probability introduced, but then it's called a probabilistic finite automata, but we're not gonna be talking about that. But yeah, it's a small extension. It's just, yeah. So one thing that is of interest to us is this definition of, even before coming to that. So remember, we talked about, for example, having computational machines, such as a finite automaton, is minimal as possible, right? Because for example, I give you some strings and you're processing these strings and let's say that you have two DFA's, okay? DFA like m and m prime, and it has three states and it processes these strings and that is given by some language L. And let's say that it accepts this language in the sense that it recognizes this language as we define just the moment ago, like in a formal manner. And this has, I don't know, 20 states, okay? I give you these two computational machines and you want to sort of like, I don't know, do some pattern analysis and so on and so forth. Which one do you prefer? And I give you a finite time to carry out this computation. Just super trivial. It's not even a question. Yes, you know the answer. Any idea? Okay, so I give you a language like a set of strings, okay? And I give you two deterministic finite automata that accepts this language so that recognizes the strings of that language, okay? And so for example, the way that we define computation was that I'm giving you some strings, you're processing the strings and you're accepting or rejecting and so on and so forth. Right? And I'm asking this question of if these, for example, one of these automata, it comes with three states and the other one comes with let's say 20 states or like 500 states or something like that. And I give you a finite amount of time to carry out a computation in the sense of processing strings and so on and so forth. And you want to sort of like, I don't know, for example, do you want to do either pattern sequencing, pattern analysis, or you want to basically identify the structure of the VFA. So which one do you prefer to run the computation on? Exactly. Exactly. Because when you're designing computers, this is something that you prefer, right? Make everything as minimal as possible. Everyone loves efficiency and so on and so forth. So we're going to be talking about this minimal size VFA's. We want to keep the number. Sorry, but in terms of computational time, they should be exactly the same. Yeah, yeah, exactly. Yeah, this is why. So it's just that the second one is a little bit more redundant. Exactly. You don't want to have redundancy. Like if you, for example, if I, I mean, we're not actually using finite, deterministic finite automata in computers, but there are some elements in our computers that implement deterministic finite automata. So if you, when we, for example, even for just like pure engineering aspects, if you want to increase this efficiency of our computers, one of the things that engineers do is to try to keep like this number of states, so to say, as low as possible. Yes. Yeah, so building on the comments that both of you just made. So for example, right now, anybody who's running a compiler, the modern compilers and the optimization step, they use a deterministic finite automata. If anybody ever remembers in their past or perhaps present, they use Unix, the regular expression checkers that look for regular expressions. The regular expression is precisely defined as the set of words in a language accepted by finite automata. Concerning the definition of the importance of minimal number of states, that in the case of a Turing machine, that is what people will identify with the complexity of an output string is going to be the minimal size of the input string such that you compute that output string. And when I give my presentation on Turing machines in what, like 15 minutes? 10 minutes. 10 minutes, okay. Then I'll be talking about that. You can define the complexity of a finite automaton as the number of states in, oh, sorry, the complexity of a language as the number of states in the minimal automaton that actually accepts that language, which is very similar. It's like a finite automaton version of a common word of complexity for Turing machines. In contrast to the case with Turing machines where there are these hugely profound open questions like whether P equals NP. By the way, the N in NP is non-deterministic in exactly the sense that Gilger was introducing the term. All of those questions are open, they're, you solve that, you are fame is instantaneous and will last forever. Those have all actually been solved. The analogous questions for deterministic finite automaton, which is the only reason that it's not an active ongoing field of research. But what that means in terms of stochastic thermodynamics is that deterministic finite automata provide us a sandbox in which we know actually the answers to all the computer science questions and we can see how those answers to the computer science questions relate to answers to the thermodynamic questions. We can see the connection between the computer science properties of things like minimal number of states and things like the minimal amount of entropy dissipation or whatnot. So in any case, sorry for the monologue. No, that was super helpful, I think. Yeah, yeah, exactly, yeah. Yeah, that's I think most of the last part that basically where he talks about the why we would be using stochastic thermodynamics, we would be using stochastic thermodynamics by sort of basically we're seeing this fact that computer science already is done with, mostly done with deterministic finite automata as an opportunity for investigation from a physics perspective because we know at least what we are dealing with from the computer science perspective. All the problems are mostly the problems are decidable unlike the Turing machines where you have an incredible amount of undecidability when you consider problems. So you're good to go with, for example, a model that is as simple as deterministic finite automata. So again, considering this problem of minimal size, by the way, maybe I should actually give this kind of a, yeah, think first, yeah, before talking about why we would care about that. So why do you consider finite automaton and also like in general in computational machines, there are two central computational complexity issues or measures of interest. And one of them is just like the main, you know, it's lying at the heart of computational complexity theory from this computer science perspective which is like time, memory, space, and a circuit depth, and blah, blah, blah, and all these cool computational complexity classes. And so this is one of them. And the other one is descriptive complexity and it comes with many different measures. It's a bit like a crazy sort of thing, but size complexity is a measure of descriptive complexity and what is size complexity? Size complexity is basically something that is defined in this most simple way possible. Again, I give you, I mean, I'm going to go back and do this formally, but I give you two different DFA's, different DFA's in the sense that let's say that they have different number of states but they both accept the same language. I mean the same set of strings, okay? So they are computationally equivalent. So we call these kind of DFA's that accept the same set of strings to be equivalent DFA's, okay? But we are considering equivalent DFA's that come with different number of states and the size complexity is defined as this minimal, okay, maybe, I'm sorry, I'm super tired. I had like a physical problem before the lecture so I'm getting super distracted. Well, the set of strings, so as Gouda is emphasizing, there's an infinite number of DFA's that will accept any particular language that is accepted by any DFA. If I've got a language, a set of strings that is accepted by one DFA, you can construct actually an infinite set of DFA's that will also accept exactly that same language. So they're computationally equivalent. And so the size complexity of a language that is a regular language that is accepted by some DFA is the number of states in the DFA that has the smallest number of states that actually accepts that particular language. So that's the size complexity. And yeah, so there's a theorem in, yes, formal theory of automata, my home neural theorem, where you basically prove that if I give you a set of equivalent DFA's that recognize or accept the same set of strings, same language L, we know that for sure for each regular language L that we can consider, there's a unique minimal size complexity DFA. For example, if I give you like this language of the set of binary strings that are divisible by tree, you know that it has three states. I think it's just this one. Oh yeah, it's this one on the right one. Or this one that we saw here. Okay? But you can construct some other DFA's. For example, the one on the left one is like, come with a higher number of states, right? And still recognize in the same language. You can see that it's recognized the same language. By the way, just as a, I don't know, when I put it on the Slack channel, just sort of like track these state transitions, okay? And then you will see that they are accepting or rejecting the same set of strings. So they are equivalent in this computational sense, okay? But to actually keep track of this formulae, this kind of an equivalency, we are introducing two different equivalency relations. One of them is over the language. So the set of strings that we are feeding into any kind of a DFA, okay? And one of them is over the DFA's themselves. So we are starting with the first one. So this is the definition of this first equivalence relation. So basically, I give you a language that is defined over, let's say, a binary alphabet, okay? And I give you a pair of strings in this language, okay? And we say that they're equivalent with respect to this language. I think this is the crucial point with respect to the language, okay? If they satisfy this condition for each string that you can consider. So when I give you an equivalence relation like this, it's incredibly abstract. Does it make super sense to you? Like, can you imagine it without actually imagining a finite automata that processes these strings? Exactly, yeah. But still this is useful. And we say that this defines an equivalence relation because if you want, you can check it. It's reflexive, transitive, symmetric. So this is how we define equivalence relations. And this suggests that because we have a equivalence relation that is defined over a language, we can construct what is called an equivalence class, okay? And equivalence class of a string is basically the set of strings that is identical or, I mean, equivalent to it in the sense that, mm-hmm. Wait, they are so boring. Concertination. Concertination of the strings. Yes, exactly. Okay, and so this is all good, but it's so abstract without introducing any kind of a computational machine because I mean, I'm giving you basically a set of strings and this kind of an equivalence relation, but why? So one thing that we want to ask is that even in a simple machine as DFA's, if two strings are not equivalent, you want to be able to distinguish them. If you're not able to distinguish them, we cannot compute with these DFA's. Remember how we define computation for the DFA's? We are feeding the strings, we are processing the strings, we are given an output of except or reject. So if these DFA's cannot distinguish the strings that are not equivalent, basically the strings that are not ending up in, like for example, you have, let's say, you have a string U and it's ending in an except state. And there is some V that is ending in reject state. And you need to be able to distinguish them. They're not gonna belong to the same equivalence class because they are not going to be satisfying this property. So now, building on this, we are defining another equivalence relation. So this is an equivalence relation over the language or with respect to language and this is an equivalence relation with respect to the finite automata itself. And we define it in the following way. So let's say that you're starting from your Q zero, start state, okay? And you process the string U, okay? And in another trial or run of your computer like the finite automata, again, you start from Q zero, start state, but now you process the string V. And what we would like to say is that if processing these string U and processing this string V take you to the same state, then these guys are equivalent. Okay? Strings are equivalent because you cannot distinguish it. Mostly this makes sense because deterministic finite automata is it doesn't come with memory, right? So for example, I don't know what it is. Let's say that one one, I guess, I don't know. Or something like that. So you start from this one, you go here and you go here and you go here, back here. So one zero zero, okay? Or you just read this one. So you go here. So these strings, these two strings are equivalent to one another because when you read them, you end up in the same state. Okay? Okay, but so here is one problem that we need to consider. Is this equivalence over language? It's sort of like, is it identical to equivalence over the computational machine over the DFA itself? No, not exactly. So we can actually discuss that but this is, yeah, this is something that is really important. If you have, first to start with, if you have equivalence over your DFA's, then you definitely have equivalence over your language. Okay? Because language is basically something from the reverse perspective, like the reverse direction, something that we can define as the set of strings that are accepted by the DFA. So you know that if you process strings then you end up in the same state of the DFA's then, and you have this equivalence over the DFA's, then you have the equivalence over the language. Or I think in the same way, but this is, yeah, contrapositive. So if you do not satisfy language equivalence, then you cannot satisfy the machine equivalence. Okay? So, yep. So, building on this, what we come up with is an argument that suggests that each equivalence class that is defined over the strings, that require a different state, the existence of a different state in your DFA. Okay? This is basically to keep track of the fact that there is a DFA that recognizes the language and you need to sort of like have this entailment between the equivalence relations over the DFA and over the language. Otherwise you would keep, if you don't satisfy this kind of a condition that each equivalence class requires a different state, then you are going to lose some strings that are equivalent to each other. You're not gonna be able to keep track of these, for example, we are not gonna be able to satisfy this condition, right? Because we want, if they are equivalent or not, we want to know that when we process these strings in the finite automaton. Okay? So we know that then each equivalence class will require a different state and one really important insight is that if this equivalence relation defines, let's say, you know, like K equivalence classes, like K, I don't know, any kind of like a non-negative number, then any kind of finite automaton that recognizes this language, L must have at least K states, just because we need to satisfy this one that is written here. Okay? So we define then a minimal DFA in the following way. If you have this DFA that recognizes a language, then you can construct a DFA that actually has the number of states which is exactly equal to this K number of equivalence classes that is defined over the language. Okay? So as in this case, for example, you have this three state finite automaton that recognizes this language of language of, like, for example, the binary strings that are divisible by three, but one thing that you can do is construct some different finite automaton. It's gonna be something like this, I guess. Okay? This is gonna be, I mean, we can, I can send you like, I mean, I think it's here. Yeah, sorry about that. This one on the left hand side. Okay? You can construct this one and it's gonna accept the same one, but it's not gonna be minimal. Okay? There are some ways, algorithms, that you can use to, for example, construct a minimal DFA when I give you a regular language vice versa, but we're not gonna go into that because I think we don't need that. So again, then the final point is that by my general theorem of computer science, we know that if I give you a language, a regular language, this minimal DFA that we introduce here, it's gonna be unique. Up to isomorphism, which is a basically cooler way of saying that relabeling of the states and just like the state transitions. Okay? So, one thing that is really important is that, I think, just taking a step back and looking at what we have done or what I have tried to do. So, just the perspective. We want to solve problems, computer science, okay? And we wanna solve them efficiently. And we introduced this basic computational machine that is called the deterministic finite trauma term, where again, as David mentioned, you basically know everything or most of the things, 99%, okay? And these guys are capable of great things, okay? For example, with these kind of concepts that we introduced this language as a set of strings that I recognize by DFA's and so on and so forth. And one thing that we are interested in is to actually start tackling the issues of complexity, computational complexity, okay? Oopsie, what is going on, okay? So, you can try to, you know, you know, just start to investigate in time and space complexity and so on and so forth. But actually, the complexity class of deterministic finite trauma term is like a baby computational complexity class. If this is P, it's like here, okay? So, it's not really interesting. But still, there is something that you can do and it is to investigate descriptive complexities in terms of the size complexity, which we describe just as the number of states in your minimal DFA, okay? So, one thing that we want to do is, again, for engineering aspects or just for fun, I guess, and the theoretical reasons probably from stochastic thermodynamics, what we want to do is to actually ask the questions of what happens to your thermodynamic costs of, you know, like just carrying out this computational process in a minimal DFA where you don't have redundancy and in a different DFA where you have redundancy, okay? These are the things that we are gonna be considering. And I guess this is the thing that you need to know to get an intuition about compute, yes? Mm-hmm. Discussing size complexity to make the transition to take a break. It's done. No, I'm done. So, I'm here. Okay, excellent. So, let's all take a break now. Five minutes. We've been at it for over an hour. And then I'll start on the Turing machine stuff and as I mentioned, that will be continued. I'll continue on next rock as the story went by R.A. Laffley, but continue tomorrow morning. Okay?