 Look, welcome everyone to this session today. We are really excited to have Bartosz Malewski with us. He has done many things in his career, which some of you will know him more for some of them for others. He started out as a physicist. He worked for Microsoft for a period of time. He started off doing C++ programming for the longest time before he switched over to Haskell. And he's moved into category theory of late, which of course he is of some renowned and has written a book on category theory for programmers, which I'm guessing probably what has brought many of you here as well is some of his Haskell work in the past. He did quite a bit of training, I believe. So Bartosz, we're thrilled to have you with us today. Over to you. So let me share my screen. So thank you for having me. I wish I could be there in person. I always wanted to visit India, but we'll have to do it virtually. So the title of my talk, Teaching Optics Through Conspiracy Theory. It's a sort of a clickbait title. But this is a thing that fascinated me always. You do data hiding, implementation hiding when programming, just to like get rid of too much detail so that you can work with writing code, right? Because if there's too much information, then you get overloaded. So we try to hide some stuff, whether it's implementation or data. So it sort of looks like a lot of this stuff, it looks like conspiracy theory that there are some hidden things that you don't know about. And sometimes you want to extract these things from your data types. And it's an interesting problem. Like how much can you actually extract from your data types? So here's a disclaimer. No data types were harmed while extracting their secrets. And we don't use coercion, which is like unsafe coercion in a hospital. There is this function that can do a lot of damage, but we will not be using it. Okay, so let me start with like a simplest example. So I have a function that's called double, okay? And when I call it with two, it gives me four. When I call it with three, it gives me six. With minus one, it gives me minus two. So what is this function hiding inside? I mean, it has a very suggestive title, right? The name double. So it's probably hiding two, right? So my guess is that it's probably implemented as two times X, right? Can we extract this value from it? Well, it's easy, just call it with one and it will give you this hidden value two, right? Are there other possibilities? Of course there are other possibilities. It could be multiplying by four and dividing by two. It could be doing some logarithm exponential and stuff like this, but it always boils down to the fact that it can be implemented by just multiplying by two. And this fact that it really is equivalent to multiplying by two is called isomorphism. It's like this function is isomorphic to a function that just multiplies by two. So this is sort of a warm-up example, but let's talk about something more interesting. So probably you know what a functor is. I mean, if you are at a functional conference, you know what a functor is. Functors are things that hide values and they can hide values of an arbitrary type. So you have a functor of type that's hiding something of type A, right? And all this functor lets you do is to modify this value. Doesn't let you do anything else by being a functor. So if it's a functor, you can say if it's a functor that contains, hides something of type A, you can apply a function that turns A into B and it will turn this functor into a functor full of Bs. So if you have a functor full of A's and you have a function from A to B, it will give you a functor full of B's, right? And this is all that you can do with this functor. And here's an example of a functor, a list, right? A list of A's, a list of A's. And notice that in special case of a list of A's is an empty list. So it doesn't really contain an A, right? So it sort of potentially contains these A's, maybe zero of them, maybe many of them, maybe 10, maybe 100, right? But all it lets you do is to apply a function and will modify the contents of this list. So if you had a list of integers, you can turn into a list of characters. If you have a function from int to char, right? There are usually ways of verifying whether this thing contains something of type A or not and how many of these things it contains, right? So there are like ways of doing this but they are not related to this thing being a functor. It's just that it's a data type and it probably has some other methods other than being a functor, right? So here's a few more examples that maybe is a functor like this. Maybe I have it, maybe I don't have it but you can apply a function to it, right? If I don't have it, then I will still have it. If I don't have something of type A, I will not have something of type B as well, right? So, but if I do contain something of type A, then you can apply the function to it, okay? Another functor, this is maybe a more interesting functor because like it doesn't really contain the A's but it can produce them. So it's a function type, a function type from some E environment, you know? And that produces an A. So it can produce it. Like if you call it with an E, it will produce an A for you. If you call it with different E, it will produce a different A, maybe, or maybe the same A, right? But it's a functor because you can apply a function. What means applying a function to this A? It means that you are post-composed this function G after this function F, right? So post-composing will change a producer of A's into a producer of B's. And there is this functor called IO, the input output functor, IO functor, right? Which really doesn't have, there is no way you can extract anything from it. It just says, I have something of type A. For instance, I have a string, right? What string it is? I won't tell you, right? Trust me, I have it, but I can't tell you. But you can look at your screen, you know? I can print it on your screen. But this is outside of the program. Here's another one. This is an interesting continuation. Continuation is something that says, don't call us, we'll call you, right? So a continuation says, I have something of type A, but I won't give it to you. However, you can call me with a function, you can give me a function that turns A's into some R. Whatever this R is, for every R of your choice, I'll return an R to you, okay? So this function is called a handler, right? Usually, or a continuation, you know? I'll call your handler. Don't call me, I'll call your handler, okay? So a continuation is hiding something of type A. Can we extract it? Yes, we can extract it, just call this continuation with a handler that's identity, an identity function. So it turns A into A, right? And if you call it with identity, it will return you this hidden value A. So this is a way of extracting it. But there is a one-to-one correspondence between A's and continuations of type A. So not only I can extract an A from a continuation, but given an A, I can produce a continuation of type A, right? I'll just do a lambda that takes a function K, that's this handler, right? And I will call this handler of yours with the value of A that I captured, right? So this is like a closure. This is a lambda that is a closure. It closes this A and then calls your handler with this A. So you have this pair of functions, run continuation and make continuation, and they are the inverse of each other. So if you run continuation on something that you created using make continuation, you'll get back the same result. So this is a very important thing, the isomorphism that you can show. Continuation of A is isomorphic to A, okay? And here's the generalization of this. This is like the whole of category here is based on the oneida lemma. Oneida lemma is sort of like continuation, but it uses a functor. So it's really hiding a functor full of A's, okay? And it says, if you give me this handler that turns A's into X's, then I will give you, not just X, I'll give you a whole functor full of X's, okay? So you can create a type onida of list, so F would be a list, because a list is a functor, right? And if you call me with something that turns A's into X's, I'll give you a list of X's for whatever X could be integers, could be characters, I'll give you a list of characters if you want, I'll give you a list of it. What is it hiding? Well, it's really, we know it must be hiding a list of A's, right? Or in general, a functor full of A's, right? Because if it has a functor full of A's then it can apply this function using F map, right? To turn it into a functor full of X's. So in fact, there is a two-way isomorphism against, like with the continuations with Ioneda is like, run Ioneda, if you give me a Ioneda, I can call it with an identity function and it will give me a functor full of A's. And I'm saying, this is the functor full of A's that you are hiding from me, right? And you can also give in a functor full of A's, you can create a Ioneda data type, right? By just capturing this F A and applying F map to it. So whatever function you give me, this G, whatever function G you give me, I will F map it over my F A and I'll return to you a functor full of X's. So these polymorphic functions, they just look at commonalities of arguments. These are polymorphic functions. Whenever I say for all X, it means for whatever type X, right? But this function has a certain structure. It says, well, give me a handler that turns A's to X's, right? So even though I work for every X, what I'm expecting is a function returning X. So it's, or some other thing may be a list of X's, right? So now let's talk about existential type. This is Jean-Paul Sartre, a great philosopher and writer of existentialism, right? And we'll talk about existential types. So usually for every data type, we sort of have this pair of constructor and distractor, right? So a distractor doesn't mean that destroys it. It means just like extracts information that was put in the construction. And in type theory, it's called introduction and elimination rules. That you have a constructor for a data type and then you have a way of deconstructing it, right? But there are some data types from which you cannot extract the value that was used to construct it. So here's an example of a black hole. It's a type that has a polymorphic constructor. So this BH is a data constructor, right? But you can call this constructor with any type A, with a value of any type A. So you can call it with an integer, you can call it with a character, you can call it with a function, you know? And we'll just hide it. And there is no way to extract it. It's like if a photon falls into this black hole, you know, so I'm constructing using this constructor BH, I'm constructing a value by giving it a string, okay? But since it was constructed using a polymorphic constructor, there is no way I can extract this photon out of it, right? But in many cases, there is a way of extracting that. So I mean, it's like if we think about black holes, that there's this thing called Hawking radiation which extracts stuff from the black hole. So here's an example of data type called sum array that is very similar to black hole, except that it accepts an array of arbitrary type. So you know something about the construction. You know that it's an array, you don't know an array of what, right? But you know it's an array, okay? So you can sort of undo this polymorphic constructor by applying a polymorphic function. So if you have a function that acts on an array of anything, for instance, length, length is a polymorphic function, takes an array of A's and returns an integer. But it works for any type A, right? So the sum array that hides an array of A's, you can actually extract the length of the array from this by applying this polymorphic function. And the important thing is that it extracts the length of an array, but it will never disclose for you what type was hidden inside the array. So you know the form, the array, but you don't know what is inside the array, right? And indeed, you can test it, you can build an array of 10 integers, for instance, and apply this function length to it and it will return 10 the length of the array. And you can define other functions like trim, that will just discard the first element of the array or the list, right? But if you try to extract the value of type A, this is the bad function, right? It will not compile. You can apply head to this list, right? But it would disclose the type of A and the type of A is hidden. It's an existential type. This is why it's called an existential type. You know that this type exists, but you cannot extract it. And often these existential types, that's like the most common use, they contain inside two things. The producer of some value of some arbitrary type and the consumer of this value. So here's an example of a data type called height that whose constructor takes two things, a value of type A and the function that turns A's into B's, okay? Now B is the visible type, because this is like parametric by the type B, but A is hidden. A is the existential type. So the only thing that you can do with this type is to match the producer with the consumer. Since consumer is a function, you can just apply this function to this hidden value A. So you can define a function on height, that pattern matches on this constructor and applies F to A. But it will never disclose what type A was. It will return a type B, right? Which is visible. So int is visible in this example. So it's height of type int. So it can return an int, but I'm creating with the type that's of type character and the function or that turns character into an integer, right? And indeed when you call it, it will give you 97, which is the ASCII for lowercase A. So there is like a dual thing to the Yoneida lemma that the Yoneida lemma worked with this polymorphic type that was for all X. This one works with the existential type. So it's hiding type X. And the constructor, the polymorphic constructor has both the producer and the consumer. The producer in this case is a functerful of Xs. So when you construct this thing, you put a functerful of Xs and a function that takes an X and produces an A, right? So what is this function? What is this data type hiding? It's hiding a functerful of A's, okay? And you can show that this is isomorphic in the sense that given a co-yoneida of functer F and A, you can extract F of A. And again, this is simply done by F mapping this function over Fx and you can make it by given a functerful of A's, you can put this functerful of A's into co-yoneida and the consumer will just identity function, right? You can also have a contra variant consumer. So a contra variant functer is like when I say you have a consumer of something, this is what I mean. This means that it's a contra variant thing that a consumer has, instead of having the F map, it has a contra map. Contra map looks exactly like F map except that it inverts the source and the target. So here's a function that takes B and turns it into A. And given a consumer of A's, you can turn it into a consumer of B's, right? By just prepending, right? And there is a contra variant version of the co-yoneida in which you have a consumer of A's together with a consumer. So a producer of X's together with a consumer of X's, right? But this time F is a contra variant functer. And again, you have isomorphism between these two. You can take this co-yoneida prime and you can extract the consumer of A's and you can also, given a consumer of A's, you can produce a co-yoneida prime. So now let's talk about optics, okay? Lens, lens. So there is an existential version of a lens. It's a type that hides the residue of type C. So it means that you have a data type of, you have something of type S and you can split it into a focus of type A and the residue of type C. So like if you have, for instance, a pair A and C, right? You can split it into the focus A and the residue C or if you have something that contains a lot of other data, you know, beside the focus, but you can focus on this A. However, it doesn't matter what C is, because it's an existential. There is for all C and the constructor, right? And again, this has the producer of CA and the consumer of CA together. So you can match them. And this is how you can, you can, these are the only things that you can do with it. You can define a function get and the function get will just take this first function from that splits it into focus and residue from residue and just picks a focus from it, discarding the residue. So because it's discarding the residue, right? It's not exposing the type of the residue. So it's still hidden, right? And the other thing that you can do is to replace focus with a new value of the focus. So you can create a function that takes the residue using from, right? Combines it with the new value of A and then uses two to produce a new whole object. So there's a whole object, there's focus and there is the residue, right? So these are two things that you can do with this lens. And this is usually when people introduce lenses they define get and set. But we have an existential version of the lens. But extracting the residue is not possible. This will not compile, right? Because the residue is existential. So if you try to like take a from and then first which should give you the residue, this will not compile. Residue is hidden. So now the thing that combines this is so when I'm telling you, okay, the only thing you can do is get and set. But how do you know that this is the only thing, right? So there is actually a proof using the coionata that this is equivalent to get set, okay? And this is done by rewriting this lens. So here are these two parts of the lens, right? One is takes S and produces a pair CA. But the function that produces a pair CA, right? This is equivalent to two functions, one producing C, one producing A. So I'm going to split this. So I have one function takes S produces C, another takes S and produces A, and then the rest, okay? So this is just a form transformation that simplifies this. And now compare this with the contra variant version of coionata. So the contra variant version of coionata has a functor full of C's and a function that takes S's and produces C's. So this is the producer of C's and the consumer of C's, right? And if you compare this with the line above, you see, okay, so there is an S to C. So the rest of the stuff, the two other things will be the consumers of C's, right? And indeed you can define a functor or contra variant functor capital F that's parameterized by S and A, but it's a consumer of C's. And it just contains these two things, S to A and CA to S, right? The second part of the lens, right? And then you can write the lens in terms of coionata with this new functor and S. And in fact, you know, since coionata prime of F S is isomorphic to F S, you can plug it into F. Now this F, the lowercase F now is capital F S A, right? If you plug it in, you get a pair of functions. You get a pair of functions. You get a pair of functions, one is S to A and the other takes S A and produces S. And this is exactly get and set. So this is a formal prove that this is exactly isomorphic to get and set. Now this can be generalized. There is a type changing lens in which you can take the focus and you can replace this focus with something of a different type. And if you replace the focus in with a different type B, then the result will be some other type T. So a lens that changes types has these two parts, but it can replace a focus with a different type, right? And again, you can define a get and set for this. A get is the same as before, but set changes the type. So set is actually a function that takes an S and the new focus B and produces a T. But the implementation is the same as before. So lens is just a special case of optics that uses a pair. So it splits an S into a pair C and A and reproduces a T from a pair C and B, right? A pair is called a product type, but there is also a sum type. A sum type in Haskell is implemented as either and you can have an optic that replaces the product with a sum, replaces the pair with either, and it's called a prism. So prism has the same form, existential form except that instead of a pair, you have an either and you can go ahead and have more interesting kinds of optics like you can combine a pair and a sum. So this is called an affine traversal which is parameterized by two existential types C1 and C2. And then can be even generalized even more by taking any kind of monoidal action. So product, sum, products of sums, sums of products and so on, it's like goes on and on. So there is a whole slew of optics that can be forming this way. And finally, there is a data type that combines the producer and the consumer in the same data type. It's called a pro-factor. So a pro-factor is like it consumes A's and produces B's, okay? So PAB is a consumer of A's and producer of B's. Now, if you have a pair of functions, one that turns S's into A's and one that turns B's into T's, then you can apply both of them and you turn a consumer of A's and producer of B's into a consumer of S's and producers of T's, okay? So it just applies two different transformations to the consumer and to the producer. In a typical example of a pro-factor is a function type. Function type is a consumer of A's, function from A to B, consumer of A's and producer of B's obviously, right? And you can implement this dimap, this transformation that takes a pair of functions in and out and it simply pre-composes within to transform the input and post-composes without to transform the output, right? So now here's a puzzle. We have a data type that says for all P, right? But P is a pro-factor. I mean, so far we've been dealing with polymorphic functions that we're taking, they were polymorphic in types, but now this is a function that's polymorphic in a pro-factor. So it's a higher order that's a type constructor that takes two additional types as arguments. So it says whatever pro-factor you give me, if you give me a pro-factor that consumes A's and produces B's, I will give you back a pro-factor that consumes S's and produces T's, okay? What am I hiding? Since the only thing you know about P is that it's a pro-factor, then the only thing that you can apply is this dimap, right? So it has to hide a pair of functions that could be applied to this producer of A's and consumer of A's and producer of B's to give you the result, right? So it has to have inside a pair of functions. It's hiding a function from S to A and a function from B to T. And indeed, you can implement this type ISO for the, if you give me a pair of functions like this, I can implement an ISO for you. Give me a pro-factor value and I will apply dimap to it. Okay? So this was just a small example of pro-factor optics. They're like pro-factor optics for all of these lenses, prisms, whatever, all the monoidal actions. That was just a simple example. A teaser sort of. That does, we've got about five minutes left. This is my last. Oh, good. All right, so takeaways from this. We talked about producers and consumers and specifically producers and consumers are functors that are co-variant, producers are co-variant and consumers are contra-variant factors. So this idea of co-variant contra-variant factors that might seem that this is from category theory, very abstract, but this is really about producers and consumers. And then there are these two, there is a duality between polymorphic types and existential types. Polymorphic types are the ones that accept arbitrary types, but look at the structure of this type, right? And existential types, they hide a type. And existential optics, like existential lens and so on, they combine producers with consumers in one package, right? And the producers in this, in these existential packages, they decompose a bigger data structure into a focus and the rest, the residue. And sometimes the residue is like multiple residues, they're multiple foci and so on. And consumers are the ones that recompose these data types from the same residue, but they can change the focus or multiple foci and functions can be polymorphic not only with respect to types. So like for every type X, I do this, but they can be polymorphic with respect to functors or even pro-functors, okay? So this was a lot of information, probably. If this is the first time you've seen optics, maybe a little bit too much, but if you can like re-aview this at your leisure, maybe this will give you some idea. Thank you. Great, and look, thank you so much. Vartosh, that's been a real insight. I can see some people in the comments are like going, oh, as you said, if you don't understand optics, some of that will be a bit tricky, but I think people have gotten a lot from this talk, plenty to think about and take away and reflect upon. We do have a question there, if you'd like to take that from Jaisung Han. Okay, one question I see. I'm having a hard time understanding that the lens is defined on every type. So the lens is not defined for every type. So if you define a lens for a particular data type, right? You can only define a lens if this data type can be decomposed, right? So you define a type changing lens is defined for four types, right? So these have to be fixed types. This has to be like, okay, this is a pair. For instance, I can split this pair into residue and focus and so on. So these are not arbitrary types. No, no, these are types that are decomposable. All right, Jaisung, I hope they helped answer your question. Does anyone else have any other questions that they would like to add now is the time? Once again, Bartosz, thank you so much for your time today for sharing with us your ideas and your thinking on these things.