 Okay, hi. Good afternoon, everyone. Thanks so much for coming. Welcome to Obvious. I'm Padmini. I'm head of communications here. We just wanted to tell you that there are just a few housekeeping rules to observe. The bathrooms are to your left. And there's coffee and tea and water. Please help yourself during the course of the conversation. We have a code of conduct that will be enforced during the event in case you want to take a look at it. It's on our playbook, which is available at the link on top. If you need anything, do just come and talk to me. Otherwise, enjoy the event. And I think we're gonna focus. Gautam's gonna talk, maybe a little less than he anticipated. He will because we think that we need more time for discussion and questions and answers. So that's probably going to be the, that we're gonna get. So yeah, just hand over to Kiran, who is from, hosting this event. Thank you. So I'm actually hosting this event on behalf of the Kona community. We are a group of people from technical and legal backgrounds who have been working on decoding how technology intersects with the law. And Gautam's been part of the group for years now, at least. And Gautam's been a huge source of understanding of how to interpret the law for those of us coming from a technical background. And so that's something I want to understand. How many of you are from a technical background? Okay, and how many from a legal background? Okay, and how, other backgrounds? So it's designed, there's marketing in the back over there. I'm sure there's a huge diversity of it. Yeah, music, okay. So part of what we're going through in our modern society is used to think of the law as being somewhat moat while we went about our day-to-day lives. And the law was just something to be afraid of because you would get caught crossing the road when you're not supposed to cross and then you pay a fine or something trivial like that. And over the last few years, at least for us in the tech community, realize that our work intersects a lot more than we are comfortable with. And a lot more than we have awareness of. And therefore, the software that we write has consequences for human rights that we did not see coming. And we did not know how to think about at all. So part of what the Kalana community has been trying to do over the last couple of years is bring that awareness into mainstream society. And obviously, we don't do this by saying, here is something to read, but we try to do this by saying, go where people are already reading, which is mainstream publications. So if a journalist is writing about software problems and how they're affecting citizens, we try to help them understand what is the software in the first place and how is it affecting people in ways that may not yet be manifesting as problems. So that's one side of it. And Gautam, obviously, a constitutional scholar has been helping us understand how to think about the law in the first place. And the books, I think, are fantastic reading. This is his second book. He's got another one on free speech. Both of them are really worth reading. I think Gautam's an incredibly good writer. He also has a blog on Indian constitutional law and philosophy. It's a blog that is both extremely readable and is also well read in that you will find Supreme Court justices reading the blog to understand how their own bits are making sense outside of their own courts. So Gautam, over to you. Thanks. So first of all, thank you to Hasgeek for inviting me and thank you to Obvious for this beautiful space. It feels very nice to be here. I should begin by also clarifying how I come to this whole issue. And so around 2015, 16, I was approached to help out with the Adha litigation, the constitutional challenge to Adha that was then pending in the Supreme Court. And over the next three years, I was heavily involved in research, in some drafting, in briefing lawyers to argue the case and so on. And that was when I began to understand the interface between technology and the law and more specifically the constitution and human rights. So that's how I came to all of this. And I should also therefore clarify that I am a lawyer and my background is legal. I'm not a technologist. And so obviously my expertise is not in that domain. And so I think what at least what I will try to put across through the course of this discussion is to maybe bring to light some of the constitutional principles that I found coming up again and again in cases like Adha and in other similar cases and how I as a lawyer found the interface between technology and human rights to be. And perhaps that might give some of your ideas about how these principles, these values might be relevant in your work. And we could of course take that forward in the discussion. So I want to begin with a small anecdote that I came across very recently, although the broader context of that is something I've been reading about for a while now. And this story begins in Chile in 1968. Chile 1968 is when a man called Salvador Allende who used to be a doctor working in rural areas, bringing medical services to poor people finds himself the president of Chile at the head of the Socialist Party and he's won the election. And this is the second attempt. He lost the first time. The second time he is successful. And one of Allende's first real goals is to mitigate the severe imbalance in the control of resources in Chile because over the course of history, a few private players had basically captured ownership over most of the important national resources in Chile. So Allende begins with nationalizing some of the important natural resources. But unlike the top-down socialist model that we've come to associate with nationalization, he doesn't stop there and he doesn't think that that is what this project is about because what he's really keen to do and to bring based on his experiences working as a doctor in rural areas is to bring a genuine democratic participation in decisions about how resources are to be used, how factories should work. And so his idea of management is decentralized and it envisions people having significant decision-making control over these kinds of activities, what we now know as the idea of industrial democracy. So and the other unique thing about Allende is that he is very, very curious about technology. And so he combines these two interests and he wants to use technology to implement his idea of a certain democratic decentralization of management and control. To do that, Allende's advisors approach an eccentric Englishman called Stafford Beer who has been engaged in all kinds of ideas and writing about how tech can be used in management theory called informational cybernetics that he pioneers in many books. And so they approach Stafford Beer and Stafford Beer flies to Chile to advise Allende on how to develop this idea and that becomes something called Project Cyber Sin and that is a photograph of the control room of Project Cyber Sin back and this is built in 1970. So much before the real explosion of tech and its omnipresence. So they build Project Cyber Sin and the way it's supposed to work is that you have a number of telex machines that connect individual factories to this control room and you have inputs coming in from these factories that are on a regular basis about production, data, figures, what's happening there. And then these are fed into economic models that give you information about what that means and then decision makers sitting here have access to all these screens and then they can take calls on whether something is going well or going badly and so on. An interesting application of this was when, So Allende is severely disliked by the outgoing establishment and so to attempt to bring down his government they organize a strike of truck drivers to attempt to prevent resources from being transported across the country and therefore create a shortage. So Allende and his advisors use Project Cyber Sin to understand where to bring the resources to at what times and with the help of small and small number of trucks 200 trucks, they're able to break the 40,000 trucks that are part of the strike by strategically going to where resources are most needed. So that's one application this is put to. Now the ending is not a happy one. So Allende is overthrown in a military coup in 1973 and General Pinochet takes power and the first thing he does is to destroy Project Cyber Sin. So that it ends without it ever actually being taken forward and genuinely becoming a model or a project and Stafford Beer then also has to flee and he has survivors guilt for the rest of his life and doesn't really do anything after that. That's how the story ends. But the really interesting anecdote I want to put out could we have the next photo please? Is that yeah, so when this is a photograph an illustration from Stafford Beer's book where he's basically connecting how the firm works and how the human body works and he's selling the project to I&W using these diagrams. And so he's basically comparing the control room that you saw a few minutes ago to the brain and how the brain directs the intelligence inputs of the body and directs various movements in the body. And so he's selling it to I&W using this illustration and he's about to say that look this is how it's all going to work and that brain there in that illustration, that's you. That's what he's going to tell him. Before he can say that, I&W leans back in his chair smiles, looks at the brain and says that's the people. And that for Stafford Beer is a moment of conversion where a number of assumptions that he had held all his life about management, about control, about how the firm should work. In one line and one sentence, in one illustration are completely overturned and later in life when he's being interviewed about this project, he recalls that moment as being one of the most important moments of his life where he understood exactly what I&W's project was about and this whole idea of if you think of the brain as being one person in control, the tech that you're going to then use will be utilized in certain ways, certain assumptions. If you think of the brain as not being the president or the leader, but being the democracy or the demoss, the people, you think of it very differently and its use becomes very different. And in these small illustrations and in these small words, the entire idea changes. So that's the anecdote I think that at least is stuck with me since I came across it a few days ago and for me it helped to clarify what was so unique and different about the project. And if you're of course interested, there are a lot of articles and scholarship on what this whole thing was about and where it might have gone had it not been terminated early by a military coup. And that's kind of the framework within which I want to have this discussion that we often think of tech as being something neutral, external to the world and its effects as being divorced from it. But often the effects come in the design itself and in the ideas that animate why we are doing what we are doing. So I want to start by the two quotes that that I've come across. I came across during the course of research for my book, The Last Chapter, of which deals with some of these ideas. The first quote I came across was in a piece in Factor Daily, an interview with the CEO of a company called FaceTiger. I don't know if the company is still around and if it's going. So the CEO was talking about the uses of FaceTiger and he says, and I'll quote, imagine a guy walking on the road at 2 a.m. who is looking suspicious. The police patrol can take the suspect's photograph with our app and within a second receive details of his crime history. This was in the context of the Chennai police using the app in a marketplace before a festival was going to happen. And the CEO obviously thought that this was a very useful thing to check crime and to ensure law and order in the streets and so on. That's the first quote and I'll come back to that. The second quote is something that Sham Devan, the leading lawyer for the petitioners in the Adhaar case, said during the course of oral arguments in court, I was a few feet from him and I heard him say it and it was quite a powerful moment. So when talking about Adhaar and how Adhaar resets the relationship between the individual and the state, Sham Devan said that the constitution is not a charter of servitude. Those were his lines. And I'd like to, at the end of this discussion, draw the link between these two quotes, which may not immediately seem obvious, but I think that there is a fundamental and important link. I'll begin by unpacking what Sham Devan said in a little more detail. So when he said that the constitution is not a charter of servitude, what he meant was that the constitution as an idea, as a document, as a vision is fundamentally about power. It talks about the existing relations of power, that exists in society, and it provides various pathways, various signposts, various methods to alter some of those relations of power. That comes across in various specific provisions, which we could talk about. It comes across most vividly in the three words that you find in the preamble, which is now becoming quite popular. It's been read out in various protest gatherings. The three words that originally come from the French Revolution, liberty, equality, fraternity. In a very basic way, the idea of liberty is to increase freedom by constraining state power. The state as we have found out in vivid terms recently is a very, very powerful institution, and it can do a lot to all of us. So the idea of liberty as enshrined in various constitutional provisions, like the freedom of speech, the right to life, the right to personal liberty, the right to conscience, and so on, seeks to create a zone of freedom by putting limitations on the kinds of power the state can exercise upon you, and in the ways in which it can do so, and when it can't do so. The second idea, of course, is equality, and equality is also about power. So if you think about the long-lasting debate about reservations, good, bad, should we have them, should we not have them? It is fundamentally a question about how over the last two centuries, relations of power have put certain kinds of barriers between people and access to very basic goods like education, public posts, and so on. And the vision of substantive equality that the constitution subscribes to attempts to break those kinds of power relations by ensuring that access is now provided through even proactive and affirmative means, like, for example, reservations and other ideas that we could talk about. The third idea is paternity, in very basic terms, the relations between individuals without the state playing any part. And that idea comes from a unique Indian history where the state, despite being very powerful, was never the only or the sole center of power. They were always social institutions, cultural institutions that exercised as great and as severe a kind of domination and power of individuals. Cast, of course, is the most vivid example of that. And it's interesting that Ambedkar, one of the chief drafting people of the constitution, one of his foremost political movements was for access to water from village drinking wells because Dalits weren't allowed to use the same well as Apakasvar. And Ambedkar framed this movement as about being the right to access public space. And that found its way into the constitution through provisions like Article 15, clause two, that prohibits discrimination between private persons in access to shops, hotels, restaurants, other public spaces on grounds of caste, gender, and so on. So you have these three ideas, liberty, equality, and paternity, each of which are concerned with the different kinds of power relations that exist in society. And the idea of the constitution is to interrogate them, to undermine them, and to create a new kind of equation where democracy, which is the equalization of these relations, comes not just in the equation between the individual and the state, but also in private relations involving private persons because the kinds of power that operate there are as serious as exist when the state is involved. So, and so for many years after the constitution, this entire idea of power revolved around access to physical spaces, to physical goods, reservations being the one example, surveillance of a very basic kind, watching your house, and so on, that was often an issue. Sedition, all these ideas, they were about, they were set in the actual physical world. Over the last few years, what's happened is that introduction of tech into this space, and therefore the connection between tech and the constitution is that tech intervenes into these relations of power. It could actually entrain some of them, perpetuate some of them, but also continue with the constitutional vision of undermining them or democratizing them. So, to understand how let's come back to the face tagger code, right? So, just to reread the code, imagine a guy walking on the road at 2 a.m., who is looking suspicious. Police patrol can take the suspect's photograph with our app and within a second, receive details of his crime history. I want to focus on four phrases here. The first is the phrase, looking suspicious. If you think about it, you think about 2 a.m. at night, walking down the street. Who is it that looks suspicious and to whom? In this case, obviously, this interview is given to a factor daily. The people who read it are people like you and me. So, who is it to us who looks suspicious? Now, if it's one of us, dressed like one of us, walking down, say, Church Street after a late night, that person wouldn't look suspicious to us. They would look what we think of as normal, and so they would to the police as well. Also, the addresses of this interview. The people who look suspicious are people who don't look like us normally, who are not dressed like us, whom we think as perhaps being from a socioeconomic background that is not as elite as us, and we think of them as perhaps being on the way to commit a crime or there for purposes that are not particularly healthy or good. And I'm recalling this incident just from yesterday that there was that sting that India today did about the attacks in JNU. And one of the people whom they've interviewed said that he was going with a stick and he saw a person with a beard who looked Kashmiri and so he went and beat him up. So, you can see how the very term looking suspicious is not a neutral term. It's all about who is doing the looking, the watching and what looks suspicious to them and what they're in a position to do about that. And in this case, obviously, the police who will, if they think someone looks suspicious, can take their photograph and upload that into the database. So that's one thing. Second is the police patrol. Now, in which neighborhoods do you find police patrols? Where does the police go and patrol? These are questions of political geography. And if you do a study in any big city, you'll find that certain neighborhoods and certain areas receive far greater police scrutiny than others do and these are along the lines of income, along the lines of class and often along the lines of religion. Srinivas Kodli, I'm sure you all know who's not here right now, has done a lot of research in Hyderabad and shown how in Muslim neighborhoods, police have these cordon searches where they just randomly go in and ask people to give their other details. So the moment you mention the word police patrol, you automatically make certain assumptions of where the police is going to be and in those neighborhoods, are the police thought of as protectors, as people who are meant to keep you safe or are they thought of as invaders who are meant to cause you trouble? That changes depending on which neighborhood you're in. Look at the U.S. there's a lot of research on how you have what they call stop and search where the police can stop you and search you if they think that in your car you might be carrying heroin or drugs and there's a clear link between a black driver being stopped and searched which happens far more than a white driver. So that also happens. The third of course is the suspect's photograph and when does the police feel entitled to just take a photograph of an individual without their consent and upload that? Presumably again in church street, we're walking down the street. There would be a little more hesitant before just taking a photograph and uploading that. A kind of hesitation that may not be that apparent in other neighborhoods. And of course the last is crime history and if you look at the statistics and we'll come back to this later on, you will find that the prison population in India has disproportionate representation of Dalits and Muslims. So the relevant correlation between the population that the Indian population has of Dalits and Muslims vis-a-vis the number of them in prison is very disproportionate. There's a far greater number of them in prison than actually in the population and of course, a simple answer to that is that it could be because they commit more crimes but if you go a little deeper, you'll find that a lot of biases operate in the criminal justice system that result in this kind of a ratio. You see it a lot in the recent protests in UP where the police themselves have admitted now that they have taken a whole bunch of people from Muslim neighborhoods that weren't involved in protests so they just pick them up randomly and these kinds of figures then inflate the prison population of these communities. So the moment you're uploading some of that photograph and checking against the crime history, you should know that the data that already came in and made the crime history itself is not free from all kinds of biases. And so you see how this seemingly straightforward statement of taking a photograph and uploading it into a crime history at two in the night on the road where somebody is walking and looking suspicious carries within it implicitly a whole host of assumptions, biases that can perpetuate already existing kinds of marginalization and vulnerabilities that exist in society along various fault lines. And but once you have this app that does this, it's out of your control, right? The police will use it and they will use it in the various that they didn't fit. And so in the design of the app, you have given the possibility of these kinds of uses which will then be magnified in the actual use. And if you want more examples of this very vivid and concrete examples, there's a great book called Automating Inequality by Virginia Eubanks and she takes the examples in the US of the housing market, for instance, of the decision of custody of children and so on. And she shows how it's not that tech is itself creating problems, but if you have existing fault lines in society, then the use of tech can magnify those fault lines or it can mitigate them. And that's where the key question arises. Now, I want to link back briefly the example, this is the statement that the CEO of FaceTiger made with the Constitution because I've talked about the problems that it creates. But what does the Constitution have to say about it? And let's begin with the idea of equality because that's really at the heart of a lot of debates around the interface between tech and the Constitution. Now, for a long time, we used to operate with a very narrow understanding of equality. The idea was that unequal treatment can only take place if there is an agent, a specific individual, who is doing that unequal treatment and that agent has to be guided by malice, by bad motives, by bad reasons. So, if a cop, if a policeman deliberately and intentionally stops a person on the street because they are black and searches them, that's discrimination, that's unequal treatment, but for that, you have to have a specific person, the policeman doing that, and you have to show that the reason why he did that was because he's a racist, he doesn't like black people because he's motivated by, you know, really intent towards that race. That was the definition of equality and our understanding what unequal treatment means, of course, a very narrow understanding. Over the last couple of decades and in the Indian constitutional context over the last few years, in cases like the 377 case which you all know about and the privacy judgment which also I'm sure you all know about, there's been a much more nuanced and evolved understanding of what equality means. And this is an understanding that no longer looks at inequality as being the result of individual choices to discriminate individual maliceous motives, individual racism, but as a function of design, of design that goes into how our institutions operate and how our social structures operate. So this kind of an approach looks at not the reasons or the motivations underlying actions, but at the results of the effects of those actions. So if you find, for example, that in a workplace, the proportion of men to women far outnumbers the proportion in the larger population, that should give you an indication that something is wrong. You may, after detailed research, find that the reasons underlying that have nothing to do with gender disparity, but more often than not, you will find that actually they do. So for instance, if a particular workplace does not have a pregnancy leave policy in place, then it will discourage a number of women from applying for the job in the first place. And then that will result in a skewed workforce. This is a very basic and simple example. But of course, inequality can have many more nuances that are not as basic as this. So this understanding of inequality takes into account, first of all, a long history where there was a prior history of intentional discrimination, which is now being done away with, but whose effects still persist. The philosopher Bernard Williams is a very interesting example where he talked about how imagine a society in which only a certain class of people can become soldiers, informally in law, you have the discrimination written into the law. Now at some point, you remove that. And so now formally, it's all equal. Anyone can apply for the army. But you also find that over the years, because the army is revered in the society, those who were in the army, people of a certain class, had access to the best kind of health care, the best kind of access to the most nutritious diet, all of that. And so now you have a seemingly formally equal test. But you also find that only those people who are already in the army are now in the position to make use of the neutral criteria you have put in place, say a physical test and so on. So obviously you have to go beyond just neutrality and ask yourselves, what is the underlying reason for the effect being unequal? So look at the historical reasons, look at how institutions work, unquestioned norms, unquestioned biases, and look at the effects. And then in that sense, you get a much broader idea of what equality means and requires. And so when you come back to the face tagger example, what you find is that under the old and narrow view of inequality, you wouldn't find anything wrong with this. But because, okay, you have built this app, the point of the app is actually to ensure law and order. So the police will use it to try and check suspicious people moving around. And so there's nothing in the app itself that is biased. But okay, maybe the odd policeman has a prejudice against Muslims or against people of a certain class. But we can always deal with that by training him or giving him or her lessons. Whereas this broader idea of equality goes much beyond that. And looks at the systemic presumptions and biases that are underlined, the police force, the criminal justice system, and asks how that links to what the app is doing. And so of course, if you ask those questions at the beginning, think about what this app will do, what your goals are, what your purposes are, then you may end up designing something different. What you would do, obviously I can't tell because that's not my expertise. But if those ideas are in your mind when you're starting off, as opposed to at the end of the process, then you may find that the design is something else that takes into account all these variables. You're still talking about, in the police example, you're still talking about a more conscious bias in the police force. But it might be even more subtle and even more nuanced. So one thing that's on the radar right now is the DNA bill, which talks about DNA data banks and having a data bank of criminal suspects. So what this data, the proposed law does, for example, is that it says that if you're convicted of an offense, then your data will be stored permanently in the data server and in the DNA data bank. And if you're the trial going on, then there's something more. But even if you're a suspect of a crime, still the police can take your DNA and put it in the data bank. Now again, coming back to the whole idea of how the criminal justice system substantially over represents certain sections of the population in prison, even as under trials without being convicted, you'll then find by extension that the data bank will then have a disproportionate number of people belonging to those communities, who's previously violated, who will be subject to the possibility of false matches in the data bank and so on. So in the US they actually found that because the DNA data banks disproportionately had the DNA of certain communities, the risks of false positives, and therefore of false convictions, exponentially arose for those communities. And so again, you may think you're working on a DNA data bank, but if you see the chain of causation, you'll find that it leads to consequences that you wouldn't even imagine when you're working on the bank itself because those aren't necessarily in your mind because what you're working on is just making that data bank as good as it can be. And so I think that at the very beginning of the process, thinking about these constitutional values and principles and how they would impact the project at all its stages would go a long way towards thinking about issues of design. Often these would require having difficult conversations with people that are not always immediately within your immediate context. So I remember that at the whole time of the Adhaar debate, there's a person called Bejwara Wilson, some of you might know him. He runs this organization called the Safai Karamshari Angola. And for the last 30 years, he has been involved in the fight against manual scavenging. So as you know throughout India, we still have this practice, technically outlawed, but it's still very much there. And he has been fighting for the rights of people who are forced into manual scavenging for many years. Now, he was one of the strongest opponents of Adhaar and his reasons were reasons that I, coming from a very different background, couldn't understand fully back then and I still don't understand completely myself. So he said, and he wrote an article and it's online and you can read it. So he said, he was talking about how certain schemes that were specifically targeted at the community of manual scavengers required mandatory production of Adhaar in order to avail of those benefits. And he said that this whole process is essentially fixing us in a particular identity as manual scavengers. And all our lives we've been trying to escape that to move beyond that. And now this mandatory production of Adhaar to get those schemes essentially locks us into that identity, makes us things of us as being fixed there in that position. And we want to go out of that. So we don't want this whole idea of one nation, my entry that the government is pushing. We don't want that because identity for us is not source of pride. And we're trying to leave that behind and you aren't letting us do that. And I couldn't understand it back then fully. And as I said, I still can't quite figure out what that means in concrete terms. Maybe if I spoke more with him and listened more to him I could understand more where he's coming from. But clearly you're talking about a person here who's really been in that field for many, many years and who has an objection to this particular piece of tech that requires a lot of listening and understanding to even grasp for those of us who have never been in that position. Much more basic issue, of course, was that a person who is homeless doesn't have a permanent address, had particular problems in getting an Adhaar. At the time of designing it, at the time of asking those questions, people who have never had to deal with what homelessness means. And there's no reason why you should be thinking about those issues. But there are real issues that then come up after it's all done and it's too late to, and then all you have are little fixes here and there, which actually again shows you that the issues of constitutional design are really important at all stages of thinking about these projects. And I think, again, Eubanks has the best statement on this and I'll just quote from there. She says that when automated decision-making tools are not built to dismantle structural inequalities, their speed and scale intensifies them. And just thinking back or something that Nilekani said, way back, move fast and break things. Ultimately, you're not just breaking things or breaking people as well. And that's something I think which is very important to remember because the point Eubanks is making is that what tech brings is speed and scale. And if you have those fault lines which I've talked about, the speed and the scale then tends to intensify them unless you're thinking about how to actually undermine them and mitigate them. So what is to be done, and I'll close with that, but just an idea, in some countries and now among some scholars, they have developed what they call technological self-determination. It's a big series of words, but basically the idea is that at all points of time, individuals should have the right to decide for themselves how they will engage with a particular technological system or a piece of technology. They should have the power and the ability to decide on what terms they will bring that tech into their lives and on what terms they will engage with it. And they may choose not to engage at all. And that's fine, they don't have to engage with it. Because again, coming back to something else that Nilegani said, he said once that if you don't have Aadhar, then you will lose the right to have rights. And that's again a way of telling statement because the right to have rights comes from Hannah Arendt and it comes from the idea that in any political community, you have citizens and citizenship is marked by the fact that you are a person who has the right to access all those rights of equality, of free speech and so on. On the other hand, if you're not a citizen, then you lose even the right to access those basic rights and that fundamental distinction is what has been responsible for much of the oppression that we have seen over the last century. And so technological self determination tries to bring that concept into play and tries to set out a very abstract and basic idea about the link between constitutional values and principles on the one hand and take on the other. So I'll close with that and I guess open it up for discussion and questions and yeah, so it's open. Thanks.