 Right good evening that's loud how's everyone doing you're lying exhausted is the answer how many people feel exhausted that's a good thing right hopefully you've got a lot of interesting food for thought throughout the day yes no go one of the things that keeps coming back to us over and over again is essentially what as a data science community what are we doing about ethics how are we ensuring you know that this you know for a lot of people AI and anything that you can do with data is really powerful tool so what are we doing to ensure that it's not getting misused right is that a question you guys have there has to be an infractive session this is the last session of the day right so that's kind of the theme for this how many people have been in a fishbowl before quick show of hand two three people all right so I'm gonna quickly explain what a fishbowl is so the way a fishbowl works is we put some chairs over here we're gonna try and invite a few people from the audience to come up here and express their viewpoint on the topic that is being discussed always there will be one chair that is empty which means anybody from the audience can come and take that chair when they want to add their viewpoint they believe that they want to add something to the conversation and at that point one of the people who's who's already been sitting will have to leave to ensure one chair is always empty so this is a structured way of having like a large group discussion where you know we you kind of control you know to have a kind of focused discussion but then it allows people an opportunity to come and join the discussion all right so it's a participatory panel if you want to think about it where people come up they join the panel and then they kind of leave when someone comes and takes the seat right this is to involve everyone here especially when it comes to ethics AI ethics in AI I think this is a this is an area where we don't have quote-unquote you know experts per se everyone's kind of trying to figure this out everyone trying to think about what is the best way to do this so I'll quickly introduce the kind of overall theme for the panel and then we will invite a few people to kind of seed the panel to start with the fishbowl to start with and then we will kind of rotate through this so let me did you guys have a chance to look at the abstract yes no all right so just for the benefit of everyone I'm going to try and quickly cover the kind of key topics that we want to highlight in this discussion and it's more like you know looking for people who have a viewpoint on how to handle this so the first one is especially when it comes to AI I mean how many people are concerned about black box AI we heard the term black box AI right everyone's heard the term black box AI so the big concern with black box AI is basically you know certain decisions have been made on behalf of users without the user being able to rationalize or explain why right so there's there was a session earlier this today by Deepanjan on explainable AI right and there's been a lot of interesting work that's happening in that space around explainable AI and ensuring that you know AI algorithms can be explained the decisions that they are made so you know I would want to invite people who have experience of viewpoint on building you know not just explainable AI but you know in general AI that is transparent that you know allows the users to understand the rational behind certain decisions the second kind of area broader area is around safety you know we collect a lot of data especially when we are doing any kind of data science on it how are we ensuring that that data is safe and being used in the right way right so lots of different companies have different policies and things like that like one of the one of the classic one at SRADA conference I think one of the keynote was that you know the speaker showed an image of essentially nuclear waste and try to draw the correlation that data is like nuclear waste right you collect a lot of data but who's responsible for the byproduct of that you know do we have strict policies around purging the data do we have strict policies around that so again I want to invite people who have experience having built some of these systems are having worked with some of these systems where they have dealt with safety aspects of you know the data that is being collected the next area that I want to briefly touch upon is around the fairness aspect so this fairness is about you know whatever decisions that are being made by the AI system is it kind of you know fair is it is it ensuring that it's not biased because of the gender because of the race of people or because of any other condition so as as far as possible we want these decisions to be completely you know out of any context of this specific user in the sense of their race their their gender or things like that so how have you ensured that you know fairness has been taken care in the algorithms that you have built and the systems that you have built and the last one I think is important as well is you know that you know it's not just about humans right it's also about you know in general ensuring that the overall you know mother earth is not being affected by some of these decisions that we are making so how are we ensuring environmental stewardship if you will which is which is kind of taking into the context the broader context of the entire environment around us in which we live so what I want to do is I want to invite people who have some experience having worked in these four areas anyone any any of these four areas and kind of share their thoughts around what are some of the things they are doing to to ensure that these things are being handled in an appropriate manner so I know you know Jared has some experience I'm gonna try and pick him for this one so if you please you know come up she moves if you want to please join in got them you want to join in go to again I'm just trying to seed the panel with some people that I know have experience having worked in this space but like I said this is a fishbowl which means that other people can kind of join in when they have something to kind of contribute to this this discussion so I don't know if anybody else wants to join please kind of feel free to come up and and kind of join in anyone who's dealt with these broad four areas that we have you could get started with you know some of your thoughts on how you have tried to handle or how you think we should handle this and then we'll see if more people can join in so which topic should start with that yeah yeah transparency safety fairness and environmental stewardship right so anyone that you will transparency is fun but fairness probably affects the most people fairness yeah right because you hear so many examples of machine learning algorithms through no malicious intent of the designers going wrong just because of the bubble that they live in and so part of the answer is including people outside of their bubble but how do you do that and shy of them doing that because you get a lot of people who just for no good reason can't escape their bubble how do you institute methods statistical methods for preventing that and this has implications from you know dispensing soap and bathrooms to criminal sentencing in courthouses so it has everyday life and grander scheme of things it's hugely important yep blue right yeah and you know I can't really call myself a data science practitioner these days I mostly spend my time writing word docs for odc and I remember we started odc five years ago this really wasn't the radar but now it's it's a big issue and I would love to hear from the audience like how you're how you guys are tackling this because it really comes down to the data science practitioners and there's no clear answers here for example and you don't want your model to be sexist right so how do you do that one of the ways you do it is you remove sex as a factor of the people right but what does that teach your model so a lot of times there's kind of tough trade-offs here like you obviously don't want to make your your model bias but the issue is there's layers of issues because I think data collection there's no such thing as a universal set of data all data is bias by definition right you do a poll it's only from a sample of people you're collecting data from a certain group so I think you know the data collection point is already built in bias there and I know on supervised learning is very topical but you know what do you know how do you know what your supervisor is doing and then you do when you do when you're labeling data you might be doing it yourself might be doing a mechanical Turk and how do you control that so there's a lot of issues around that but I think the good news is the awareness has been heightened last 12 to 18 months I think does anyone know about the IBM 360 furnace toolkit you hear that yeah so not not plugging IBM for this closure they are not there are no easy sponsor but not plugging them but they did come out with an M a 360 toolkit and I think they they identify something like seven-year-old potential bias factors in the model so you know again nothing next to myself but I would employ you as practitioners to look at what's going on out there and I could talk forever about this I'll hand the mic over to me up coming from more from using data for social good we have realized most of the organizations start collecting data without understanding how they are going to use this data and sometime I'm knowing start collecting data which has potential of much more drawbacks than use cases so things like using photographic data we have seen a lot of organizations first collect photographic data and then after they have collected like millions of photographs then think how they are going to use it so this is also like a infringement of first of all privacy and also not a very well developed approach of how you are going to plan and scale your data so of creating frameworks which are much more specific to your use case much more aligned to the legalities in your geography is something which is more helpful as IBM 360 degree check there is another community-driven approach towards this known as SPAT amel fear accountable transparent machine learning it's a community of practitioners trying to create much more fear accountable transparent decision-making around algorithms and I would encourage you people to go through it and share your views on that and I think also the conversation in India is evolving Indian person data protection bill is in discourse at this moment and soon would be heard in both lower and upper houses so I think it would be an interesting time for data community to rethink how we have been using data and think through different aspects like how we would have consent verifiable consent how we would have data archival process and how we are going to do much more algorithmic audits in this case I'd like to actually plug one of my students one of my students a fellow named meals Bantalan gave an excellent talk at a meet-up about how to reduce bias in machine learning and he'll be presented he's going to actually present it to lawmakers so everyone looks up like neils n i e l s Bantalan d a n t i l a n and bias and look for his talk there's at least one video of it out there I know for sure and that will give you a lot of you present a lot of great methods to deal with this and I couldn't even do justice I would encourage everyone to check out his work so neils Bantalan will give you some really good ideas thanks so I'll talk about some of the work which we are doing in American Express about making sure you know to bring this fairness in the process so one aspect is the regulatory part of it which definitely you know make sure that we are not using any of the attributes from the fairness standpoint but again the challenge remains is that the underlying data on which we are building the model so suppose we are using the transactional data now the transactional data you know which we already get from the system think of something like you know if you have to use a very basic bureau attribute called FICO score or civil score in India now if you look at civil score the civil score for someone who is young 20 25 year of age versus someone you know about 30 there will be a difference over there because the income ranges are there the parameters are very different over there so as such as a parameter we will definitely not take age into consideration in defining our model but in terms of the outcomes of the model definitely some of these things will have more skewness towards the people who are more senior in the age but then we have different type of offerings alongside you know which we can which we bring in the picture this is on the outcome of the model that this is an outcome what kind of offerings can we make to different set of individuals so I won't call out that you know as such if there is a fundamental difference in terms of the characteristic of people there will be a difference in offering for sure but by purpose we make sure there is no skewness in the data from the point of view of fairness of you know what we what what is allowed in the system that's a good example of how you ensure fairness is taking care yeah please this is like I said it's open for anyone to sit as long as there are tears so on the topic of fairness I like to share an experience with you regarding a certain application so how many of you use the seeing AI app anybody it's basically an app that was designed for people with let's say you have a visual impairment so when you walk it can tell you who's coming you know is it a male or a female or you know what is the scenery in front of you so that gives you audio input so in real time it takes the visual input and gives you an audio input so that you can make better sense of the environment so I was trying this out because one of my uncles had a visual impairment so I was seeing is it ready for it yet right so I was experimenting with it so I started taking selfies right now interesting thing is it does detect that I'm there and if I train it it tells okay Ranga is there Ranga has my name but as an additional point it also tells my age it it predicts my age now the interesting thing to know is after that I realized that my age is not a single number it's a range from 17 year old to 51 year old I was quite amazed and I was like so I was taking at different angles and I noticed this then what I did is I called some of my friends in India who have a similar skin tone and look a little bit like me to try this and amazingly enough their results were also a broad range then what I did is I called some of my friends in the US some Americans some people of other races other than Indians and asked them to try the same experiment out and lo and behold the range reduced so it was more accurate for that set of people or let's say in this case the race of people who did not have a darker skin tone more like me it was more accurate for them so my point is so this is also an aspect of fairness so when an app has been released and people are using it is it being really fair to me what if certain judgments were made based on the age prediction that it was giving me so that was kind of my comment and experience about fairness thank you a lot of those apps if you smile they say you're younger right so maybe you haven't been beaten down by life yet but in terms of you know the skin tone same thing of voice apps Alexa recognizes my voice and what I'm saying much better than recognizes my wife because the people building this tend to be white dudes in their 20s and 30s who hang off other white dudes in the 20s and 30s so who do they test it on they need to get a better sample like you're saying really about collecting the data has to be collected by people outside their bubble and again it wasn't malicious by those people just like oh well they think oh well it's math it's gonna work the same for everybody there as well everybody's a little different and the math has to take that into account and somehow we need to get them out of that bubble I just wanted to put forth a question it's sort of a devil's educated question so humans by themselves are not fair why do we expect models to be fair could everyone get the question so his question is for everyone not just yeah folk sitting up here but the question if you want to so well as humans we are not like moment we see someone we are also judging them and why do we expect models to be fair there is empty if you think that one of these guys will get kicked out I think the question is very valid in the sense like in the morning I was also thinking on the same thing so like how can we expect a model to be very fair and all the answer to it is like a human which might be working it might be impacting a very you know maybe like 10 people 20 people and all when you try to go go with a model which is impacting like million or half a million or the larger population the impact could create a more so that is the one thing second thing it is yes as a human being we are we are biased towards some of the stuff but if the system where we are building it is like either it is held through the processes either held through multiple checks we don't believe you know one person is making ultimate decision we add you know multiple layers to it and if we hand over it that to the system we are you know trying to take out those layers so in that case it will be very difficult to manage it so that's why you know the main focus is on building those those things into the model through my perspective yeah now if I could just add to that I think I don't think anyone necessarily sets out to do evil with these models and as we already said it's the kind of law of unintended consequences but think about your learning pathway right the focus on you know building your programming skills building your algo skills building your model skills and early on the first thing is to acquire those skills and I think it's only in the last couple years that people have started to understand that's the education that that's where you need to address these problems they have to become part of curriculum to take for example the we were just talking about certification but you know is there a good data science certification but some of you may be financial analysts or where there's a good good paradigm is the the CFA certified financial analysts and I don't know when they start to introduce but they started to introduce risk and ethics especially after a financial crisis right like people just doing bad stuff and a lot of times it was people knowing they were doing bad stuff a lot of people didn't realize the consequences what they're doing so I think it really comes down to education and not just in the universities but all and a lot of online education courses and you see it like I have to I have to say it's come a long way in the last couple years but the other the other problem too is okay it's not not the problem I should we state that it's like it comes down to you as a data scientist like you have to make it your job to be ethical and making your job to be ethical doesn't mean like I'm gonna raise my hand of the ethical you have to educate yourself about the law of unintended consequences like a lot of these bias issues it's we can talk about some model interpability and deep learning is naturally opaque so it's very hard to understand what some of these models are doing so I think you know a lot of it comes down to education and so if you're if you're in that environments like start to look for those courses start to ask for them and they're definitely the same the company I think if you're if you're running a company right now if you're leading a data science team and you're not a way of these issues I would run my company because you know it's I came out of a finance background and you know we knew that's one or two bad mistakes to bring a company down I think it's the same now with Sam if you get your modeling wrong it could have severe consequences for your company and your career so I think it's something that people should be very well aware of so don't mean to put a negative spin on it but it's got to be part of your your your career path is understanding those issues and I think you'll be well served by paying attention to it so come a lot of folks in the community just to close the loop on that I think few more people want to respond to that question but I think going back to Jan's question that you know if humans are biased and if humans make judgments on certain things why do we expect machines not to do that I think you made two points one was that you know humans making you know those is has a limited impact while machines are models making that has a massive impact second with humans usually there are layers of checks and we try and reduce that with machines maybe there is no layers of check and it's it's kind of and I think to Shemu's point you said that you know if you make one bad decision it could bring down the company kind of just summarizing those two points in in one point do you guys have a contrary view just I want to make it a little bit more exciting right so I want someone to disagree so so my viewpoint is sometimes humans are not aware that they have this bias so for example if you're doing a company interview right you're asking questions about okay you know all the interview questions about machine learning AI and they answer it but but you're a human so sometimes you know maybe the dress they wear maybe how they look maybe the tone in which they answer these are things that are ineffable like you don't pay conscious attention to them you're only noting down their answers you're saying but in your mind these things register at a probably a subconscious level so which means I would say that humans are probably not natively have a bias but they are unaware of it but the beauty of you know when we bring this in a model is that it creates that awareness and when it creates that awareness I have an opportunity to correct it and lastly I would say it's easier to correct in a model than change the human being I'm gonna go along dig into the further than that now we build laws and court systems and we're supposed to do better we we expect you know we can improve and the machines are chance we can make an improvement that's you know in the past few thousand years of human evolution and human history supposed to try to get better and better at stuff so that's why I expect the machines not to be biased because we want to do better I would just like to include a point there so we were talking about how nobody makes a conscious decision to create a bias or something like that or do something unethical with their models but we actually had a session today where we were taught how we can get a person's location even though he has not approved to give that location to for example Twitter or Facebook and but still we are making a model even though going against the person's wishes wish to get not get that location but we are still doing it through a model that we created and we know that that person is not giving us that location but we are still working towards that so isn't that unethical on our part of the creator of that model? I'd say nobody had that intentions we hope that most have that good intention the argument there could be that that you're trying to for the good of the society you're trying to find you know the location of certain bad people you know bad in courts you know but you know how do you control that that's not being used misuse for bad so I think that's a great question and I would like to hear everyone's view on that question right so kind of shifting the question a little bit now the question here just if you want to kind of quickly summarize your question yeah so we said that nobody has that unethical bias in their mind when they are creating that model but when we talk about such situations where we are getting that person's location and we haven't made a decision court hasn't made that guy a bad guy we are still doing it and I was also told that Twitter if I use their API and they and I give them a geopolitical code and they would actually return the tweets even though that person has not disclosed this location but still they are giving me that information and Twitter doesn't get to make a person bad or a good guy right so your question is is that's bad I think you're kind of hinting that it's bad so what should we do about this should we not build such models or if we do end up building such models how do we ensure that it's not being used okay so should there be regulations around things like this so I was having this conversation with Sunil Abraham he runs this research organization known as Center for Internet Society and has been on the forefront for internet research in India and we were discussing how to be sure algorithmic accountability so he quoted a very important example that autopilot has been there for a while we should treat algorithms which have that much impact on human race as autopilot algorithms so autopilot algorithms go to a stringent check and it is a string industry level check which is kind of standard for every by every airline organization to build so so why can't we have certain industry level checks in in e-commerce like like you were mentioning where we have been collecting certain information without being aware of certain parameters they should be penalized using industry level check yeah and I think to add to this point so I completely agree but the ethical is kind of you know one topic which is more related to the human being the ethic value for me could be different and for the other person it could be different so as long as there is one thing which and even we we find it with you know the project or the product we work it is like the big example for the you know cigarette industry or something right it is not ethical for me I say it is bad in bad or injurious to health so I might not work on some sort of a product which is adding or increasing their revenue but there could be someone who who doesn't thinks it so in case of yes it is kind of in the regulation or the there is a system which says it is a wrong thing we should not do it and the best example the recent case of that the site the the deep and dash dash D was something which has been taken down from the platform right there was some guy who has built it but everyone who was part of that community who think you know deep learning is for the beneficial we could use it for the good purpose everyone appreciated their effort you know once one that site was taken down so that's I think it should be and from the regulation and all yes every country should be doing it and it should not be you know back on that so the Europe has done with the GDPR US is already having some of the regulation and properly you know some of the things which are coming to the India as well so I hope it will it will get stick with that just to add on walking up some of the points being raised so I couldn't help but see a parallel with the open banking initiative I don't know all the nuances around it but one key summarization I would have in my mind is that the person wants the data pertaining to him rather than the entity collecting the data about him so as long as you're able to give the power to him there are going to be certain people who are okay with their location being tracked every second of the day they can see some merit in it they wanted summarized there could be on the other end of spectrum some people who do not want any bit of information track about them so if there is regulation or other activities enabling people to have the power that could perhaps I think while we are at the topic of regulation you know one of the things we need to discuss as well is while regulations can be agreed upon as a community how do you ensure the regulations are kind of actually you know being followed and audited or whatever like what's the process after regulations have been agreed upon it's a kind of firm that UK open with GDPR and like how do you audit a company's data which has data centers across the world like how do you even go doing that and I think there were blogs on Google saying hey if I want to do that I was spending more money than the fine that I have to pay and they're okay paying the fine then what they define exactly well I mean for a business they'll be like okay I'm gonna pay the fine I don't care like in the states I'm sure most countries when it comes to finance and whatever comes to money people straight enough a little bit they their ears perk up they have regulatory authorities I'm sure it's replicated in other places that have active watchdog groups sort of like private investigators but public investigators I should say and they have machine learning people on their side going through what these banks and hedge funds are doing now of course that it's so highly regulated because it involves money and they can have this watchdog group I don't know how easily that's replicated in other industries because these banks have to turn over their books they have to turn over their research have to turn over that and that's why I don't know if you can say Google turn over your algorithms maybe make enough laws but Google has more money than the federal government they'll probably fight those laws that's the problem so because we are on the topic of GDPR and regulation I just wanted to share a couple of points both related to the data and the model and how we in our organization deal with it right so we are a global organization we have data both from Europe US and a pack but one of the points was that after GDPR came through we were very skeptical about loading the data from the European customers to say the server in the US for example so we were very stringent in following up with legal we had to make sure that we remove all personal data just the internal customer number and everything else was removed from a data standpoint right and so from a transparency standpoint of course GDPR I think enforces that you have to have a check mark to make sure that you are able to at least use the customer data so I think that comes through regulation but do we have that stringent regulation in other parts of the world I'm not too sure if it's being enforced as well as GDPR and to the point of model explainability and model transparency I think what we do even for the simple models that we build in our organization is that we have like a checklist or an explainability sheet wherein we tell in basic layman terms what the model is doing for example if you have a recency frequency monetary model like an RFM model we explain what are those parameters that are being displayed on the dashboard and what the model is doing at the back end just so that an analyst or an end user who's looking at the data knows or understands the nuances of that calculation so I know sometimes for complex models it might be difficult to put something like a cheat sheet but for simpler models if we can at least start with that like like a data dictionary more like a model dictionary I think that might be a good starting point in terms of model transfer what do you think that's a I think that's a great starting point in my opinion so what I hear if anybody else has experience having built like a model dictionary like you call it there's an R package called model down which you give it a model and it actually goes through and explains the model to the best of it can depending on the model so it exists I'm pretty sure it's called model down and it automatically gives you a good cheat sheet for that model and they give everything for you to use it even even with those things models still miss out on answering the basic question of why that model thinks that sort of brings back to your point is the model just mathematics does it even have bias sort of spinning it back but yeah wasn't there talk today about causal inference right someone gave that talk I'm not sure if they're here but that's explaining the why but doing causal inference yes about the experiment not about the data so it's hard to do I would like to comment on actually overall thing what you have commented right four topics that may be a fairness security I would like to relate this well with our daily life suppose in every one of our houses there will be a nice that will be of course very sharp but when we heard about sharp knife initially we thought of something dangerous but 99.99% will use it for cutting vegetables cutting fruits there's a truth of life in the same way when you have data even every company will have a Apache server where every user will access there will be a high P you can use MMD will look up to have complete longitude latitude so it's all about company how you want to use the data whom you want to share the data so what I would like to say is suppose if you have a knife you cannot harm me just because of there are rules and regulations by the law in the same way unless you have a standard regulation it cannot be solved it's because when you are my manager you are saying my task my mind will not think about many things I want my job security so this is what happened the many instance will not come out because who has to say suppose if I am using a model to create a civil score I have used all type of features that may raise age even his location everything I will use who knows this the company employees has to say how our company management has to say that so no one is going to say so how to control this just with the fear if he used that feature he is going to penalized whether you say it now or it going to be later so this type of fear we have to create by penalizing this by creating a log globally so this is what because ethics if you say everyone want no ethics but no one will follow even if you see index conference there are many people while eating so it's a human nature right we cannot avoid that so I want to conclude in my opinion we should have a force to control this thanks counter points to that who controls the first not quite a counterpoint to that but slightly contrarian point so to put it very simplistically let's say every model has some utility delivers and there is this newer focus on privacy fairness all of that suddenly we are valuing these aspects more than the utility the model has been delivering so if a company is trying to comply logically they should you know stop offering this model to the set of people until they can get the fairness all of that big then so in an attempt to comply with the fairness expectations the utility is being denied now is the utility more valuable or is fairness more valuable and who are we to decide for every single person some people value the utility so fairness should be incorporated yes but how much of a weightage does it have these are utility that's why he was making the point that use the force so I have a counterpoint on that I feel panelizing is definitely one way to look at it but I think where the more energy and focus needs to go is the explainability so if suppose someone's civil score is low now the question is that you know when we are asking for a civil report typically it provides you a reason that you know why your score is low because of you have defaulted on one of your loans or your income is in this range so my take is that as long as we have more governance and more focus on explainability some of these discretions should go away from the system because penalizing is definitely a part of it but the force should be more towards the explainability I think Jayampi has made a point yeah I just want to give a certain viewpoint right I think it is a similar point that was sitting here before it's fairness is subjective right it's not how sharp the knife is is the person who is holding it that matters and it's a very tricky topic and it's it's hard to solve it's not like algorithm they can build and solve it and the multiple layers of takes that we've been trying to incorporate but they're still failing at it and it'll take a lot of time to build it but at the same time we haven't been able to even define what fairness means like huge philosophical questions that needs to be answered like similarly do we know what intelligence is oh yeah so just to mix fairness of regulation right very tricky topics and I don't mean to be an expert on it but you know there's there's different approaches to this problem is it the company's problem if it is a science problem it's a regulation problem but you do see okay we're all engineers who get a scientist and you know you want to solve something you need a simple answer and so I'm sorry gentlemen just here American Express right you're in the finance business you know in the States when you make a loan to somebody you're not allowed to discriminate that's it that the model is determined alone like whoever's working that model trust me they know this model should not discriminate and I was talking to a gentleman here working the aviation industry and this gentleman was sitting beside us also mentioned about in the aviation industry they have very strict requirements around software and I think you know heavy regulation I don't know if we'll ever be successful in this because you can't you know when I work in finance half the time three months later we couldn't explain our own models because we've forgotten how they work and so really like in the regulation side of things I think it really has to be said it's you know fairly transparent rules around what is fair and what's not fair now that's not going to be universal solution very heavily I really think at least to make to make some real real progress in this problem you have to identify areas of real impact and put some regulation against that and I know a lot of that's going on in India I know a lot of us going on around the world and so I do see progress there but a lot of work to be done. Any other thoughts questions here towards the end of this discussion? There's another way of solving the problem or another idea that I've heard of is like having make data people's commodity let people sell it change the whole dynamic rather than in exchange for a service you don't trade your data you get paid for it. There was an article floating around maybe a week and a half ago saying I my friend sold his face to Google you saw a bunch of you saw this right so for a $5 gift card not even cash a $5 gift card take a bunch of pictures of my face now how different that Google was just calling your face anyways now it's like I got five bucks for it is that really selling it over five bucks I mean what rights did you sign off the guy didn't even read the forms so yes but much more depth in that what do you depends what you're selling what values your face versus what values your social security number versus then what do they get to do if it once you sell it and how many people can you sell too? Mix that with transparency might help but yeah it's still hard to think. There was even one person who floated stocks on his name so basically people could own him own part of a man take decisions on his behalf he would run a poll it could be simple decisions like whether he should go for a jog or not it could even be more intimate decisions and interesting talking about inclusivity just want to highlight that this is the first you know women's participants. That is exactly why I came up. Thank you. I feel like I guess just as we're closing up I we've been thinking about this from the top down how we can handle things with regulations how we can impose different techniques on our data set on the algorithms improve the training data be sure on the question of what is fairness but as we all know as engineers when you're training an algorithm trying to make it not discriminate is not as simple as just like getting rid of gender or getting rid of past or getting rid of whatever it is that you're worried about it's about also paying attention to all the contaminating variables that go into that experience and go into that identity and I think when you don't have people both making the laws and making the algorithms across a diverse set of experiences there's these blind spots you just miss those elements and they're unknown unknown and so I just it's really great to see everyone here but I it's one thing I definitely think a lot about like how we can get more people from diverse backgrounds into the space and I didn't notice the full time that we didn't have one woman up here so yeah I just it's something I'm always really interested to talk about it's a good activity to make it happen I think that's a great point about the blind spot right it's not as simple as just removing certain data but just having that diversity in your team and outside as well so I'm listening to all the points that were discussed there I just felt like to throw out a question do we need to protect ourselves as a community you know something like Cambridge in Antarctica happening and affecting so many people do we really need to think about as a community how it affects the developers and the image that gets created or how people perceive us as developers here so just a thought to I think I'm not qualified enough to answer these kind of questions but just to just a thought that do we really have to protect how other developers don't impact our work offer all the discussion that I have you know I want to say something that if I think all of us network subscription right how many of you have network subscription please watch the great hack if you haven't till now it is the documentary on what Cambridge analytic I did and I think the best way to protect ourselves is being data aware we have to be aware of the data that we are sharing on social platform for example for today you know a lot of us have posted pictures that we are at ODSC we are at ODSC conference here and sometimes they also ask to share us our locations right so people know that where we are right now right and I have read something that right now thieves are also following in our activities on social platforms right to just go to your home and take all the stuff and go back because we are posting a photo on Facebook that we are on a very virtual with family enjoying and there is no one at home right so I know there are a lot of parents here make your children's aware that data is the most important entity that we have right now and we should not share this with you know freely right we don't pay Facebook anything because they are taking our data from our something that there should be a model you know that they should start paying us for our data right so please make everyone that data is the most important thing that we have and we should make people aware of that and that's the way to go forward. Just wanted to close on that last comment this guy I don't know the gentleman's name who came up here but I think so so far we've talked about one is top-down regulatory you know way of handling this I think you made a point about you know having diversity to ensure that we kind of aware I think that was kind of the third dimension in my opinion is that do we have some kind of fear pressure to ensure that other developers or other people building models are not doing things which as a community we kind of influence each other to make sure that people behave in ethical manner right so there's a community aspect I just wanted to kind of highlight that that was a third interesting dimension that that was brought to the table here. Yeah and I kind of touched upon this a little bit but how many people here are involved in the interview process you may not be a hiring manager but you get the interview people are coming on your team so you have to ask yourself do you do you raise these questions you ask people how do you address bias how do you think about it how do you how do you think through these problems I'm not saying the people you're not necessarily looking for direct answer because a lot of these problems are hard so that's a really good way to make sure that people in your team like the man said how do you protect yourself so to speak I'm not sure is that the right word but better way of phrasing is how do you make sure that your team you're coming where but it starts with a hiring process like just imagine if every company started doing that and you get go like pretty sure I'm you know especially given the turnover in the data science community you pretty much solve that problem in five years and the story you need to make sure people who are already in companies are qualified to make those decisions. So on this issue of fairness I think a lot of the concerns about privacy today well the idea that you need to have some privacy might have been taught to us in our schools in our English and social studies classes we might have read books like 1984 where there's this big brother who knows everything about your life and we like learn from an early age that things privacy breaches can be very dangerous things so I think to just increase fairness in AI or technology over time we should help to develop curriculums in our literature and social studies classes perhaps talking about the impact of tech in the 21st century talking about Cambridge Analytica in the social studies classes on and its impacts on society and also reading like dystopian books about tech companies gathering up all your data and causing bad things with that that would allow students from all different backgrounds to just go out into society and then encourage all society to be data aware and tech aware. I think another aspect of this is definitely not the time to think about early education or developing developers accountable but especially if I work for a non-profit we do machine learning for different government agencies and NGOs and we try to we work on poverty issues a lot of the time our clients can't pay us directly and so we have to get external funding and so I think another place for accountability is thinking critically about how you're engaging with funders. There's definitely been times when there's pressure to take a data set that is incomplete and use it to build a machine learning model that probably no matter what you do is going to be kind of biased because you just don't have the data to do something that is exactly what you want and so I think engaging with funders critically and really pushing for better data collection for better training data sets and filling in those gaps before you're at the point where you're trying to fix the algorithm itself is a really important area and there's often not a huge willingness to pay for that but I think some of that comes from a gap in the understanding about the real thing that you can get from not just fixing algorithms but fixing that to be needed. Yeah I would say it's kind of extending it's not just funders because even inside companies people have those pressures to kind of you know finish something get something out and people end up taking some of these shortcuts and I think having the peer pressure maybe not to do that might be a good way to stop that but yeah I mean how do you deal with pressure inside from the company and from funders is a hard problem I think. I just wanted to bring up about a basic human tendency of loss aversion that somehow doesn't apply to us because we don't think loss of privacy or the data trail that we leave around as any particularly huge loss so going back to the point that was made around you know educating or inculcating these aspects early in the life of every person that could help and somehow quantifying it monetarily or even otherwise that could potentially empower each person to realize that they're losing something when we saw the practice plan. So add some kind of financial value to your telephone number to your location to each of these things. Build an app that will show you how much money you've lost today. Okay I just wanted to add my perspective on two three items together. I'm trying not to be cynical but talking about regulations. We have had finance regulations for ages together we still have regulations happening we still have you know companies going down overnight right nobody is able to protect that but we do have a sense that there are some regulations which are helping us move forward in the right direction. It's not that it is actually helping us but we all do have a sense we are all being reassured that regulations are there to help us. Second regarding data privacy collection usage and all that whether you like it or not people who want the data are going to get the data. That is the reality of it right. You can't really help we can't go back to being capable we cannot there was a gentleman right who said teacher it's the important of data whether they like it or not I am sure people who are tracking me they are also tracking what some kids are doing whether we like it or not they do have the data right. I don't think it's going to be practically possible to get the ideal state that we are striving for does not mean that we should not strive for it and the irony is all the models that we are talking about everything is probabilistic nothing is deterministic so we also have to just base it on probability that overall probably our regulations at some point in time will start kicking in and that making the right thing for us. Do you want to end the panel on a cynical note? I said I tried not to be cynical. Let's do better. Let's do better. Yeah I agree with that let's do better. We're just going to have to know like we didn't talk much about it but as you know we're just at the very cusp of this journey to sound very hand wavy but if we're going to build trust in AI systems, eponymous systems it's not that we must we're just going to have to otherwise it's not going to be it's not going to be introduced in the society like if people don't trust it they just won't use it and that's the end line because you're seeing a big backlash now and that's backlash I believe unless we do some of the things we talked about here today it's only going to grow. I don't see it any way. And I think listen we can do great things that we put our minds to it and you've seen it in the last couple of years people paying much more attention to this problem if companies get behind this and if countries get behind this and individuals get behind this it can and will and must happen and I can see that happening and whole industries will be built around it so if you're a startup founder like I know a lot of startups from the US building AI companies around this whole all of these issues in different domains and it's a pretty hot field and there's a lot of money flowing into them we haven't touched upon AI and climate change like at ODSC we've been looking for the last couple of years like we want some good climate change talks and people a couple of years ago I'm not from mixed metaverse here we're skeptical about AI climate change talks and now it's becoming big business it's a real issue so I'm very hopeful there Let's hope that we all put our minds and hearts together to really tackle this problem and we believe good things will happen Cool I'm just a little sensitive we've kind of overrun the time thank you all for coming to the panel thank you for sticking around dinner is going to be served out so we can continue this conversation over dinner and have a fist fight about should we or should we not Cool thank you again