 So, how do we re-think what we think privacy is about that privacy is one thing that is super important and how we can take care of it. So, as you see you know more and more companies are deploying the ML and DL models, there are lot of models coming in production. The thing is a lot of data that is being used for this are crowd sourced and even after you know basic attempts to de-identify the personal details of people it is still present. So, what happens is basically high dimensional data with a lot of features is actually quite unique and high the number of features it is the easier it is to identify the person whose data it is because if you have a lot of features it will be easier to find out which person or which group it can be. So, if you see high dimensional and high resolution data is essentially unique. So, if you find that you know someone whose office is in London working in IIT department the data of joining is this, salary is some x, data of birth is this and more the dimension if you see it is easier for me to actually you know build down and find out which person this might be, with lower data sorry lower dimension or lower resolution if you are aggregating something it is more private because it is difficult to you know find out a person, but again it is less useful because the model accuracy will decrease because of this aggregation. So, where is privacy needed like medical records, genetic data, search history, if you think of it perhaps everywhere like privacy is important in each and every sector or use case that we talk about. So, we will talk about a few of the reaches that was there the biggest reaches. So, AOL for that matter in August 2006 for research purposes released the search data for a lot of you know customers and then what happened it was like it was an intent. What happened is though they did not identify the person in the report and there were a lot of numerical you know representation for each each person what happened is New York Times was actually able to locate an individual by cross referring all those details with the phone book listing. So, what is happening is there is one data set you think it is private because you have you know remove the few of the details and only considering your data set it is you know difficult to find out the person, but there may be other data set which in like used together can actually give a lot more information that you can even think of specially in case where you know a lot of companies use crowd source and open data. So, AOL's reach was one of the most known then then was Netflix. So, Netflix was aware of the privacy laws the there was no consent from the users that you know will be viewing your records and then what happens is so there was a challenge there was a Netflix challenge. So, you were given ratings by some users and you were predicting what will be there will be there rating for another movie for the recommendation system or whatever. So, the users were on a some users on the data set were identified using the data set from IMDB and again it was you know for some of the users it was exactly known that you know which user was that person in the challenge. So, that was another instance of it then we are thinking of you know what is the what are the possible ways to release then like what are the possible ways to release the data. So, one is you know you can perhaps use the trusted party use data to learn their classifier or general study or general statistics. So, you are giving your data your trusted party the second way can be you know you can only give up you know give out a general statistic. The third way is you release the class you know classifier which will learn on your own data and rather than releasing the data you are just releasing the data and now we will see if even these are secured or not. So, like if you think even you know rather than giving out raw data if you give this will just make it make things you know really private let us see that. So, in the first case like you know let us imagine for the illustration that Alice is one of the user Bob is you know the deep learning engineer maybe who is using Alice's data to train his model. And he was one party was trying to recover reconstructed data as provided by Alice which was you know which was shared by Bob. So, the first case was you know only allowed trust party use the data will learn classifier for general statistics. In this first case Bob may share a data set to allow others to train their model. So, again with Bob sharing it there is a possibility of them to you know again use it with several other data and then maybe finding it. So, if you think of it it is really not you know completely private it similar to what was there in the Netflix challenge as well. The second thing can be you know releasing only the general statistics, but so what can happen is Bob may now share only the neural network weights of a model that he himself trained on the data. He can find out if Alice data provided maybe is included in the database by looking at the weight put on a particular feature. So, if even if you are releasing just a weight they can look at the weights for the feature and you know if it is known to be characteristics to Alice. For example, consider Alice rate of movie that very few people have bought. Now, the weightage of that feature is significantly different with Alice in the data. So, yes again there are ways to find it is not really private. So, if you know one thing about Alice that you know she has bought this movie there is a lot you can find out about it. The third way may be is you know only the classifier learn from the data is released. So, this is perhaps one of the most consummative of these three ways is Bob may you know just only allow third point is query an API that provides an output of a model that he has trained on the data himself. But even this it turns out you know all these three ways of sharing data is not really you know very very private and still has a potential to expose Alice's data. So, that is what brings us to you know our concept of technology called differential privacy that we will talk about. So, once again when we look at privacy we need to familiarize ourselves with the fact that this is our holy privacy trinity and you cannot have all three. You can add the max choose two. So, you can either you cannot have privacy a good data size and accuracy together because sometimes we have privacy expectations saying that you know I wanted to be private but I wanted to be private for free and that is wrong you cannot have that. When you do a privatization it requires rooming at least a small amount of information and the second thing that we say is that oh you know I could probably have absolute privacy again that is a myth you cannot have absolute privacy and if you do have absolute privacy you would also be in able to distinguish between two different data points. But the reasonable privacy expectations that we have is that our privacy model should have a tuning that I can choose how much I can have a balance between accuracy and privacy loss. It should be plausible deny it can be plausible you can have plausible deniability. Means the presence of a data point in the database should not be able to be a certain means you cannot say that this particular person is 100% present in the database or not and you can prevent targeted attacks these are reasonable privacy expectations. So when we talk about what is differential privacy what the person who came up with this concept said that you will not be affected if your data is used in a particular survey or any data collection mechanism that is the main I that is the core principle that he wanted behind differential privacy. We can go through the textbook definition but you know it is just a lot of really difficult work. So hypothetically if I take 2 and X is trained on the full training corpus whereas Y has the full training corpus excluding one data point. In an ideal situation what would happen is X and Y would be the same like you would not have that much influence of a single data point that is what you would think. However the reality is not so nice we do not have an ideal situation and the more that single data point had the more these two differ and the more there is a chance for a privacy leakage. So if a single entry does not change the result at all and if when I say a single entry I mean every single entry then it is 100% differential privacy of course that would make it pretty useless no offense. Let us go back to Bob Alice and Eve. So what Bob does is Bob says that I will introduce Alice's data and I will introduce some sort of noise. Now when Eve tries to reconstruct Eve does not know if that is Alice's data or if it is noise. If we go back to the Netflix challenge which is like a baseline example here Eve would not know if that particular rating is the actual rating that Alice would have given or if there is some noise that is introduced by Bob. But when you do this that you would say that oh this affects the accuracy and that is the real subtle art of differential privacy is that you inject noise in a way that you are able to compensate for it. So this is like the formal definition of differential privacy. I thought once you do a talk about differential privacy and you have that formula somewhere I will just roughly touch upon it. So if you have an input space X and an output space Y our privacy parameter epsilon is what is the main differential privacy parameter and if epsilon is 0 it means that you have perfect privacy. Trust me that is not a good thing you also have a pretty not a very useful data. So epsilon 0 is perfect privacy and epsilon is unconstrained you could have like a really really not private data. Another thing that we also should talk about is approximate differential privacy where there is an additional parameter delta. And this variant was introduced by D-Walk where basically what it means to say is delta is the possibility with which privacy being broken. Finding out that it is Alice's data is delta and you preferably want it to be less than 1 by n where n is your input space. This is the main important part that we need to talk about is where do you insert noise in ML or GL model. So the places where your noise could be injected is the feature data set layer or you could injected during the model training as part of back propagation STD or you could inject noise at the activation output of the model layer. Now when you inject noise it has to be done intelligently to balance the considerations for algorithm performance and the calibration of noise for properties. So basically this is the subtle art of differential privacy implementation is finding a balance between privacy and utility. So this is an interesting concept that is mostly still in academic research. We have generative adversarial network as most of you might know. What they say is that why do you even want a data set that has private has any information that matters. Let's create a synthetic replacement of the data set wire again which has characteristic features that are similar to the underlying data but no particular data that can be personally identifiable. You create another complete data set that has the same features as your current data set but it does not have that personally identifiable network. So basically differential privacy acts like a privacy barrier between your private data set to release to the public. Now why is differential privacy so useful for deep learning? It is major properties that differential privacy has which is some possibility and that enables you to have modular design, group privacy which ensures you have graceful degradation and robustness to auxiliary information. When you do want to design a differentially private additive noise you need to follow these three steps which is first you approximate the functionality. Then you choose the parameters of your additive noise and you perform privacy analysis. The third one is very important. A quick catch up on TensorFlow's differentially private model and we all love to use TensorFlow. TensorFlow is our favorite. Those who are fans of other know TensorFlow is my favorite. So I very recently found out that TensorFlow has a differentially private model that performs really well if you take into consideration everything considered and so I just wanted to touch upon what the differentially private model of TensorFlow does and the main idea was that they train a deep network using differentially private sdd and then they use the moment account to track the privacy loss. In addition to that we also had gradient flipping, mini batching, data augmentation and a little algorithm overview of how that actually worked and I just wanted to share the results of that because most of you are developers and if you are thinking about deploying TensorFlow in your differentially private TensorFlow in your next project or next work item these results do matter. So there were results that were released as part of your paper on two major standard data sets. One is MNIST, if you are not aware it is a data set of handwritten numbers and the results are pretty good. When there is a large amount of noise you can see that there is quite a little bit more of a difference between the differentially private TensorFlow. Pretty good. Like to the experiment they used a 60 dimension PCA projection 1,000 hidden units and it was trained on a lot size of 600 and the clipping threshold was 4. And as you can see the noise level and okay I am running out of time yeah so if you are not aware the CRFR10 database is a database of code they have basically 60,000 32 by 32 color images in 10 classes. So the training and test error came close to the baseline of non private deep learning methods and to get moderate loss in performance epsilon and delta non negligible and so you know if you say oh I want it too private and you increase the epsilon too much it will be trouble. When is differential privacy practical? It is best suited for population level statistics and when something does not depend strongly on a particular individual and large sample sizes is when you should use differential privacy if you have a small sample size the importance of a single data point would be more and then it is not really advisable to use differential privacy it is a small very interesting thing that I found out but I think the real world applications some of the real world applications of differential privacy is that Apple has used it in iOS 10 Google has used it for sharing historical traffic statistic Google has again used it for their learning statistic and US thank you thank you so much for your time and attention