 Hi everyone. Today's talk is about machine learning privacy. The title of the talk is Secrets Our Lives. Sharing is caring, privacy is theft. My name is Nahid Farhadi and I am a software developer in Capital One. I will be giving this talk with my colleague Vincent Pham. The outline of our talk is as below. We will give an introduction on importance of privacy and we will talk about privacy attacks. We propose some defense techniques and finally we will show a demo of an attack as well as defense techniques. Machine learning needs data. So what if this data has sensitive information like medical data or financial data? Even if we trust the algorithms that are generating the model or model developers, sometimes just using the statistics by looking at the output of a model, we can find out some information about the input of the model. This type of thread 2ML can be black box or white box, meaning that the adversary doesn't necessarily need to have any access to the model specifics. So what is really privacy? Privacy means that if we are using some data from our users for training a model, but just by looking at the output of the model, no one should be able to get information on the input of the model. There is a tradeoff between privacy and utility. If we have too much privacy, then it will be very difficult to use the model. If we don't have much privacy, then it will be useful, but then we will endanger our customers. PPMO is there to define or find the optimized point between utility and privacy. How can we sacrifice less by getting enough privacy? My colleague will be talking about privacy attacks next. In terms of privacy attack, there are two main categories. One is the classical, where the private data is in the training, where it's in the raw set, and there are the ML-enabled one, which I will be focusing more on. In the next slide, you can see that there are several attack surfaces that attackers can utilize to perform a privacy attack. The first is the physical domain, where in terms of network contribution detection, it could be attack network traffic. Then there is the digital representation, which is the TCP dump. There is also the machine learning model, which is just the model itself, where you have an input and do a prediction on the output. In terms of this, it is the attack property, and then you have the physical domain, which is the shutdown infrastructure. For this talk, we will focus on the machine learning model. In the next slide, we will see our first type of attack, which is the linkage attack. With this attack in the early 90s, we found that she was able to identify the identification of the governor of Massachusetts by linking his health records to voter information just by using three variables, his gender, date of birth, and the zip code. In the next slide, she was able to identify that using these three variables alone, she was able to uniquely identify 87% of the population in the U.S. using the census, the 99 census. In 2000, Philip Gold identified that using these three records, he was able to identify only 64% of the population. This could be a result of people urbanizing more, moving from the rural areas to the cities. But this provides an important point, where using just very simple features, attackers can identify back to the original source. In the next slide, we look at another popular linkage attack. In 2006, Netflix released a public data set, a competition where they're offered a prize to improve their recommendation. In their FAQs, they noted that there is no private information in their data set itself, but researchers have found that if they were able to link the Netflix accounts back to the IMDb, just by using the name of the person on IMDb, their public dates and public ratings, they identified personable information back to Netflix. You might be wondering why this might be damaging, since the people tend to put their public information on IMDb. This could be because they might be biasing or selecting a set of records to disclose on IMDb, but not on Netflix recommendation. For example, they were able to identify that for a particular person. They were able to identify just based on his Netflix record that he might have voting preference from his voting record on power and terror, Norm Chomsky and our Times in period 9-11. They might be able to infer on his political preference based on the Jesus of Nazareth and Gospel of John ratings, and also his eating habit based on his recommendations on supersizing. In the next slide, we look at a new type of tech called the reconstruction attack. This is more prone to models that tend to keep the training information in the structure itself, such as nearest neighbor classifier and current base SVM. But this is also prone to neural networks as well. As we could see in the next slide, where in production environments, researchers and even implementers tend to reuse neural network models just because of something called style transfer or style transfer where they're able to utilize one model and create a new prediction from that same model using the inputs and architecture of it. And we construct back to the original input. In the next slide, we also see a similar type of tech called model and vision tech, where in this case, a feature such as an image is sent into a neural network model and produces a set of probabilities. And then the researcher can use these probabilities and reconstruct an image back to the real thing. For example, in the right side of the slide, you can see that in the left image, there's this reconstructed image. And then the right is the original image. And they're very similar to each other. In the next slide, we also have a membership inference attack where an adversary can generate multiple shadow network models that resembles the production environment, produce a probabilities from these models, create a new model called the attack model to feed in these probabilities, and learn what observations are trained by the production model and what are not trained by the production model, identify what are the members and what are not the members. In the next slide, we also have a core attack. This is more prone to models such as LSTN or models that learn from a time series of data points. For this example, you might have a Google search that you start typing in social security and it does an autocomplete. And one of the autocomplete could be some sort of sense of information, such as the phone number or the social security of a person. This is more prone to maybe like if you're over-memorizing from the neural network model itself, where the records might appear like these five times. In the next slide, we have a summary of a different privacy attack that we talked about earlier on. You can take a pause here and just look through the different, compare the different approaches and look at the examples, but we can move on if you're ready. Next slide. All right. So now we're really talking about methods and techniques to preserve privacy. To preserve privacy, we have four main points to protect. The training data and input and output of the model shouldn't be visible to anyone who is not authorized to visit them. And finally, the model privacy, which makes sure that the model cannot be stolen by any malicious party. Two main categories to preserve privacy are secure computation and privacy preservation. In case of secure computation, we transform or distribute our computations in a way that is not readable by any unauthorized person. And in terms of privacy preservation techniques, we use injection, noise injection or masking and hiding techniques in order to preserve privacy. The first method of privacy preservation is homomorphic encryption. Homomorphic encryption is a type of encryption where you are doing the same computations and a third party can do your computations for you, but not on the original form of the data, on an encrypted form of the data. This means that if you use a third party company for your ML computations, you can totally give your data to them in a transformed form, which is using homomorphic encryption, and they can do the computations for you and give you back the results. And then you can decrypt the data and get the plaintext or ciphertext as you want. The second method is secure multiparty computation. In case of this technique, we have multiple parties who all contribute into building an ML model or a function or to compute a value for a function. But the thing is that the input is distributed between all of these parties. So as you see, part of input A1 is here, A2 is here, A3 is here and so on. So this way, if an adversary wants to attack the output, they need to be able to hack into all of the parties in order to make sense of the data. Otherwise, getting information from one of the parties is meaningless. Another technique is federated machine learning. Federated machine learning is very similar to the previous method. However, the difference is that we have multiple sets of users, and these users all contribute into building a classifier. The way that we handle the creation of the output is that we assign some sort of weights and locally update the average of classifiers by assigning weights to each of the trained models. The most important technique to preserve privacy is differential privacy. Basically, differential privacy means that it's a noise injection technique so that the output of your model shouldn't reveal any information about the input of the model. Whether your model is trained using the data of this person or the data of this person shouldn't be revealed in the outcome of your algorithm. It can be non-intractive or interactive. For example, in case of non-intractive, we pre-compute the stats, the amount of noise, the amount of relationship between the input features, and then inject noise to that. In case of interactive, the amount of noise is injected based on users' requests. So, what does really differential privacy mean? It means that if a person changed their information from X i to X prime i, then the probability of their output should not change by that much. So, what does it mean again? It means that as a person, if one of my features is changing and as a person who is contributing into training a model, that change shouldn't cause a huge change in the output. Because if that causes that huge change, it will reveal some information about this instance or observance as an input. For example, if I change the color of my hair from brown to green, and that has a huge effect on the output of the model, then just by observing that change, an adversary will know that I changed the color of my hair. So, using differential privacy, we want to make that change as minimal as possible. And how do we do that? By computing what is the effect of that change and making that as a noise input into our model. So, looking again at the differential privacy, you see here that we have two datasets. We call these neighbor datasets, meaning that all of the features in both datasets are actually equal except that one feature. In here it is x3 and x3 prime. We want to make sure that when we change the input data from x3 to x3 prime, the output that is generated by the model is changing minimally. So, as you see, the probability of generating a specific output in the first case to the second case is minimal or epsilon. The next method is called PATTE, Private Aggregation of Teacher Ensembles. In this case, we have several classifiers that use some data in order, use some features in order to train a model. Then we have two sets of voters. It's like an n-version programming type of technique. These voters choose the majority voting generated by all of these classifiers. If we have equal amount of votes, for example, in this case, we have two cancers and two healthy results, then we will calculate a Gaussian noise to inject to the voter's data. Basically, we again go back into our training data and we take a look and see that how much noise does each classifier needs to be injected into. Then, again, we look at, we inject that noise and we get the results off of this chart based on the results. So, based on the amount of noise injected, the model might generate the value cancer or might generate the value healthy. The important thing is that whatever the output is, for example, in this case, it is cancer, the adversary cannot correlate it back to the data that is trained in order to get the value of cancer here. Because even if we don't have, we have the value healthy here by the noise injection, these might also generate the value cancer. So, we don't really know is GEN-SMIT the cause of generating the result cancer or if someone similar to GEN-SMIT in this training data set is contributing into that result. Here is the comparison of the defense methods in terms of what they're emphasizing on, what are they trying to protect, whether it is the data owner or the model protection and their use cases in different applications. Most importantly, most of these methods are really effective when they're used in combination with each other, one from each category. For example, homomorphic encryption and differential privacy, or S&P and differential privacy, PATA and federated lengthening process and homomorphic encryption. Here are the practical methods for privacy preservation. They are very easy to use. There are already packages that are generated for that, like TensorFlow or PyTorch or IBM privacy package. We are going to show an example of how to use one of these packages in a real or sample. In this demo, we're going to show how you're going to implement a solution from an adversary point of view and then from a defense point of view, how you can easily apply a differential private method into your model. So, here in the references, you will see the data that we use, which is the source on Kaggle of the purchase dataset. This contains information or that you can use to create labels such as a banner classifier. In some cases, it could be like file or if it's fraudulent or not, or it could be abusive or not. Then you could also create a set of multi-class labels itself, such as the class for a customer or it could be category of a merchant or any other multi-class that you can think of relevant to your use case. You can see that we're also going to show you the TensorFlow privacy method. First, we're going to look at how a shallow network is created. It's really easy. First, you create the set of shallow networks, then you output the predictions from the shallow networks, feed that into a tag network, and then train a target network that is similar to what you think the production environment would look like, and then see whether the prediction from that target network is predicted as an in-member or out-number from the tag network. If we look at the shallow network, all you have to do is create a very simple for loop, sort of you go down. You can see that all you have to do is create n number of shallow networks. They could be similar to each other or they could be of different structure, but for our use case, we're going to create one type of TensorFlow model, a very simple one with an initial density of 64 nodes and then dropout and then another density of 24 nodes, and then finally the softmax for a prediction of the multi-class or binary class. Each of these n shallow networks will have the same networks, and it will produce a prediction of how likely it is to be in a certain class or not. If you look at the tag network, this is where then you're feeding the predictions of the shallow networks, we're then treating a very fine model to distinguish the threshold of whether a particular observation is within a network or not. So this is very simple in your training samples from the shallow network. You're going to identify if it came from that particular shallow network or not, and that would be pretty much your label. If we go to the target DP, this will reveal how easy it is to implement a differential private method for your defense mechanism if you already have a deep learning TensorFlow model in production. So all you really have to do is import the TensorFlow Privacy Library and then update your optimizer. So for in cell 11, you can see that the model is the same, but in cell 12, you can see that the only thing that changes is the optimizer, which is a DP gradient per sense caused in optimizer with four inputs that you would have to find out to norm clip, noise multiplier, non-microbattress, and learning rate. And then you have your differential private method. And then if we go back to the first one, here we show results of the different number of shadow networks with a different number of classes, 10 classes, one inch classes, and the binary two classes, and whether it was differential private or not. And you can see the different types of accuracy, test accuracy, and then the recall and precision rate as well. This is just a summary if you're interested and you could pause here and look at it for further details. Okay, so this is the end of our presentation. Thank you so much for your time. If you have questions, please reach out to our emails included in the slide deck.