 Good morning everyone and welcome to the Machine Learning Security Vision Competition 2020. My name is Zoltan Balazs and this is a project together with Hiram Anderson. We are going to talk about machine learning detection bypasses and in the past you probably have seen some researches where people modified existing images and the goal was to bypass machine learning classifiers. So for example for a human viewer the new image looks like the original one, but for a machine learning classifier this looks like something different like it will believe that this looks like an ostrich. In this talk we are going to present machine learning bypasses when it comes to malicious software and there have been some interesting researches in the past regarding this topic as well. For example last year there was a new research published where people extracted strings from a known game executable, appended this to a known malware and it was able to bypass production machine learning model. I also did some research in the past and like four years ago you were able to bypass some machine learning models just by packing a sample with UPX. In order to advance the field of offensive and defensive machine learning based malware detection last year we created a challenge where you had to download 50 working malware samples, you had to download three machine learning models with it weights, modify the malware samples to evade detection by all models and if you were lucky and you had the most points you were able to win this nice GPU card. In total 70 people registered for this competition and at least 11 people were able to bypass at least one machine learning model. Congratulations to the winner William Flashman and I highly recommend you to check out his blog post on how he won this competition. There were some other write-ups and papers as well. I do recommend you to check out those as well. You can see one from Jakub and one from Fabrizio on the following links. When it comes to win this competition multiple approaches were used. Some people started with a simple packer like the one I mentioned but unfortunately some of the samples are already packed in a way and this means that if you test with UPX or something similar it will not work anymore. Another great approach was to add new sections to the executable. For example you can extract the end user license agreement resource from Microsoft files and add it to the malware samples multiple times. This approach was really good at bypassing the detection for the ML models but unfortunately again this broke some of the malware binaries. Fun fact if you just simply add sections to a malware sample you might be able to bypass some antivirus detection because for performance reasons some AV engines checked the number of sections before they evaluate the rest of the rules. At the end the winning strategy was to just append random data to the end of the executable. This is called as an overlay. Even though this is a very simple strategy it worked during last year's competition and this is also an easy way to bypass if the sample has any kind of self-protection for example. Just by increasing the size of the sample again you might be able to bypass some antivirus engines again they can have a size in their rules. And one important thing as you can see on the top right image that's visual representation of a malware and I just appended some random strings to the end of this sample and if you look at the green visual you can clearly see how this changed the visual representation of the sample. For us there were some key takeaways from last year's competition. For example some of the machine learning models are way too academic but not very effective in practice. Turned out it's not just us but everybody thinks that the DF tool is awesome. This is a Python package you can use to modify binaries. And as it is the case with malware it is always tricky to deal with them. For example some of the samples do not reproduce the same indicators of compromise over time. This can be because for example the command and control server is down and dealing with packed and protected samples can be hard sometimes. I also checked the SSD patches of some of the samples and it was interesting to see that whenever people added repeating patterns to the sample for example the same section or they added the same overlay over and over again to the sample then it created the repeating pattern in the SSD patch as well which can be used for detecting a sample which uses machine learning evasion for example. This year we created the defender and the attacker challenge. In the defender challenge you had to create your own machine learning model and submit this to the composition in a docker format and in the attacker challenge now the machine learning models are not available for you so this is now a black box challenge. And if you win this competition either the defender or the attacker challenge you can win some Azure credits for your machine learning research plans. The defensive track is already over. We received two submissions that passed the minimum requirements and the offensive track already started so I highly recommend you to go to our website Anansak.io and check out what the competition is. In this year we have used the following malware families. If you go to our website, review the terms of service and then you can download the 50 provided malware samples and after that it is your time to modify the samples in order to evade the detection and new to this year you can use an API to check your samples or submit your samples. I also recommend you that you verify that the malware functionality remains the same in your local windows box. Then when you uploaded the zip files or you can just upload the partial zip files meaning that you only submit some of the samples and not all of them. You can receive one point for each bypassed machine learning model which means that for every sample you can get up to three points and as usual highest score wins. The details about this will be provided by Hyrum and in order to claim your price you have to publish your solution. Please note that you have to keep the file names as it were in the original zip file. This helps us to track which file you modified originally. Also provide some additional tips and tricks you might use in this competition. Some of them may not make sense but you can modify an executable in a lot of different ways. You can add or remove signatures, change section names, properties, modify the import or export tables, create TLS callbacks, change the PA header, fix or change the checksums, add, modify or remove the version information, create new entry points or just change some code or data in it. Still it's not allowed to create droppers or self-extracting archive because this will kind of defeat the purpose of the whole competition and this year keep in mind that multiple registration is against the rules and it will result in immediate disqualification. Please do join our Slack channel where you can discuss everything with us and you can also discuss your progress with the other participants of this competition. Just a side note, the whole front end was created in Python Flask admin and we are using Cloudflare and GenXIg Unicorn for scalability and performance reasons. There are some backend scripts running with Python scheduled by Chrome and as it was the case last year we still used the VMRay sandbox to evaluate the samples. As mentioned we already have an API. So if you want to check your sample against all the machine learning models or just against one machine learning model you can use the API just to do that and also you can use the API to get the results and if you are satisfied with bypassing the machine learning models you can upload your zip files and query the zip status and the sample status as well with the API keys. This is all I wanted to share with you guys but please welcome Hyron who will present you some other tips and tricks you can use to win this competition. Thank you. I'm going to describe to you the example solution in the machine learning security evasion competition's attacker challenge that has just begun. The models that you'll be attacking this year have been submitted by participants of the previous round in the defender challenge. Two of the models from the previous round have qualified to be included in this round. In addition we have hosted our own model for you to attack. That model is trained on the Ember dataset and includes some basic capability to detect adversarial examples. The source code and model weights for this defended Ember model are provided on the competition's GitHub site. However, the remaining models are to you complete black boxes where you only get to observe the hard label predictions that is a zero or a one for an output that you provide to the machine learner models. The final leaderboard ranking will be set by the following rank-ordered criteria. First the total number of evasions with one point for each of the three ML models times 50 malware samples meaning that the maximum score is 150. Remember though that each evasive sample must reproduce its original functionality in a sandbox in order to be awarded a point. Functionality is verified only when you upload a zip file containing your candidate malware samples. It will not be verified when you merely query the machine learning models through the API. In the event of a tie for point number one contestants will be ranked by the number of model queries used through the API. And lastly the time stamp of your final zip upload would break any subsequent tie. More than likely though this we won't get to point number three so you should feel incentivized to continue competing right up until the competition deadline. Even if you see a perfect score on the leaderboard because you might achieve that same perfect score but do it more efficiently. So as a contestant you can choose any strategy you'd like to compete but to demonstrate one possible strategy we have released some example code on the competition's GitHub site. You can find more information about the nitty-gritty details of this approach on the website as essentially as using a discrete optimization technique over a space of functionality preserving file modifications. However, the general strategy might be more useful for you to adopt. The strategy consists of is really simple consists of doing a bunch of bulk work using an algorithm in part A and then kind of batting cleanup for manual manipulation of malware samples in part B. And I'm going to be describing and demoing the code for part A today. In it because we'd like to be efficient in the number of queries against the hosted machine learning models will actually break this attack into two parts. An offline attack where we use the defended Ember model for which we have code to kind of work out our strategy and generate initial malware samples. We hope that those seeds might evade some of the online models that are hosted. And then in the online attack will take those initial seeds and the algorithm will further optimize and discover additional file modifications required to evade the online hosted models. Some tricks that we're using here are include label smoothing where we're converting the hard label outputs into a soft score by averaging four things, the three hard label outputs from the hosted machine learning models as well as a local score from a local machine learning Ember model that will be used as a heuristic to kind of guide the optimization process. So as I demo this code, I want you to please be aware that this code writes malware to disk. So please do run this code only using a Linux VM. To begin, we initialize the attack by analyzing a connect collection of benign files. This init sub command extracts elements of these benign files that will be later injected into the malware. To launch our offline attack, we'll run a local copy of the Ember model in the top window then in the bottom window, we'll use the run command passing in malware samples that we downloaded after registering on the website. The tool will then write successful evasion attempts to pass one slash success that we've specified in the command line and failed attempts to pass two slash failure folder. And also included in each output directory will be the history of file modifications that will be useful if we'd like to pick up to resume a failed attempt. So to demonstrate that in a second pass of the offline attack, we'll start with the pass one failures and iterate on the optimization approach against storing successes and failures to a pass two directory. So after doing that a number of times and having collected a bunch of candidate samples offline that evade the local defended Ember model, we'll then use those candidates as seeds for an online attack which now counts against our API query usage. To do an online attack, simply use this tool with a dash dash online flag and the optimization will then continue trying to find file modifications that will bypass all three of the hosted models. Of course, you want to do perhaps as many iterations as necessary and the online version of this attack, but after you've done so in a final pass of the online attack, you can now collect the successful samples into a zip file that you would then validate in a Windows 10 virtual machine and then upload to the website for validation and leaderboard scoring. I want to point out that since there is a chance that by running this code file modifications might break some of the samples, you should always run these samples in a Windows 10 box before uploading to the competition website. Also note that zip file uploads contribute against your API query count. So it is to your benefit to double check your work and make sure that any files you upload are functional. So you don't have to redo that work and upload again. As a final note, kind of tricky that since the host of models might be actually changing state and learning from the queries that you and others are giving them, there's a possibility that a evasive variant sample that you discovered along the way may no longer evade a model by the time that you upload your zip file. So I don't know that will be the case, but please be aware that that is a possibility. So with that good luck on the competition, visit the website at mlsec.io. The competition will run for over six weeks and those who are ranked first and second on the leaderboard will win our grand and first prizes, respectively, so long as they publish their solution. And with that, I'd like to thank our sponsors, Microsoft and Kujo AI with partners, MRG FITS and the MRA.