先到了我們開始hi everyone welcome to this seminar我今天給大家帶來today i'm going to talk aboutnew harbor update images by digging out and automatically fixing the vulnerabilities這個話題其實是源於這些事情從社群網站開始的這個題目很多社群網站的使用者他們喜歡立刻在帳戶上把帳戶的帳戶搜集到一些截圖從帳戶上那些帳戶會有一些結果會有一些問題如何我去解決我的帳戶還是如何去更新我的帳戶I get a result and today I'm going to explain that this speech will be delivered by me and my colleague ruling.He is the Dao cloud develop architect and I am also from VM.Well, I'm the harbor maintainer.This was actually ruling's idea and he would like to make some contribution to answer the questions from communities.For C, we'd like to talk about some of the background knowledges.For C, what is CBE?CBE is common vulnerability exposure,which means this is public disclosed to cyber security vulnerabilities and exposures.It can be seen as a dictionary and all of the cyber security vulnerabilities and exposures are publicly disclosed.Whenever there is a new vulnerabilities or exposure got capturedand that vulnerability or exposure will get into CBEand once it enter the CBE dictionary,it will provide you with your IDand you can see the format of CBE.It has a CBE with the time,with the year when the vulnerability was identified.This is a four-digit serial number.I will explain later.NVD is a national vulnerability database.It is a database based on CBE.And this is actually an expansion of CBE.It provides enhanced information for each entry.Let's look at this graph.This is the total CBE entries over the past few years.You can see especially recently since 2017the vulnerabilities identified each year was already beyond 10,000.Before 2017,each year we only identified 7,000 vulnerabilities.However,after 2017,we identified 10,000 vulnerabilities each yearwhich means every 30 secondsyou might be able to identify a new vulnerabilitywhich means after this session,you might be called back to your officeto restore the vulnerability,a new vulnerability.I'm not sure whether you've attend this morning's sessionthe keynote speech I delivered.I talked about a Linux,one of the Linux master class talked aboutLinux latest RC4 delays by security issueand then also another Linux mastertalked that since 2016 to 2018there are more than 1,000 CBEs assigned to kernel.You can see that security is of great significance currently.Let's talk about image security.The biggest advantage of image is immutabilityand that ensures its consistencywhich means from testing developmentto distribution to the clientI can ensure the consistency of the imagevia binary.However,this kind of immutabilityalso has disadvantagesif the original image has some vulnerabilitiesyou can see from this diagramthat all of the users around the globewill be exposed to the vulnerabilitytherefore,your development costwill be really highbecause you have torestore all of the vulnerabilitiesacross the world by each usersnow that security is such an important issuewhat can we doI'm sure you all traveled with subway beforeand you can see the security gatein the convention centerbefore you enteryou can see that they didn't actually unpack your luggagehowever,they can identifyor scan any illegal stuffsjust by scanning your luggagehere,image securitycan also use the same methodyou can also have static scanof the imageto see whether the image has some cvethis diagram shows harbourone of the scan result from harbouryou can see after scanningon aimage,you can seeall of the informationsassociated with this scanof this imageand then finally you will also havea resultto see how vulnerable it isor how risky is the vulnerability掃描一個具體的一個靜態的靜態now I'd like to talk about topologyof the static scanfirstly the harbour will trigger a scan taskand then the jobwill be dispatched to job serviceand then job servicewill do the job scanwhich is static scanuse a tool called flareclare is an open source static scan tooland then in harbour jobfirstly harbour trigger a scanand thenrager will get theall of the layer informationof the imageyou knowthe information includes layerand manifestso firstly you need to put all of the layersand thenthe layers will be posted to clareand based on the CVE sourcethe data gather from CVE sourceclare will conduct the scanclare will be connected to CVE databaseand these database come from four sourceswhich aredebian woven to redhack and oracleand these sources are updated realtimebecause CVEwill be growing each dayeveryday we will identify new CVEtherefore the sources are updated realtimeeverydayjob will realize this circleand push all of the layer from harbourto clare for scanand then harbour willget the result from clareand then summarize the resultand then you will know how manywas the priority of each CVEand then what kind of resultit is for each image关于扫描in terms of scanharbour also has another featurewhich is policy controlnow that we've already scannedand get a scan resulthow do we utilize the scan resultin harbourin harbour codewe add someinsectorwhen dockerinitiate pull requestwhen the request haven't enter registerwe will capture the requestand then if I need to checkand I need to checkwhether there are some high priority vulnerabilityin itand I also have set a ruleif there is high priority vulnerabilitythen the rule will not be appliedthat means the requestwill be rejectedthat meansno one can useimage that has highpriority vulnerabilityyou have to restore the vulnerability firstand that will ensure thatthe docker imagedelivered to the end userdoes not contain any high priority vulnerabilityand for image scanwhat does harbour do nextfirstly webhookeverytime I scanan image successfullyI will post the resultto the webhookand then you can getall of the information of the imageand then secondweblistwhitelistso you can set up your ownwhitelistwhich means these are the vulnerabilitythat your image doesn't carefor example there are somesome of the images areonly for testand the test imagesdoesn't really care so muchabout the vulnerabilitiesthe priority are not so highotherwisethe rest of the imagesshould have policy checkand thenif harbour wants toexpand third party scannercurrently harbour only support clearand now wewould like topartner with anchorcurrently anchor engineerare partnering with harbourandwant to establish connectionwith anchorso that we can provideplugable server for end usersand the end userswill have more optionsin the third party scannerso you can haveboth player, anchor, driveror micro scanners scan resulti know some of the usersuse their ownscan mechanismso if we have already definedthe interfacethe user can also usetheir own scannerto connect with the interfaceand then usetheir own scanner resultnext ruling is going to talk abouthow can we resolvethe vulnerability once we identify somethank you for attending this seminarpreviously we talked aboutafter we identify the vulnerabilityhow can we restore the vulnerabilityand in the settingwe identify the vulnerabilityif the vulnerability is highlevel or high priorityvulnerability thenthe system willstop the user fromdownloading the imagehowever the businessmight continuehow do we restore the vulnerabilityto ensure theconsistence of the businessthis was ourprevious speedandwe basicallystill use HABAon its clear auto scanning mechanismand then wewe wrote an externalprogram tocheckthe image of the vulnerabilityas well as the priorityof the vulnerabilityit also includesthenumberand label of thevulnerabilityand to see whether it hasany restoredsoftware packageif we do have itthen we willredefine the imagethis isvery simpleprocess firstly you haveautoscan andthen the level of the vulnerabilityautorerestoredand the confirmationof the scanthat will meanthat the vulnerabilityhas been restoredclear isa open sourcevulnerability analysistool by CoilOSand the databasically come from CVEand some of the other databasesince 1.2 versionin 2017HABA started tosupport theembedded queer to havevulnerability scanand in the current versionit supports all kindsof strategiesfor example if the vulnerabilityis high prioritythen we can stop the userfrom downloading the imagethe level of the vulnerabilityincludessevere, medium, lowandignorablesevere means thatwe don't recommend you to continueusing it in your production environmenthowever you can still use itin the test roomin the QA environmentand for the medium levelyou canwait and seeif you are usingsome basic infrastructurethen you need toreally attach great importanceto the vulnerabilityand think about how to deal with itbefore restore the vulnerabilityyou need to have some preparationfor example currently HABA alreadysupport the constructionof the historyfor example when you construct the imageyou can use the docker fileorderand you can use this historyin the futureand then second you think abouthow to determinethe basic image systemand that willbe very relevantto what kind of orderyou need to use in order to restore itfor example you mightuse some young updateand if it is apithen you might need to use apkor upgradeso currentlywe still need to use docker filewe need to use itto determinewhat kind of basicimage it isand what kind of part should be updatedin practicewe also need to determinewhether the imageis allowed to updatesome of the packagewill have the requirementon the versionif you update ityou might also update the whole versionand that willnot be souser friendlywhich means we need todetermine whether the imageis allowed to updateand then also the vulnerability componenthas already gotit's restoredversionand then later you alsoneed to call docker buildand then repushwe need to captureall of the imagesfilter outthe imagesthat hashigh level vulnerabilitiesorthe level is beyondyour settingthen you need to deal withthose vulnerabilitiescheck the informationof the vulnerabilitydeterminewhether it has already gotsome restored versionand then you need toenrate a reportto seewhat kind of vulnerabilitieshas already been restoredso that we canidentifythe version of this imageand then we shouldthink aboutwhether we just do the systemupdating or the third partysomewhere packagesupdating so we must thinkthat the systemupdating joband last but not leastwe mustcall the APIto checkthis recoveredimage to see whether it isrecovered or notif there isother vulnerabilitieswe will dothis scanning within this cycleI mean keep scanninghubwe use this systemone tocall this hub APIto finish itand this program has been delayedfor almost one yearand believe that in the futurewe willreleasean externalAPI port versionwe canmake itincluded into the hublet's saydisk taskor some triggering systembut now we have some dependencieslet's saythis fundamental or basicimage dependencyproblemsso that's allfor my presentationthank you any questionsfor the image recoveryfor thoseimages that have been pushedhow do you deal with themfor those pushed imageswe haven't had thishandling or processingit should be determined by your user scenariosthank youplease usemicrophone otherwise interpreter cannot interpretplease use a microphoneturn on the microphoneyou mean the signaturewhat kind of signature請打开麦克风要不然反应听不到没法翻译you mean use the cvd databasesoftware不打开麦克风反应没法翻译no microphone no interpreter请打开麦克风in terms ofscanninghover is responsible forpushing it intothis layerI don't knowI don't understand your questionwhat do you mean by the signatureit means aboutversion or additionthe cvd databasewill identifythe vulnerabilitiesso there will be a lot ofall kinds of software versionsand it will knowall the softwarepackages within yoursoftware it will justcapture the version for youbecause it saysa static scanningfor your projectfrom the scanningto the resultor to this policy filteringand I think it'sokay but for this autorecoveryafter recovery there will bea rescanningif there is water but it isand it will continue the scanningbut maybe this recoverywill see a failureif there is a failure how do you handle itif there is a failureit will repeatthree timesif there is a failureit will be just showingthe reportsso there should be some human interventionin this case I know thatbut in terms of thisrecovery I believe there are a lot of thingsthat need human interventionand the software versionor this hardwaresoftware dependenciesas if you update one softwarebut this softwarelie on other packagesthat depend on otherpepages so we knowthat only one softwarevulnerabilityrequiresseveral kindsof software package updatingso in this caseI believe the human interventionis a must in this caseand I think that this willrepeat three timesthree times will be this maximumso for the whole recoveryand I believe there willvery possible to see many failuresin terms of vulnerabilityif you do the scanningI will know thatit is static imagebut for many vulnerabilitiesthey should be handledwithin the operationI know I mean thatother this vulnerability isusefulorif it is notworth itI don't think we should recover itso itadvise thatwe must know that whether thisimage is useful or notso it means that it stillrequires human interventionbased on scanner resultbased on scanner resultthere should be this human interventionto determinewhether you need this recoveryam I rightplease use a microphoneand why the interpreter cannot interpretother questionnow you do the scanneris it clearyes and I believewe will use more scannerswe use othertype of scanner to do the scanningmaybe we willget different resultshow do you handle this situationor scenariosome companiesdoesn't try the third partyopen source toolsome companies just relyor trust their own toolsjust rely on their own scanner resultthat's why we do it in this waythank you是的是的we can do this limitationthe challenges thatwithoutbreaking the original commandswe just updatethese software packagesthat should bethat should see this updating如果是自己的代碼的話一開始不設計的話for this systemfor this systemsoftware packagesbecause we know that it's thatscanningfor your code scanningshould be done insideyour programsinstead ofother channelsand believe thatyou give me the boundaryin terms of the darker imageyou just give me the gold boundaryscanningso the source codescanning must be done in your ownany questionsok thank youthat's all for my presentationthank you all for joining usthank you