今日はソフトサプライチェーンについてお話しします。このセッションは、ソフトサプライチェーンのセキュリティについてお話しします。特に、CDTOPシナリオについてお話しします。そのため、これについてお話しします。I'm Yuji Watanabe from IBM Research.I'm a senior technical staff member.I'm leading the several software supply security projects in the research.Hiro?Yes, I'm Hiro Kitahara from...Okay, so I'm Hiro Kitahara from IBM Research Tokyo.I'm working on CDTOPs earlier,and I'm doing several OSS contributions,like CIGSTORE or today's main topic, ROCD.Yup, thank you so much.Okay, so today, the first,I will talk about the background of this topic.Then, what we are trying to discuss in this talk.Then, we will explain the several building blocks,Yabano Manifest's signature, and its enforcement.So, then discuss how can we apply the technology to the CDTOPs,and we found several...and talk about the proposed solution called interest.Then, Hiro will do the demo.Okay, so the first software supply chain is a product.So, they're protecting the product from the source,and its build, the factory,and go to its package,then become the product.But from the end-user perspective,we need more protection.It should be securely delivered,and it should be securely maintained,and it should be protected at the time of the use.So, the whole end-to-end is the scope of the protectionfrom the user perspective.Then, we think about the same things in the application deploymentin the corresponding case.The very high-level picture.The first source is in Git repo.It goes to the CI pipeline.Then, build the image,and push it to the registry,and Manifest is created at Git repo.Then, when deploy the application,the image goes to the cluster.Manifest goes to the cluster by a CI pipeline.Then, application stands up on the cluster.So, this is kind of the end-to-endfrom the source to the actual time of the use.So, what is the risk?So, maybe in some cases,image on the registry may be the target,and Manifest on the Git repo is also the target.So, even after the deployment,Manifest is on the cluster.So, it may be changed by someone.So, those attack and measures activityis the target for the protection.So, for preventing this,actually, for protecting the integrityof the artifact and the application,usually, the signature is applied.So, the many technologiesexist for the signing the imageand how to protect,how to enforce the image signing on the cluster side.For the application,image and Manifest are equally important.So, we also need to protect the integrity of the Manifestto secure the entire pipeline,the entire lifecycle.So, in this talk,we are mainly focusing on the Manifest side.So, how the Manifest can be securely signedand deployed to the target cluster.Okay.So, for that purpose,actually, we contributed to the signature projectfor the Manifest signing.This is a CLY2 QBCTL pluginby using this command,some annotation.Signature annotation is attachedon the original YAML Manifest.So, in this case,one is encoded messageand second one is encoded signature.So, the signature is encodedincluded on the Manifest.So, if you can directly deploy this cluster,signature is also attachedon the deployed resource Manifest.So, how to verify the signatures.So, this is a flowhow the signature is used to protect the Manifest.So, the signer first signs the Manifestthen push into the gate repo.It will be pusheddeployer pulls the Manifestto deploy the cluster.So, before deploy that,deployer can directly verify this signatureby using the CLY2.And then, if the verification is okay.So, the deployer can deploythis Manifest to the cluster.Cluster sites, the admission controlis enabled.Independently, this signatureattachedon the Manifestis verified in the cluster site.That verification happensin the admission timeor even after that,the signature is continuously monitored.So, you can enable this type of the protection at the cluster site admission control.So, it'sthe signature library toolfor the Manifest sign is usedin the admission control.So, the one admission controlis the equity sheet.This is the project we arecontributing to theopen cluster management project.It works withOPA gatekeeper as admission control.The integrity sheethas integration with OPA gatekeeperto provide the YAML Manifestverification.Signature verification.Also, the very recent releaseof the Qiverno hasYAML Manifest signatureverification capability.So, theSeveral tool can be usedto enable the admission control.Okay.So, wethen nextwe will seethe more complicated situation.The some case,you don't push the Manifest directlyto the gatekeeper.Instead, you push the sourcematerial to producesome materialmanifest.Actual Manifestis generated some tocustomize command.Customize command is already integratedin the QCTel command.Also, you can use the directlycustomize command.That producesmanifest from the sourcematerial on the gatekeeper.Plus, you can alsoinclude external dependencymaterial.So, you can input sometemplate parameter.Then, this commandproduces generated Manifest.So, in this casethe discussionis a bit different.Before we start,let me explainmorethe situation of thistemplating.So, the sourcematerial is not just onematerial.You can have multiple repositorieslike the application repository,configure repository.So, you can usetwo repositoriesas the kindof parent child relationship.But, you can also havevery different complexscenario.Each organization has differentapplication componentas a kind of sourcematerialis composedinto thesingle setof Manifest.So, this is alsokindof the software supply chainissue.Very similar tothe imagebuild case.So, yousometimesif you build a sourceimage, you need to pulla lot ofthe external dependency.The same issuehappened also in thisManifest build.So, in this case,from the signature verification viewpoints,the situationbecomescomplicate.So,Syner can put the signatureon the source repository.The deployer canverify the source material.But, the deployer afterthe verification,manifest buildand then deploy the generatedmanifest.That means,generatedmanifest does not have the signature.And so, that means,deployer can verifythe signature beforeactually deploy.But,it's not deliveredto the cluster size.So,crust side admission code doesn'twork in this case.So, that meanscrust side needs to rely onthe paycheckby the deployer.So, butthis isbecome thesituation in the CDGitOps.A CDGitOps case,if some configurationconfiguration is managed on the GIT repo.So, if thesource material changed,it's automatically synced to the target cluster.For thatthings, during that things,CD includes the kind ofbuild and deploy state.Build state getssource material input,then generate manifest,then it's deployed to the cluster automaticallywhenever the sourcegit repository is updated.So, if weuse this CDGitOpsvery,very usefulto,but if weapply this CDGitOps too,the generate manifest don't havethe signature as I mentionedexplained before.So, this is the kind ofmutation happened in theCD pipeline.So, no signature isdelivered to the target cluster.So, this is a motivationthat we came upwith the interest project.So,in this case,there are several questionshowwhat kind of check is donein the CD sideand how build manifest buildis properly done.So, the signatureis not attached to thegenerated manifest.So, signature-based protection does not workafter the deployment.So, those kind ofin this case,we need to trust the CD sideto the cluster side.There is no verification.We want to improve this situation.That isinterest allows us todo the kind of trust,but verify approach.The interest isas the capabilityattached to the CD.Then, first three function.One is verifythe source material signature.Then, the secondif it puts the signatureon the generated manifest.Then,these transformfrom the CD to reportto thecluster sideon the CDGitOps.We produce the province recordtotransparentto make this transparent.So, this three functionis the core capability of the interest.Howinterest works?Actually, this isinterest is a controllerattachedinto the ROCD.So, basically, it monitorsapplication seekwhich happensbetween the Git repoand the target cluster.Then, it's reportedin the application CR.Application CR isthe configurationwhich specifieshow the GitOps syncsshould be performed.So, theinterest controller monitors this application CR.Then, it detectsokay, so the GitOpssyncs happen.Then,triggerthe three functionverifies the signature.So, source material signatureand producethat's the signatureto the generated manifest.Then, theprominence record is pushed into theprominence store from the interest controller.So, by using this,the signature is attachedon the generated manifest.So, target cluster site,the admission controller cansuccessfully protectmanifest by using signatureatmission timeand also even after the admission.Okay, so,this is a high-level ideaof the internet.So, from now, the herowill show some demoand some deep.So, hand over to the hero.So, okay.So, let me send my demoand I will show howargo CD users canprotect their own CD GitOpsby playing with Argo CD Interest.Okay, so during my demo,let's assume that I amargo CD userand I have already configuredmy CD GitOps pipelinewith Argo CDand the lefttop side, this source repositoriesare synced into target clusterbased on myapplication CRandso, the point hereisthe source repository can bemultiple and nested.So, this means the source repository in CD GitOpscould becould havevery complicated dependencyrelationships.So, in my demo,I have two source repository.The first one is the rootsource repository which is configuredin my application CRand this refersto another repositoryas an external dependency.So, thisthey have our parent and child relationshipandnow, I am thinking abouthow I can protect my CDGitOps with digital signatures.So, for thatI will introduce Argo CDInterest.And to makeN2Nsignature protectionin my CD GitOps pipeline,I have installed admission controlleron target clusterwith the signature on thecombat resourcesand to configure Argo CDInterest, what I need to dois just three steps.The first one issigning source files.So, to put the signatureson the source repositorywith my signing keythe source repositoryfiles are protected.So, to do that,I can sign the source filesand thenthese two signature filesare generated andthese signatures are usedby Argo CDInterest for source materialverification.And the second step isconfiguring Argo CDInterestwith custom resource calledinterest profile.So, this profile hasselectorto which applicationshould bein scope of thisinterest profileand also it has a public keyfor the source material verificationand also this hasa signing keyfor the YAML Manifest signing.So, this is what I'm actuallyusing for interest profileand it has application selectorand public keyand signing key referenceand also this hasmuch condition.This defineswhat type of resources should besigned by Argo CDInterest.So,as a final stepthis Argo CDInterestwill push thesign YAML Manifest into OCR registryas our OCR imageandpushed image and signaturewill be used by automation controllerfor verifying the resources.So, to specifythe location of this OCR imageI will specifythe OCR imagein application CRandthis is the one.This my application CRhas a special annotation hereand it is manifest imageand this is the actuallocation of the signedmanifest will be pushed.So, by completingthese three stepsI have enabledmy signature protectionin my CDGitOpsand let me explainhow Argo CDInterest works.So, firstArgo CD user will updatethe content inside GitHub repositoryand then it will bedetected by Argo CD.So, application sync hasstarted.So, thenArgo CDInterest iskeep monitoringthe application sync.detect sync eventand it startsverify source material verificationand if itsverification has passedthen it signs the generatedmanifest as an OCROCR imageand at the time ofadmission when Argo CDis trying to deploy the resourcesthe admission controllerwill use the pushed OCRimage with signaturefor resource verification.and all the manifest buildprocess are recorded as aprovenance data and it will be sentto provenance store.This is how Argo CDInterest worksand when the signaturesare all okaythen Argo CD shows thissynced status.This means the application synchas been correctly doneand all the resourcesare deployed on the target clusterthis is a successful casebut let's imagine thatthere is an attackeron the target cluster firstso this attacker is tryingto target cluster resourcewithout signatureand then this admissioncontroller will blockthe attacker's changebecause the attacker'schange is not signedthat change does not matchwith the signed state in the OCRimageso this isactual exampleof the admission controller deniesthe attacker's changeusing QPC to editand so this meansno one can change thedeployed resource on the target clusterwithout signingso Argo CDInterestonly generates thesignature for thegeneratorial manifest by Argo CDso Argo CD onlycan dothat kind of operationhow aboutsource repothere is an attackerwho has access tothe github repositoryand this attacker is trying to changethe file inside github repoand actually this attacker canchange the file inside this repositorybut thenArgo CDInterestverifies the source material signaturebut signature is not valid anymorebecause the file has been changedso thenbecause of thisArgo CDInterest does not signthe manifest hereso there is no signature forthe attacker's changethat's why admissioncontrollercan block the attacker's changeand application sync for itso this isactualwhen blocksthe attacker's change for the source repoand it is showing syncwell so theattacker's source file changehas not been deployed on the targetclustersso far I have beentalking about the signatureprotection on the github pipelinebutI can verifythe deployed resourcewheneverso far I want to checkby using this kubectl sigstore commandthis is provided by sigstoreand by specifyingresource kind and public keyand OCI imagewhich I have mentioned in thespecial annotationand then it reportsthe resources sign stateso if it hasit is showing sign throughit means this resource isresource keepssign state and no changechanged were madeto the resourceand not only this kind ofbinary informationbut also in some casesI want more details aboutmanifest buildand for that I wasinterest where producedprovide provenance datawhich contains all themanifest build informationfor exampleall source repo URLsand all the versionall commit versions as the time ofmanifest buildand also it includesapplication crsnapshotso this is used for checkingthe argocd statusas the time of manifest buildand alsothe OCI image and its digestincluded in the provenanceso this provenanceis provided by argocdinterest and pushed into provenance storeand once this datais ready on the storeI can query this provenancedata by very simple commandand it showswhat source reports areactually usedand what versions wereactually at that timeand build timestampor other metadatacan be checkedby seeing provenance dataand the actuallow provenance data is something like thisit isinterest.json formatand it hasrestore materialslike URLsand commit versionand also snapshotis recorded as a build parameterso I can checkthe status of argocdand also of coursethe image referenceand digest are hereby checkingby checkingby checking provenance dataI can checkwhen my application was builtandwhat source reports and what versions were usedand how wasthe argocd statuswas like that kind of informationcan be checked by provenance dataok so let me summarizetoday's overtalkwe have addressedissue in the separationinitibility of the modern citygithub enginesand to serve that issuewe have introduced argocd interestand actuallywe have confirmed that argocd interestcan work for enablingn2n signature protectionand verifiable provenanceand overalltrust about verifiable approachcan maketoday's application deploymentmore transparent and accountablemore traceablethat's our conclusionsothese are our contactand please feel free to reach out to usand alsoplease visit ibm boothso orderany feedback and commentand questions are all welcomethank you so muchso pleaselike any questioncommentactually I have a very quick questionyou keep referring to interlaced profileis it equivalent to policykind of policywhat's the differenceso theinterest profile isnot therule to how theinterestapplication syncapplication sync eventwhat kind ofgithub syncshould be monitoredwhich resource should be signedand what is recordedthose kind ofconfiguration is included in theinterest profileso that iskind of the policy for usthank youany questionokokso the mutationin the clusterin themanifest build stateok that's a good very good questionthank youthe admission controlok so thequestion isso sometimesif you deploy theadmission controllerother admission controllereven the native admission controllersometimes sidecarmay change themanifesthow we can deal withthat's a modificationin this signature checkingam I correctok so that is a very important questionso this is actuallywe are addressingto develop the admission controlsegregation verificationenforcementtool 6tothis is a toolokanywaywe have theinternally have theconfigurationexplicitlyexplicitly ignoring some fieldit's expected mutationby third sidecartrustedadmission controlleror somethingthat is one approachanother approachwe are delivering the drylandso internally admission controllerinadmission controllerdryland createdexecutedthen computes the expectedchanges in themanifestthen computes which part isthenthen two approachwe discussedthisthis kind of a challenge and a solutionin the different talkin the 6to joint presentationwith jim baguaria kibernowe actually we enabledthis capability in the kiberno ratewith jim baguariaplease checkthat talkif possiblethank you so muchit works this timeso if i amsign itit'sanifest with cosineis there any reason you diddesigning the git commit thatmodified the files directly withcosigns like git sign insteadthat is a reallyimportant question butwe need to look into moresoobviouslyyamlmanifest signing is totallydifferent from thegit commit signingbecause the use case is differentcommit signing is moreindividual commit is signedbut yamlmanifest signing istargeting the signyamlmanifest concept is signedbut newto git signits signfrom the 6th rowwe need to look intomore how we canapply that new approach to thisscenario thank youany other comment or questionalright thank you so muchthank you for your great question