 Today we have with us Michael Michael a harbor maintainer and director of product management at VMware Michael first of all welcome to the show. Thank you. Thank you for having me Let's assume that you and I are stuck in an elevator and I certainly asked you what is harbor So, please explain what is it? Hopefully we're not stuck for the in the elevator for long But harbor essentially is an open source cloud native registry think of this as a repository where you can store and Serve all of your cloud native assets your container images your home charts And everything else you need to basically build cloud native applications And then we can put in post on top of that some very good policy engines that allow you to enforce compliance Make sure your images that you're serving are free from vulnerabilities and making sure that you have all the guardrails in place So an operator can manage this registry and deliver it to his developers in a self-service way Harbor came out of VMware China So I'm also curious that What was the problem that the team saw that point because there were a lot of projects that were doing something similar That you know, you saw unique that harbor was created. Yeah, so essentially the need there was There wasn't really a good way to for an enterprise to have a hosted registry that has all of the enterprise capabilities they were looking for While at the same time being able to have full control over the registry like a lot of the cloud providers have their own registry implementation There's docker hub out there Or you can go and purchase something at a very expensive price point but if you're looking for an open source solution that gives you end-to-end registry capabilities like your developers can push images and pull images and then your Your operators can go and put a policy that says hey I want to allow this development team to create a project but not use more than a terabyte of storage None of those solutions had that so there was a need A busyness need here to develop a registry and on top of that we realized that it wasn't just as that had the same need It was a lot of users and enterprises out there in the cloud native ecosystem The project has been out for a while And and based on what you just told me I'm curious what kind of community the project has built Around itself and how the project has evolved because we'll also talk about the new release version 2.2 Zero but before that I want to talk about the evolution of the project and the community around it project has as evolved fairly well Over the years who have increased our contributors and the contribution statistics That cncf is creating are showing that you know we're growing our community We now have maintainers in the project from multiple organizations And there are actually three organizations that have more than one maintainer on the project So it's kind of showing you that the the ecosystem has picked up We are adding more and more functionality into harbour and we're also making harbour pluggable So there are areas of harbour where we're saying hey, here's the default experience with harbour But if you want to extend that experience based on the on the needs of your of your users Go ahead and do that and here's an easy way to implement an interface and do that That has really increased the popularity of harbour. That means two things We can give you a batteries included version of harbour from the community And then we give you the option to extend that to fit the needs of your organization And more importantly if you have made investments in other tooling you can plug and play harbour in that I want to say other tooling. I mean things like CICD systems You know those systems are primarily driving the development Life cycle. So for example, you go from source code to a container image to something that's stored in a registry like harbour The engine that drives the pipeline that workflow in a lot of ways is a CICD engine So how do you integrate harbour well with such systems? We've made that a reality now and that has made harbour easier to put in an organization and get it adopted With existing standards and existing investments. Now, let's talk about the the the recently announced 2.0 Talk about some of the core features functionalities that you are excited about in this release Yeah, absolutely. There's like 304 features are really really excited me. A long time coming is the support for OCI The OCI is the open container initiative and essentially is creating a standardized way to describe what an image looks like And in harbour 2.0 we are able to announce that we have full OCI support in harbour What does that mean for users? In previous releases of harbour you could only put into harbour two types of artifacts a container image and a home chart Great it satisfies a huge number of the use cases for customers But it's not enough in this new cloud native ecosystem There's additional things that as a developer as an operator as a kubernetes administrator You might want to push into a repository like harbour and have them Also adopt a lot of the policy engine the harbour provides give a few example Cineabundles the cloud native application bundle you could have OPA files you could have singularity And other OCI compliant files so now harbour tells you that hey you have any file type out there If it's OCI compliant you can push it to harbour you can pull it from harbour and then you can add things like quotas and retention policies and immutability policies And replication policies on top of that think about that now just by adding A few more types of supported artifacts into harbour those types immediately get to use the full benefits of harbour In terms of our entire policy engine and the compliance that we offer to administrators of harbour I think this is one of the like first or earliest projects which are like, you know From the street point of view which are like was OCI compliant What does it mean for users because by being compliant you have to be more strict about What you can and cannot do so can you talk about that and also How does that also affect the existing users should they have to worry about something or it doesn't really matter Yeah, existing users shouldn't have to worry about there's full backwards compatibility They can still push their container images which are OCI compliant And if you're using a home chart before you can still push it into charts museum That is a key component of harbour, but you can now also put A helm chart as an OCI file So for existing users not much difference backwards compatibility We still support them the the users that brought us here. We're not gonna forget them But what it means now is actually it's not more strict. This is a lot more open If you're developing artifacts that are OCI compliant and they're following the standard way of describing an image And a standard way of actually executing an image a runtime Now kubernetes is also OCI compliant at the runtime Then you're getting the benefits of both worlds you get harbour as the repository where you can store your images And you also get a runtime engine that's OCI compliant that could potentially execute them Really great benefit here for the users A couple of other features the harbour to the low brings are super super exciting The first one is the introduction of trivi by aqua security as the battery is included built-in scanner in harbour Previously we use claire as our built-in scanner and with the release of harbour called 1.10 that came out in december of 2019 We introduced what you call a pluggable framework Think of this as a way that security vendors like aqua and ankor and dusec can come in And create their own implementation of a security scanner to do static analysis on top of images that are deployed in harbour So we still included claire as the built-in scanner and then we added additional Extension points now we actually liked trivi that much our community and our users love trivi its ability to Enforce and do static analysis on top of multiple operating systems on top of multiple application managers It's a very well aligned with the vision that you have from a security standpoint in harbour And now we added trivi as the built-in scanner in harbour. We ship with it now Great great achievement and queued us to the aqua team for delivering trivi as an open source project Yeah, that's another question I was going to ask but once again, I'll ask the same thing again that What does it mean for users who were using claire? If you're using claire before and you want to continue using claire by all means We're going to continue updating claire claire is already included in harbour. There's no changes in experience However, if you're thinking That trivi is a better scanner for you. And by the way, you can use them side-by-side so you can compare The scanning results from each scanner and if trivi is a better option for you We enabled you to make that choice Now the way harbour works is that we have uh a concept of multi tenancy And we isolate a lot of the settings and the policy and the organization of images And on a per project basis So what does that mean you can actually go into harbour And you can define A project and you can say for this project I want claire to be the built-in scanner And then claire will scan all your projects in that Or the files in that project and you can say that by second project and say Well, I know I want trivi to be the scanner for this project and then trivi will scan your images And if you have the same set of images you can compare them and see which scanner works best based on your needs as an organization and as a user This is phenomenal, right? We give users choice and we give them all the data But ultimately they have to make the decision on on what is The best scanner for them to use based on their scenarios The type of application images and containers that they use and the type of libraries that they use in those containers Excellent Before we wrap this up If what kind of roadmap you have for harbour, of course, it's an open source project So there's no such thing as when the 3.0 release is coming out But when we look at 2020, what are the major challenges that you want to address? What are the problems you want to solve and how What does the basic roadmap looks like? Yeah, absolutely I think that you know One of the things that we've been trying to do as a maintainer team for harbour Is to kind of create some themes around the releases kind of put a Blueprint down in terms of what is it that we're trying to achieve and then identify the features that make sense in that theme And we're not kind of we're not coming up with this from a vacuum like we're talking to users We're talking to other companies where we have cubicon events in the past where we had presentations and Individuals came to us asking us a set of questions. We have existing users that give us feedback When we gather all of that one of the things that we came up with as the next theme for our release Is what we call image distribution? So we have three key features that we're trying to tackle into that area The first one is how can harbour act as a proxy cache to enable organizations that are either Deploying kubernetes environments at the edge and they want a local harbour instance to proxy or mirror images From the mothership like your main data center and we're networking inside the premium Maybe some of the kubernetes nodes are not even connected to the network And they want to be as equal to pull images from harbour and then harbour pulls the images from the upstream data center Very very important feature Continuing down the path of image distribution. We're integrating harbour with both dragonfly by alibaba and project kraken by uber To facilitate peer-to-peer distribution mechanisms for your container images. So how can we efficiently Distribute images at the edge in a multiple data centers in branch offices With with that don't have a good network or thick network pipe between them And how can harbour make sure that the right images land at the right place? Big bit features that we're trying to work with the community and obviously we're not doing this alone We're working with both kraken and the dragonfly communities to achieve that And last the next feature that we have is what we call garbage collection without downtime Traditionally if you do garbage collection And this is kind of the process where you get to reclaim some of the files and layers of Basically container images that are no longer in use Think of an organization that pushes and pulls Thousands of images every day they retag them they create new versions Sometimes you end up with layers that are no longer used In order for those layers to be reclaimed at the storage and and by the system Their registry needs to be locked down as if nobody can be pulling or pushing images to it In harbour 2.0 We actually made a significant advancement where we track all the layers and the metadata of images in our database Rather than depending on another tool or product to do it So now this actually paves the road so that in the future We could actually do garbage collection with zero downtime where harbour can identify all the layers They are no longer in use Corey claim them And then that will have zero adverse impact or downtime to the users that are pushing and pulling content Huge huge features and that's that's the things that we're working on in the future Awesome. Thank you Michael for for explaining things in detail and talking about harbour I look forward to talk to you again. Thank you. Absolutely. Thank you so much for the opportunity