 Thanks, Dan. So we are going to talk about service authentication and specifically how to choose whether or not you're going to use tokens or certificates in your authentication. Real quick, who am I? Like Dan said, I'm the CTO of Tremont Security. I've got 20 years of identity management experience, big Fortune 100 companies, governments, various vendors over the years. I've been working with Kubernetes since 2016, wrote a lot of the documentation around OpenID Connect, worked on implementing authentication in the dashboard and in most recently in Keali, and co-author of Kubernetes and Docker and our prize guide. So enough about me. So when we're talking about whether or not we're going to use tokens or certificates to authenticate your services, start by asking yourself a few questions. First off, does your service use end user data? Some services are used by end user applications, and a lot of that will dictate the easiest way to go for authentication. How much is automated? If you're not automating at least something, then you should probably take a step back and think about how you would automate it, because trying to do this stuff manually is going to get you into a lot of trouble. How complex is the network? A lot of times when we start talking about certificate authentication versus token authentication, a big thing to think about is how close is your authentication point for the certificate to the actual system that it's protecting? And then finally, how does a transaction span multiple service calls? Most complex services are more than one call. You're not just going to say, OK, call this one service. Get your answer back. You're done. Often you'll call one service. I'll call two other services to get the job done. Does the transaction span that? Do you have to be able to get user context across those services? That makes a big impact on which way you decide to do your service authentication. So this slide's really the biggest slide. I'm actually going to see if I can get rid of the. Make it a little easier for me. No, no, maybe not. Am I still sharing? I am not. Try that again. All right, cool. So this particular slide's really the one that you want to pay the most attention to. It brings together a lot of the information on comparing and contrasting certificates and tokens. When we're talking about certificates, the big strength of the certificate is you have no secret going over the wire. Very, very secure when you do it correctly. A big benefit there is the fact that you're generating your private key. You're generating a certificate that's getting trusted. Your service doesn't need to know anything about that private key. There's no shared secret. You look at tokens, the big strength is its ability to be able to move across networks easily. You're able to say, okay, here's my token. Doesn't matter how complex the network is. The other big thing that's great about tokens is that it can generally include identity claim information in it. So you're able to put, for instance, user context information, group information, perhaps transactional information that can all be verified, so you don't have to go back to the source in order to get that data. Now, when you're looking at the drawbacks of these two methods with certificates, the biggest drawback is one that security that I mentioned only comes if you do it the right way. And by right way, I mean, you're generating a private key in a secure store. You're generating a public key and a certificate signing request. That certificate signing request is getting signed properly and then re-imported. Using something like Istio or other service meshes where this is done for you, that makes it a lot easier. If you're generating a P12 file and then downloading it, chances are you're not doing it correctly. So doing that correctly is very, very difficult. Another aspect of certificate authentication that can be much harder is revocation checking. Certificates tend to be very long-lived because you don't have that secret going over the wire. And so with that long-lived credential, there's a chance that that credential's been revoked. That certificate's been revoked. Well, there are only three ways, really two ways honestly, that are standards-based that you can do that. Either you're checking what's called certificate revocation list, might be pulling it from a file system or LDAP, or you're querying OCSP, very old protocol to check to see whether or not the certificate's still valid. Kind of a flaky protocol, truth be told. And it can be very problematic. So if you have the certificate, in order to do it securely, you need to make sure that you're generating it securely. You also need to make sure that you are checking for revocation. That revocation part's very hard to do in most instances. With tokens, the big drawback is you have to send that secret over the wire. So whether that secret is really well-built cryptographic secret or a really poorly-built hand-encrypted secret doesn't really matter, the fact is it's going over the wire. So if somebody gets a hold of that token, they're able to use it without having any other piece of information. There have been attempts to build standards to bind a token to a connection. They had one a couple of years ago that Microsoft was really pushing, didn't really go anywhere. Chrome decided to take it out of its own implementation. So that's always been the biggest issue with tokens is how do you protect that secret over the wire? The biggest way to mitigate that risk is to have very, very short-lived tokens. So if your token's only alive for a minute, chances are by the time the attacker gets a token, figures out what it is, figures out what they can use it for, it's already expired. So that goes back to the automation piece about why automation is so important for your security. And then the other major drawback for tokens is that doing a proper service-to-service exchange can be really painful. There is a specification that lets you do it. It's called the OAuth 2 token exchange. I just realized I'm still wearing my mask, sorry. The OAuth 2 token exchange, it's doable, it works very well, but it's something that you need to know how to do inside of your code, inside of your token. The big risks with certificate authentication boiled down to are you using it properly and then how far away from the application is the certificate. So when you're doing something like Istio where you've got that sidecar that's validating the certificate for you right up against your pod, that's really great because there's very little distance between your certificate validation, your applications. The fact that it's just injecting some identity data as headers is not a big deal. If however your certificate validation point is four proxies up and now you just have some standard contact information going through these proxies, there's no way to validate it, that becomes a real security risk at that point that anybody can inject that information along the way, you can't really track it from level to level. The major risk for tokens is gonna be the fact that it is a bearer token, it's something that goes across the wire, somebody catches that, somebody logs it by accident, that's where you're gonna run into issues. So food for thought while you're looking at which one to choose, don't be afraid to mix a match. There may be scenarios where you're going to start off, let's say you have an application, that application's gonna talk to your services, that part might be token based because it's really easy for a web application to work with tokens, users aren't generally going to have certificates, there are some instances where that's not true. But for the most part, your users will have tokens, they won't have certificates. But then once you get into your infrastructure, maybe you then switch from certificates or from tokens over to certificates, that's perfectly valid to do that transition. Just make sure you're logging it, keeping track of it, things like that. Service meshes and API gateways are really important and the line has gotten fuzzier and fuzzier over the last couple of years as to whether you'd use a service mesh to do authentication or an API gateway, but whichever one you choose, and I'm not gonna say one should be used over the other, it's gonna really depend on your situation. Don't do it alone, don't try and reinvent the wheel because chances are it will go horribly wrong. And I kinda say like inventing your own authentication systems like inventing your own crypto, don't do it. There are a lot of standards out there with thousands of hours of peer review, thousands of implementations, stick with those standards, they might not be perfect, you might not think it suits your need or what you want, but chances are the terrible documentation for those systems is really what the problem is. So yeah, some links and resources. A couple of blog posts that I've written about certificates versus certs for the API server specifically, and then using tokens and pipeline authentication just gets into the real nitty-ditty gritty detail of how service accounts work. And then this blog is written by a friend of mine, Alex Ellis, he's the author of Open Fuzz and Inlet. And this blog's really interesting because it talks about using OIDC with tokens from GitHub Action. So using GitHub Action to securely call Open Fuzz. He based it off of a new release of a tool that lets you use GitHub Action's OIDC to call GKE resource. And this really shows both the benefits and the risks of using tokens because in this one example, you're able to both generate tokens across different mediums, but at the same time, that means secrets are going over the wire. So you're getting better security by not having a secret stored locally, but then you're still getting a security boost because now, okay, I'm running my GitHub Action but I don't have to generate a secret in order to use GKE. And then finally, some shameless self-promotions. Say hi on Twitter. If you are looking for an identity provider, we make Open Unison, open source identity provider to Poise Wright and the Cates. And then our book, Kubernetes and Docker and Enterprise Guide. That's it, thanks. Thanks so much, Mark.