 Hello and welcome to conformance testing the Kubernetes API tooling that makes the difference. I'm Caleb And I'm Steven We're with II a group in New Zealand with a focus on cooperative coding pairing is sharing for us You can find us at II diet in Z Who is II eyes made up of the following humans myself and Steven who are your speakers? Brenda who makes II function happy. Who's the founder of II? Rian who's the project manager and Zach? Who's the data wizard? Do I talk about conformance and it's tooling? What is Kubernetes conformance? I? Program by the CNCF to ensure that Kubernetes is the same everywhere with it you get stable API's November lock-in and Portability of workloads When I run my workloads, I expect them to run the same wherever No matter the vendor The general current conformance information about the program its vendors and distributions check out CNCF.io CK Calling that makes the difference We define and verify it conformance through tests Today we'll show some tooling that I use is to help improve Kubernetes conformance coverage The primary tools will focus on our API snub and pay dot sharing that I am First we have the API snub suite the suite is responsible for collecting and processing the data that makes up the definition of conformance That's allowing community members to generate the data that displays the results for conformance progress and also build out tests There are three components that will walk through First is snubDB, which is a Postgres database that processes the Kubernetes open API spec and any Etoedo log that you wish to make it ingest It runs as a job or it runs live in your cluster for querying And it also statically renders that data, which is then later picked up by web, which we'll talk about soon Next is audit logger. This is useful for live test writing It is a shim that sits between the API server and snubDB It captures logs using API server audit sync and audit policy to point to itself And it's also what we use for our live conformance testing development Finally we have web, which you might be most familiar with it's available at apisnub.cncf.io This is what picks up that statically rendered data and displays it as some very pretty dashboards and useful statistic information for sharing with the community Next we have pair.sharing.io Over the last year, IEI has built an open source development environment that runs in Kubernetes We think it's pretty cold to be working inside of a cluster Pair, governance name, it's about pair programming. With Pair, you get in a cluster With any clusters, any repos that you wish to work with In our case today, we'll be working with Kubernetes and apisnub Thanks, Steven. Now let's take a look at pair.sharing.io Here you can see the home page You're greeted with two buttons after you sign in. The first one is to list all instances and also this button to create a new one I'm going to go to this one to create a new instance, which will eventually give me a cluster Here I'm going to input the Kubernetes repo because while we're writing on we're writing Kubernetes tests and also the CNCF apisnub repo so that we have live test writing Enhancements. I'm also going to write apisnub ticket writing, which is a repo that IEI uses for all about ticket writing for Conformance tests. I'm also going to invite Steven as a guest so that we can both pair on it Normally now it hit launch, but since we already have an instance prepared. Let's jump to that one Going to the instance After clicking this list, you'll be greeted with a page to display a whole lot of information about it such as this name the creator, the time when it was created, who it was shared with, and various repos that it's loaded up with On the right hand side of the page, you'll see some beneath the metadata You'll see the sites that are available right here. These are ingress endpoints that are exposed in the cluster There's several ways to connect. You can connect through such things as a kube config through SSH or even through the web We have a pre-prepared window up to connect to the pair session right now. So we're going to switch over to that one Once in You'll be greeted with a A teammate Session teammate is quite similar to T-Mux if you're already familiar with it teammate makes those shared terminals Makes those multiplexed terminals shared With this cluster you can see that we're actually running the latest version which is 1.22.1 as of today If I run hostname, you'll see that this is the hostname for where we are right now Which happens to match this pod, which is the first of this list So you can see that we're actually working inside of a pod That's pretty cool. It's called humax Which is an Ubuntu base with teammate emacs and a whole lot of utilities that I use You can also see that You can also see the ingresses that are available This list matches the the one for the instance page also you have DNS access to your instance so you can manage any record that is at your instance name and beneath What's also special is that you have full TLS across all of the your pair host and subdomains You also have a web server That is running on dub dub dub With this you're able to to read and write files into the public html folder This will allow you to expose any files that you wish and Yeah, we'll look more into this later comes in handy Let's take a list of All of the pods that are running right now As you can see at the top you have Snoop db and audit logo. Those are the components from earlier And we're going to use them again very soon Why don't you tell us about emacs and org mode steven? I think that there's some very interesting things that we can do with that Yes Caleb. These are a ton of things that we can learn from emacs and org mode he's a basic Document that looks a little similar to markdown, but with emacs we've got some interesting superpowers that we get with org mode So inside of uh, code blocks not only can we document Sorry, we can both document the actual code but also the results from them. So here we've got the current working directory And here's the current pods So this lines up with what Caleb showed a little bit earlier the Abilities are Record both the content and the results is Pretty powerful and it helps us to keep on track with what's current and the next thing's on our checklist so This document here This is a starting template That we use when we're looking at starting a new issuer around potential endpoints that for conformance It's been helped to structure The discussions that we've had with the conformance group And so conformance meetings as well as with sig architecture. It's been very very helpful Down here When we're looking at testing for new endpoints, we can use api snow To ask it. What is currently untested? And by running this code block, we can start to document the process around Untested endpoints as well as what's the resources around them these resources once we Decide that They need to be tested as part of the conformance process We'll start to look at documenting them and then use the outline here as a way to start To structure how we're going to go about testing this new resource Once we've got that in place We would then update this Basic go test test that is currently working around pods To match the new resource. So instead of testing And creating a number of pods And listening them before deleting them We would end up Switching this out for another resource that's going to be part of conformance in the near future We can look we can track how the Odds are interact during this Test So that if we run it, we're going to see the Resource has been not only started but also terminated This lines up with the results that we've been given up here We can talk with and check with How api snow has seen these recent Endpoints been hurt And now a sequel a sequel query here the sequel carries showing the Create delete and the listing These endpoints have already been tested So unfortunately This follow-up query will query what is Currently going to be a change in test coverage But at this stage 402 endpoints showing full coverage at the moment Caleb what happens once we've got this far done? Great question So we're actually able to export it into different formats to use for different things For example, we can export it as various formats such as odt Markdown html or github flow markdown Along with the many more that there are i'm going to export it as html so i can view it on the web Now that it's exported i'm going to go to this folder and then copy that html file into the the public html folder From here we're able to navigate back to pair.sharing.io Once on pair.sharing.io We're able to go to the dub dub dub subdomain And view any files on it So i'm going to go to there And then i'm going to go to the the file that i put in there, which is called mock template the html Now i can view it in reasonably nice rendered html format This will enable quick review for anybody no matter what device that they're on Yeah, we we use this occasionally And another thing that we can do if i jump back to Emacs is if i go to the org file And then i export it again i'm going to this time export to github flow markdown And then from there i'm going to once again go back to Copy it to the public html and then return to pair.sharing.io And go to that subdomain and then go this time to markdown From here what i'm able to do is i'm able to Create a github ticket out of this Which i paste the contents of that into this box You can see right here that the ticket is rendered and it shows all of the steps of why do we need it What it is how we achieve it and so forth and Proving it from here we're able to bring this up into a discussion at the conformance meetings If not beforehand We we would also be committing this and pushing it to a branch for further review But i wonder what if there's an existing endpoint which just needs to be upgraded to Conformance and stable. What can you tell me about that steven? Pscale Once we've got a test that's actually part of the Kubernetes code base we'd be looking at Discovering where it is in The code base and i've got an example here in the diamond set This endpoint here has got a name A description inside of the name Uh quotes here at the top and with that Set of information i can Go back to a org file i've got org file here that's got the diamond set status and i'm going to run it at the moment and No endpoints have been hit at the moment But based on the Based on that Description that's in the top there we can go across to here and Put that information into a variable and then like at Running a ganko Uh focus test using the etui test binary This is uh testing the diamond set status and using some watches to check That various events have happened With once it's passed I can go back To the Query that we've got here on the block and when I rerun it I can find A number of endpoints have been hit This helps us to clarify with the conformance group And the appropriate sake in this case it was so gaps That we've worked with with a number of uh diamond set endpoints And this last recent lease And very well to confirm that everything's looking great So okay once I've finished all my work. What would I be thinking of doing next? Yeah, of course so Once we're finished doing our work We end up pushing it away and making sure it's committed and saved And then we make our way to pay dot sharing that I want small and then We can hit this big red button Which ends up deleting our instance this will uh Delete our cluster and uh Mean that we'll have a fresh one for the next day Why is it important to have a fresh instance every day? Because we want to make sure that our development environments are not pets This will increase automation and knowledge of how to work on the project And it'll also keep our configurations up to date and in sync and shared with everyone So that's pair Would like to give a special thanks to all of the conformance contributors of And community who have helped us get to 77 percent Conformance coverage, which is really awesome But also like to thank anyone who's contributed to make any endpoints stable or generally available Here are some useful links that you may wish to check out The top three are around the conformance testing program Ways to help out and also what's required to make sure that your cluster is certified For a particular release of kubernetes the last one further enhances the discussion around testing and how to find endpoints And just clarifying the process that we've shown in part here today Next if you'd like to get in contact Please reach us via the kubernetes like on the kates conformance channel. That's where you'll find the kubernetes conformance working group There's also the Google group and email that you can Send emails to which is on screen right there We'd like to give a special shout out to a number of open source projects that have helped not only the Test writing process become a lot easier, but also For presenting this Talk reveal js is really awesome and we're able to drive it through all mode Thank you those who attended live and those who have decided to watch later We hope to hear from you and we hope that this talk has helped you Let's move on to some live q&a