 one more minute and then we can kind of get things started. All right, I'd like to thank everybody who's joining us today. Welcome to today's CNCF webinar, What's New in Kubernetes 116. My name is Taylor Dolezal. I lead Set Reliability Engineering at Walt Business Studios. I was the 116 communications lead. I'll be monitoring today's webinar. We'd like to welcome our presenters today, Kenny Coleman, Technical Product Manager at VMware, and Kubernetes 116 Enhancements Lead, and Lockie Evanson, Principal Program Manager at Microsoft, and Kubernetes 116 Lead. Some housekeeping items I have for you today are that during the webinar you are not able to talk as an attendee. There is a Q&A box at the bottom of your screen. Please feel free to drop your questions in there and we'll get to as many as we can at the end. This is an official webinar of the CNCF and as such is subject to the CNCF code of conduct, which basically translates to please don't be a jerk. Please don't add anything to the chat or questions that will be in violation of that code of conduct. Basically just to be respectful of everybody here today. With that, I will hand it over to Kenny to kick off today's presentation. Thanks Taylor, appreciate it. And thank you everybody for joining here today. You know it's going to be a fun filled afternoon where we're looking at all of the new features as well as some of the graduated features that happened with the 116 release. Now just some quick housekeeping as we start going through here. Please note that we will do our best to answer any questions possible. As Taylor said, please use the Q&A panel if you can. Locky's going to be there to try to answer them. Also please know that we are part of the release team. That means that our goal and our job is to take the everything that is going to be going into the release and funnel it and push out something that is stable and everybody is happy with it. Now that does not mean that we know every little nuance and technical detail of every single thing that we're going to go over. So as we go through here, you're going to see that there is going to be a link at the very bottom for every issue. If you're inclined to read more about it, you can go to that issue and read the cap for the Kubernetes enhancement proposal and that will give you a better idea of exactly what's going into this as well as Locky and myself. We're also not the issue owners nor are we in every single SIG so we will not be able to answer all the details to the you know basically to the we're going to do it the best of our ability but that's what we can do. So just quickly kind of hit the agenda here. We're going to go through some of the really three major features and then we're going to go and touch all of the enhancements but before we get to that I want Locky to kind of talk about the release theme and as you can see this badge and the logo that kind of embrace for 116 and he can kind of give you an update of kind of where this where this all stemmed from as well. Thanks Kenny. So the release mascot there were two aspects to the creativity here. One aspect was going into the 116 release cycle. It was actually the 50th anniversary of the lunar landing which got me thinking about all the Apollo missions. So this badge is actually inspired by the Apollo 16 mission. So if you go and refer to that mission patch you'll take a look and you'll see a very similar thing. An eagle's been replaced with Captain Cube and a few other things there. The the rest of it represents the meme that is around my love of Olive Garden and this represents the fund that the release team had together during this release. So I think it embodies everything that we went through for the past three months getting the 116 release out so that is the history behind the release mascot. Thanks Kenny. Absolutely and so if you ever see Locky in person you all can go out to Olive Garden and just have unlimited breadsticks together and just talk about all the all the great things that happen inside of Kubernetes. All right so let's go ahead and we will dive into the 116 enhancements. So just to kind of give an overview of what this looked like it was a little bit more than we had in 115. I was the enhancement lead back in 115 too so I can kind of give a little bit of context to it but back in 115 we introduced a total of 25 enhancements that were being tracked and now we're bumping that up to 31 here in 116 and we're going to be touching on all these pretty lightly and looking at them at a high level but we had in this release we had 15 introduced as new alpha features that are coming into this. This is typically the ones that people are very interested to know about because these are the things that are going to be extending Kubernetes in a new way that you can gain more value out of. Then we're going to be looking at some of the features that have graduated from previous versions into a beta state which means that they have gone through another release cycle of testing and bug fixing and stuff like that and then we also have eight that have been graduated to stable. This is in my opinion this is a great thing because we only had two graduate to stable in 115. This just gives you a better indication of the trajectory of where everything's going inside of Kubernetes as we progress more towards a stable platform instead of always trying to introduce a bunch of new features all the time this time we have a bunch of enhancements that have graduated to a stable state. So let's again let's look at three of the highlights in our opinion some of the biggest ones that kind of came out of here. First was a bunch that were happening with custom resource definitions and that's going to all take place as we go through and dive down into each individual SIG and we'll be able to see a bunch of CRD mechanisms all move to stable with inside the 116 release. Next is looking at the IPv4 and IPv6 dual stack support. The adoption of IPv6 has increased over recent years and people like yourselves are requesting this type of support with inside of Kubernetes and this was added as an alpha feature in Kubernetes 1.9 to have IPv6 only clusters and that gave us the ability to run things either in IPv4 or in IPv6 but didn't really have a combination or there is this kind of thing called a single pod IP aware dual stack configuration but this had a bunch of limitations from CNI networking plugins system pods from the API and controller server manager there were types of like service IPs had to be either all IPv4 or IPv6 from the cluster so it really didn't encapsulate a lot of things that people were really looking for and so this enhancement is adding dual stack or single family services to support these clusters so what it's doing is providing IPv4 to IPv4 communication and native IPv6 to IPv6 communication to and from within a Kubernetes cluster and you can do this by providing dual stack addresses for the pods and the nodes but you can restrict service IPs to be a single family if you will and so as we start going through here the Kubernetes endpoints the API supports only a single IP address for endpoint then you have the addition of dual stack features so you have pods that are serving as backends for Kubernetes services so now you can have both IPv4 and IPv6 addresses the kube proxy is going to be modified to drive IP tables and IP6 tables in parallel and that's going to require the implementation of a proxier interface in the kube proxy server that is going to modify and track changes in both the tables as well and this is also required to expose services in both IPv4 and IPv6 core dns is also making changes to support multiple address endpoints and there's a lot more considerations about ingress and load balancers that weren't mentioned here so if you want to learn more please go to the issue and you can read more there as well now extending and trying to say really like pvc cloning is is something that we had kind of touched on in 115 because coming from an administrator background you kind of see this as a pretty common tool because you see it in most storage devices today that you know you see them having the ability to be able to duplicate data and you can do this for multiple different reasons whether it's for disaster recovery or whether it's to test data out on a new type of pod or a new type application whatever it is and just to understand that clones are different than snapshots so clones are going to result in a new duplicate volume that's going to be provisioned from an existing volume that's already with inside of kubernetes and as we start going through here the snapshots in their hand are going to result really as a point in time copy of a volume that is itself not a usable volume from there and so that's that can be used to provision a new volume or to restore the existing volume to a previous state so the storage sig identified clone operations as one of those critical functionalities for many stateful workloads that we all want so maybe you are a database administrator you want to duplicate that database volume or perhaps that you want to figure out a way to trigger these clone operations that decided to kubernetes api and so kubernetes users can now handle this without having to go around the kubernetes api and this is enabled in the persistent volume claim data source field and it's adding support for the specifying for specifying existing persistent volume claims in that field and that is going to be the volume that you would want to clone so with this there's no new objects that are being introduced to enable cloning instead this particular field in the volume persistent volume claim object is expanded to be able to accept the name of an existing persistent volume claim in the same namespace and so it's important to note though that from your perspective as a user a clone is just another persistent volume and a persistent volume claim the only difference is that the persistent volume is being populated with the contents of another persistent volume at the creation time and after that it behaves exactly the same way that any other kubernetes persistent volume does and just adheres to the same behaviors and rules just to note though that cloning is only supported right now for csi drivers it's not for entry or for flex volumes so to use this cloning feature you have to ensure that your csi driver is implementing this on the cluster that is deployed so if we look at this example that we have here it shows that we have a persistent volume claim with pvc one that exists in the namespace of myns or my namespace and has a size that is less than or equal to 10 gigabytes and this is going to result in a new independent persistent volume and persistent volume claim called pvc two on the back end that is going to be a duplicate of this data that existed on pvc one so just kind of gives you an idea of what this looks like going through so now let's go ahead and we will dive into crd mania if you will and we're going to look at every individual sig that had new enhancements that came into this it's going to go in alphabetical order so if you're waiting for windows you can wait till the end so here we go for api machinery so for people that aren't familiar with crds or custom resource definitions let's kind of set a baseline here so a resource is an endpoint in the kubernetes api that stores a collection of api objects and that's could be of a certain kind so for example the built-in pods resource contains a collection of pod objects now a custom resource is an extension of the kubernetes api but it's not necessarily available on every kubernetes cluster but represents a customization of your particular kubernetes installation and today there's all kinds of distributions out there that are using crds to kind of put their own sort of special sauce on it now with crds going officially stable we are going to start seeing these come even more prevalent inside of the kubernetes ecosystem and with crd the there's also new the the open api schema is moving to stable so crd authors can create their crds using the open api version 3 schema and this is going to enable server side validation for custom resources and this validation format is compatible for creating these documentation for custom resources which can be used by clients like kubectl and you can perform client side validation like create and apply schema explanation like kubectl explain and client generation and this enhancer will be using the open api version 3 schema to create and publish the open api documentation for these custom resources as well so the sub resources for custom resources is also graduating to stable and these objects are defined by crds called custom resources however it is one of the most requested features and adds a slash status and a slash scale sub resource for custom resources if the status sub resource is enabled the main endpoint will ignore all changes in the status sub path if the spec does not change and the meta generation is not updated now for the scale behavior the number of custom resources can easily scaled up or down depending on the replicas field set that is that that is set inside of the spec sub path moving on with even more crd stuff so we have defaulting and pruning for custom resources now it's a little bit different because pruning is moving to stable while defaulting is moving to beta and 116 yet they are all enveloped inside of a single enhancement tracking issue so defaulting is kind of a fundamental step in processing api objects and the request pipeline of the cube api server and defaulting happens during the during the deserialization period and it's implemented for most of the native kubernetes api types and it plays a crucial role for api compatibility when adding new fields however custom resources do not support this natively and this is all about adding support for specifying default values for that open api version 3 schema that we had talked about inside of the crd manifest and now when we have this the this new schema is going to have a support for a default field with any sort of arbitrary json values that we can put into it and now what we can do is we can apply these default values during the deserialization the same way as native resources do and so custom resources store these these json data values without following the typical kubernetes api behavior to prune these unknown fields and this makes crd's a little bit different because it could also lead to potential security and general data consistency concerns because it is unclear what is being stored inside of xed and this adds pruning of all fields which are not specified in that open api validation schema excuse me and so pruning is now moving forward to stable 116 and this is going to enforce consistency of the data stored inside of xed and this means that objects cannot suddenly render themselves inaccessible because of unexpected data breaking decoding or anything of that nature so even if unexpected data with inside of xed is of the right type and does not break decoding it has not gone through the validation and probably an admission web book either does not exist for the crds or it won't have or it wouldn't have been implementing the pruning behavior in itself and so pruning at this decoding step is enforcing this type of scenario to happen and really it's a counter measure to take care of security tax that could make use of a knowledge of say future versions of api's with new security relevant fields so if you take this into that context without pruning an attacker could prepare a custom resource with privileged fields that are set and on the upgrade of a cluster or upgrade on a version of that cluster these fields could become alive and lead to some unknown or unallowed behavior and so moving on to the web hook version for custom resources also graduating disabled plays well with the previous slide on defaulting and pruning because you can default and prune with a web hook conversion but it's not a native style and requires additional work to make that happen and so the existing problem is when a web hook needs to make a request to another service but the api hasn't progressed or changed so crd users they want to be certain that they can evolve their api before they start down the path of developing a crd and a controller and so this web hook conversion allows developers to evolve their api while using backwards compatibility of versioned api resources and this allows objects or services to hold multiple versions at the same exact time and convert that web hook from one version to another based on its particular need and continue on with even more stability features is the admission web hooks and these are currently widely being used for extending Kubernetes and had been beta for three releases up until now and it's a way to extend Kubernetes by putting a hook on an object during the creation modification or deletion process and so this web hook can mutate or validate the object if you want to as well and this supports namespace selectors and this is great because it's almost like an all or nothing in that namespace so you may not want to get all the activity that's happening so you want to be able to extend that to include a single object selector if you want to as well and moving on to not necessarily just crd's but moving into another feature called bookmark support we had talked about in 115 but this is graduating this beta in 116 and this is talking about the watch api and this is one of the fundamentals of the kubernetes api and it's watch api is there to retrieve a collection of resources using a list and then initiating a wash to start from a particular resource version returned by that list operation so if a client watch is disconnected a new one can be restarted and the last one is returning a particular resource version and now what the bookmark support is going to do is it's trying to make the api server performance a little bit better so if we take this and some of the problems that kind of existed was that different scalability tests saw that restarting these watches caused a significant load on the api server when it was watching a small percentage of changes so in extreme cases you could be you know lead to the point where you're falling out of a history window and a particular resource version two old errors could potentially occur and now the reason is that we want this last item received by the watcher to have a resource version of rv1 and we may know that that there's not going to be any changes to this particular given watcher and we want to bump that to say an rv2 but we don't have any way of communicating that to the watcher so as a result when restarting a watch the client again sends this particular rv1 as a starting point and we process all the events with that resource version between rv1 and rv2 to have that again and so the goal here is to reduce the load on the api server by minimizing the amount of unnecessary watch events that need to be processed after restarting a watch and so this is going to be called a new event called a bookmark and this type of bookmark is going to represent information that all objects up to a given resource version have process for a given watcher so even if the last event of the other types contain the object with the resource version one receiving a bookmark with resource version two means that there aren't any interesting objects for that watcher in between those states and then looking at server side apply graduating the beta with inside of here so everybody knows that cube cuddle apply is a core part of the Kubernetes config workflow but it can be buggy and it can be hard to fix from time to time and so you end up with some potential conflicts if a user performs various actions like doing a post operation but then something changes during the post operation or during the applying phase and this functionality will be regularized and move to the control plane and that's what this is going to be able to do and now apply is invoked by sending a certain content type with the verb patch on this one and you can read a lot more about this one with inside of this particular enhancement issue it's been around for a while and so always good to have a little bit more eyes on it if this is something that is interesting you as well the last one here with inside of this section is deprecating and remove the self link function here and there's been no use of self link for a while so there's no compelling reason for having self link field in there so when modifying or reading an object from the API server self link is set to exactly the year old that it was used to perform that operation and so as a part of making sure that we are following the Kubernetes process for demoting and deprecating features this field will be deprecated in one year all right moving on to SIG cloud provider so as everybody kind of knows that the entry cloud provider implementations are being removed in the future and this is going to evolve a large amount of code that is used in many places with inside of Kubernetes and entry in itself so in order to prepare this to eventually be completely vendor agnostic it's going to be helpful to see what that removal actually entails and to verify that Kubernetes will continue to function without it and doing so it's a it's a bit tricky without ensuring that this entry provider code is not being used in some unexpected fashion such as a side channel during init methods or anything like that so what this is going to be doing is building Kubernetes binaries without the entry cloud provider packages and this is going to allow verification and provide additional experimentation to smaller and cheaper binaries for anybody that's interested in just using out of tree providers or perhaps maybe using no provider-based clusters and so the goal is to enable Kubernetes without entry cloud providers and without forking so this helps test out out of tree providers with the simulation of the future removal of the entry code and enable experimentation with in cleat sorry with cloud provider-less clusters so if you're doing cloud provider-less clusters it'd be cool to kind of see some of those use cases out there so if you have something feel free to share it on the Kubernetes community i think that'd be really cool to see so SIG cluster lifecycle so kubadm for windows is net new for alpha so anybody that so maybe we're jumping the gun with windows a little bit but you kind of get to see something that's happening here because back in Kubernetes 114 was the official support added for windows containers however up until this point there's been no official tool to actually join windows nodes to a cluster except going through a series of PowerShell scripts and so this solution is going to be a part of actually making some more of these scripts but writing their config files at the same time and that's provenly cumbersome and a huge pain point for some of the windows adoption you know on linux kubadm is quickly able to join nodes to the cluster and the intent of this is to propose a design that implements some of that same functionality for windows so you're going to see a PowerShell script to install and run on Kubernetes prerequisites for windows nodes and it should also be noticed at this time that this document and this this enhancement proposes the enablement for support for windows worker nodes utilizing kubadm and you're going to see this as this continues to progress so if you are looking into utilizing some more windows nodes workers make sure you go and check out the documentation's already been updated so you can kind of get started using kubadm for windows already and so if you are also using kubadm to build your clusters and perhaps you want to do some more tweaking and configuration you can now do that utilizing customize as a part of that and because kubadm to date only allows you to define a limited set of configuration options for the cluster via the component configurations or the corresponding CLI flags and this really does provide an abstraction to define configuration settings at the cluster level and it defines a limited set of configurations at the node level using these different types of options as well and this is this abstraction does work well for most common cluster configurations and use cases but there will be other use cases where the kubadm components just won't be able to take care of it such as possible like today you can't add sidecars you can't alter the timeouts for liveness probes and liveness probes inside the control plane components and so forth and so this is going to introduce a new feature that will enable you to be able to have full control of static pod manifest generated by kubadm at the node level in itself versus the kubadm component configuration that allows mostly cluster wide configurations on the control plane as well as the ecd arguments as well and so this new enhancement should not be required to manually alter any static pod manifest that are stored with inside of ecd kubadm manifest after the kubadm in it and join so you can utilize customized and make that happen now for you. Moving on to SIG instrumentation so there's a huge metrics overhaul that's now taking place inside of 116 and the number of metrics with inside of Kubernetes today there's a lot of them that just do not follow the official Kubernetes instrumentation guidelines and this is for a number of reasons some of the metrics were created before the guidelines were implemented which happened around two years ago some of it is just missing it and code reviews but beyond that some of the instrumentation guidelines there's several violations of the Prometheus instrumentation best practices so in order to have consistently named and high quality metrics this is going to start overhauling and making those metrics being exposed by Kubernetes consistent with the rest of the ecosystem and so what we even moving further beyond this you can actually start thinking of how can we even join these metrics which shouldn't be as difficult anymore as well so one of the examples I'll kind of go through here some of the you know there's a number of components that can use a SIG so one of the other mechanisms to ensure that you don't want to panic or anything like that is when registering metrics instead of a metrics registry it should be passed to each component in order to explicitly register those metrics instead of through init methods or other global executions or something like that and so this is going to make a step closer than having those stability guarantees for Kubernetes around those metrics on the C advisor side there needs to be consistent labeling the API server histogram latency runs from about 125 milliseconds to eight seconds and this range doesn't accurately model most of the API server request latencies which could run as low as one millisecond for gets or as high as 60 seconds before hitting the API server for global timeouts and other things like the scheduler proxy and API server all need some sort of conformance changes as well for cleaning out some of these deprecated metrics and we'll also got to kind of see that the client go metrics needs its work queue metrics to follow the Prometheus best practices in naming conventions so if you are on the outside looking in from a metrics standpoint and you are pulling metrics from Kubernetes this is something that you should probably take a look at to see if your product or feature anything like that can utilize this as well moving on to SIG network since we already hit the IPv6 and IPv4 bull stack we'll move on here to looking at the finalize the protection for service load balancers this is graduating to stable and or sorry to beta in 116 and this is created because there are various cases where a service controller can leave orphan load balancer resources after these services are deleted and it's worth to note that it's better to have a mechanism to ensure the cleanup of these load balancer resources and so this is where this finalizer service comes in and this will be attached to any service that has the type load balancer if the cluster has the cloud provider integration enabled so upon the deletion of that particular service the actual deletion of the resource will be blocked until this finalizer is removed and this finalizer will not be removed until the cleanup of the load balancer resources are considered finished by the service controller itself the endpoint slice api this is net new alpha 116 as well and in the current endpoints api one of the object instance contains all of the angel endpoints of a particular service so whenever a single pod in a service is added updated or deleted the whole endpoints object whether or the other endpoints didn't change or not is recomputed and written to storage with inside at cd and then sent to all the watchers like cube proxy and this leads to two major problems so first is storing multiple megabytes of endpoints it can put strain on multiple parts of the system because you don't have a paging system and it's a monolithic kind of watch storage design and the number of max endpoints is bound by the kubernetes storage layer which is at cd and that has a hard limit on the size of a single object which is 1.5 megabytes by default and this means the attempt to write a larger object when that limit hits it will be rejected additionally there's a similar limitation in the watch path in the kubernetes api server so for a kubernetes service if its endpoints object is too large that endpoint update will not be propagated to the cube proxies and thus ip tables and ip vs won't be reprogrammed and so you're going to have performance degradation in large kubernetes deployments if this happens too so not being able to efficiently read and update these individual endpoint changes can lead to elements not actually being able to basically have their endpoint operations happen so one consideration here is if you put the watch into the picture here the situation becomes even worse as the as this traffic gets multiplied further with the number of watches that you create too and so this new endpoint slice api will support tens of thousands of back-end endpoints and a single service on a cluster with thousands of nodes and it's going to move the api towards a general purpose back-end discovery api so moving on to sig node the fremel containers now if you look at the issue down here it says 277 we've moved on we've passed the 1000 issue mark with inside of the enhancements repo so seeing something that has 277 means that this has been around for quite a long time and it's just now being introduced as alpha with inside of 116 and so what does this do well for many developers and some of the native kubernetes applications out there you want to treat kubernetes as an execution platform for binaries that are produced by any type of build system so you can forego scripted os installation of traditional docker files and instead you might want to copy the output of the build system and from a container image to say scratch or a distro list container image and this gives advantages of having minimal immutable and smaller sized images now the disadvantage of using containers built from scratch is the fact that binaries provided by this don't it makes it difficult for troubleshooting these types of containers and for ops folks it becomes the case that a person troubleshooting the application is not necessarily the person who built it so people who want the ability to attach a known good or automated debugging environment to a pod can now do that as well so looking at pod overhead as something else being new in 116 we all know that pods have some resource overhead and in our traditional Linux container or docker approach the accounted overhead is limited to the pods container but that also invokes some overhead that accounts to various system components including the kubelet for control bloops you have docker you've got kernel for various resources you got fluent d for logs and this current approach is to kind of reserve a chunk of resources for the system components and ignore the overhead from the pods container but this doesn't scale well well with sandbox pods the pod overhead potentially becomes much larger maybe a hundred megabytes and for example Kafka may run as a guest kernel it's got its kata agent in its system so on and so forth and since this overhead it's pretty big so it's hard to ignore so we need to account for it and starting from quota enforcement and scheduling and so this new feature is going to be able to take care of that and actually have that overhead in there as well so the node topology manager is also new within 116.2 and to date multiple components within the kubelet make decisions about these topology related assignments with inside of here the cpu manager it can make decisions about the set of cpus containers a lot to run on it's only implemented as its policy from kubernetes 1.8 as a static one but that doesn't change the assignments for the lifetime of the container and then you got the device manager and that makes device assignments to satisfy container resource requirements so generally devices maybe they're attached to one peripheral interconnect so if a device manager and a cpu manager misaligned all the communication between the cpu and the device can incur as an additional hop over the processor and you're going to experience even more latency and you got things like c and i you've got nix that includes single root i o virtualization these kind of virtualization functions have affinity to particular socket and those have measurement performance ramifications if it's not done properly and so the goal here is to create a preferred socket affinity for containers based on the input from the cpu manager and the device manager and provide an internal interface that they can now integrate to be more topology aware and the kubelet is going to be able to take care of some of these components too so for example if a user asks for a fast network in a virtualized environment it will automatically get all the various pieces coordinated and co-located on a socket like huge pages cpu sets and the network device as well or perhaps if you're doing say neural network training a user can ask for an accelerated device and perhaps a number of exclusive cpu's in order to get the best training performance and this is due to socket alignment of the assigned cpu's and the devices too and for that accelerated device there could be a number of exclusive cpu's in order to get the best training performance too and again this is all going to do to socket alignment and having the cpu manager and a device manager kind of working in lockstep here so add startup probe liveliness probe hold off for slow starting pods i know it's a long one but this is also new in 116 and this is because you know we have slow starting containers and those refer to ones that require those just a significant amount of time to start there's could be because there is data initialization because maybe the first startup takes a lot of time or perhaps it's just a heavy workload and every startup takes a lot of time or perhaps it's just an underpowered or overloaded node and that startup time is dependent upon external factors of actually having processes available and the main problem with this kind of container really where it stems from is that they need to be given enough time to start before having the liveliness probe fail and that failure threshold time and that which again is going to trigger a kill by the cubit before having a chance to be up and so there's ways that we can now handle the situation with the current api but none really provided an answer to actually having containers stuck in deadlock and so what we can do is now we can have this liveliness probe hold off for any kind of slow pods that are slow to start up now so runtime class scheduling and this is actually moving to beta 116 in the initial runtime class implementation it was assumed that the cluster nodes were homogenous in regards to runtime classes but it's still possible to run a heterogeneous cluster and pod authors would need to set appropriate node selector rules and tolerations that your pods landed on those particular supporting nodes however it's become clear that heterogeneous clusters will not be uncommon and you need to support a better user experience here and the introduction of windows nodes kind of presents any use case for heterogeneous clusters because some nodes are going to be running supporting windows while some are supporting linux there's also the inherent differences in the operating systems so it's natural that you want to support different runtimes so for example windows nodes may support hyper v sandboxing while linux nodes support cota containers and every native container support is going to vary on each so with run c for linux run hcs for windows and perhaps some users just wish to keep sandbox workloads and native workloads separate this is again just another example of being able to create this the scheduling so they can kind of create within side of a single cluster if you need to do that as well so moving on to sig scheduling so within side of kubernetes we now have even pod spreading as a 116 alpha feature that's coming new so in kubernetes you have this affinity that's related to directors that you want to control how pods are scheduled and maybe they're more packed or more scattering however you want to make that happen but right now you're pretty limited in your options for pod affinity so you can have infinite pods be stacked onto a qualifying topology domain or you can have them scheduled onto a single topology domain this can't be an ideal solution especially if you know pods evenly across different topology domains and you could do this because you want high availability or perhaps cost savings and maybe you want to do regular rolling or updates or scaling out replicas and if you do that previously it could become problematic and so the goal is to have even spreading and this is calculated amongst the pods instead of the apps api such as deployment or replicas and so this can now be either a hard or soft requirement so as an application developer perhaps you want your application pods to be scheduled onto a specific topology domain as even as possible and that current status is that the pods may be stacked onto a specific topology domain or maybe you want your pods to not coexist with specific pods and you can do this say such as anti-affinity and in some cases it might be favorable to tolerate violating pods but you can be able to say that whether it should or should not okay so here's another long one here so extending requested to capacity ratio priority function to support resource bin packing of extended resources again net new alpha and 116 here and what we want to be able to do here is we want to run workloads on Kubernetes which use some sort of accelerated devices and we want that default scheduler to spread pods across across them resulting in you know we don't want to have a fragmentation of these extended resources and today you're going to have that because they may potentially remain in a pending state and so what they're going to do is they can schedule pods using a best fit policy and this is using the requested to capacity ratio this priority function for extended resources so as the graphic shows the default scheduler in most cases will schedule the pods as follows if there is no priority function for an extended resource for bin packing the scheduler should submit two resource jobs on note three as utilization gets higher and this would reduce the fragmentation of an extended resource and reduce the pods in a pending state all right moving on to six storage we're chugging right along here so adding resizing support to CSI volumes this one should be relatively easy to understand but basically it just means that we can resize volumes with inside of CSI plugins now these again have to be implemented with inside of CSI and making sure that your CSI adapter is taking part of the spec and then your operation or your persistent volume can now become resizable next is looking at inline volume support i'm also going through these because a lot of these we hit in 115 previously this is also graduating to beta in 116 and we look at inline volume support because currently in volumes that are backed by CSI drivers can only be used with persistent volume and persistent volume claim objects and so this proposal and this feature is really implementing the support for the ability to nest a CSI volume within pods for ephemeral style ephemeral style drivers so they can be used to inject arbitrary states such as configurations secrets identities variables or any kind of information that you want directly inside of the pods using a mounted volume we also have support for CSI plugins on windows nodes so of course this is net new and alpha we can see more things happening but inside of windows and so what we want to do is we want to see this continue to progress as well so previously for persistent storage requirements for windows you had to depend on power shell based flex volume plugins that maybe they were maintained by Microsoft or somebody else and then they were also only being used over Samba or S&B or iSCSI protocols or you had entry plugins for Kubernetes with inside the core but as we all know this is starting to become deprecated and we're trying to move things out of tree so support for CSI in Kubernetes reached GA status in 113 windows was in 114 so the goal is to make sure that window nodes also support CSI plugins so they get all the benefits that these this ecosystem has to offer for persistent storage requirements and to make sure that windows nodes are seen as a top tier plug or sorry top tier host as we start going through here and being able to take advantage of these plugins over time all right moving on to SIG windows just a few more slides they go here so support for gmsa for windows workloads is moving into beta or group managed service accounts and this introduction actually happened in Kubernetes 114 and this spec relies on custom resources that needed to be specified through annotations at the pod and the container level and the windows implementation of the Docker shim and the container runtime interface and the low level OCI spec can already handle a username instead of a UID and this is interpreted inside the container to create in process as the intended user this however is not surfaced as a field in the pod container spec that an operator can specify and so this is going to give you the ability to specify the desired username in the pod container spec as fields and be able to pass them as configured windows runtime or pass them on to the configured windows during runtime also net new and 116 is being able to run as username for windows so this api enhancement to to here will actually capture windows os specific security options from the perspective of the windows workload identity and containers so we already talked about the ability to sort of talk to have the gmsc sorry the the gmsa but what this is going to be able to do is it's going to allow you to cover these fields pertinent to it and actually run the username which is be able to execute that container entry point so again kind of running in tandem with the last slide this is not inclusive of gmsa functionality but we'll support it later as well so what's coming next so the 1.17 release of kubernetes is already in progress we're already a few weeks into it the enhancement freeze has passed that was back on october 15th if you're interested to kind of see exactly what features will be going into kubernetes 117 there is the link to the tracking uh spreadsheet right there the targeted release for kubernetes 117 is also looking at december 9th 2019 as well just in time uh should i say right after yeah after kubernetes so if you are interested in learning more about that you can always always always check out sig release to kind of get an idea of what's happening with inside of the release time frame and the release windows next i don't see a whole lot of stuff in q&a which means locky answered a bunch of stuff but if you have any other questions now's the time to ask is there going to be recorded version later for viewing yes this is all being recorded it will be available on youtube at a later date andrey is asking is there any actions being taken from kubernetes team regarding the incompatibility with helm as reported in helm's github the sig release team doesn't do anything directly with helm that is outside of sig release so there's nothing that we will be able to answer with regards to that kenny i'm trying to find that issue just out of interest i suspect it's because of the api deprecations um but i'm trying to dig it up um so so those apis were marked for deprecation and they went through the deprecation policy which was three releases so back in 1.12 or 1.13 we we said those apis were deprecated and then they were subsequently deleted um helm had the opportunity to update their baseline installers to use the updated apis which have been around since 1.9 or 1.10 i believe so i can i can dig into that and answer it there's another question about any chance kubernetes comes with its own package manager um i don't particularly see anything like that but i don't want to speak on behalf of the community uh locky any idea there too now okay with that i don't see any other questions available or out there so thank you everybody for joining in again this will this recorded session will be available later on on the cncf webinar page which will also be available on youtube thank you again everybody and enjoy the rest of your week thank you everybody