 All right. Hello everyone. Welcome to our talk, Cloud Foundry's new .NET capabilities. I'm William Martin. I'm Pivotal's product lead for Microsoft Ecosystem, so .NET and Windows and Azure. I'm also the CFF project lead for Garden Windows and Bosch Windows. I'm Sean Neal. I'm a Solutions Architect with Pivotal. So do a lot of work with .NET and Cloud Foundry. Work with William a lot. Yes. So thought we'd review the entire landscape of Cloud Foundry's Windows capabilities, particularly oriented around .NET. So it really starts from this guiding principle, that Cloud Foundry is a standards bearer for Cloud-native practices and is really uniquely positioned to provide an open platform for .NET and Windows. So a few years ago, the Cloud Foundry teams identified an opportunity to make Cloud Foundry that place where modern .NET practices could evolve. So far, we have three teams that contribute to that effort. Garden Windows under the Runtime PMC, Bosch Windows under the Bosch PMC, and Pivotal has a Kubernetes contributors team that contributes Windows support upstream into mainstream Kubernetes. So, but starting with Cloud Foundry application runtime, it really does ask a lot of .NET in a way. Cloud-native .NET really hasn't really been a thing until the last couple of years. In fact, the ecosystem around Cloud Foundry is really around open technologies to start with, things like Node and Python, PHP and Go and so forth, and to say that .NET and Windows support can be injected into the Cloud Foundry ecosystem. I think says a lot about the state of .NET becoming more open, but also the ability for the technology to support both ecosystems. But the idea of supporting a been deployable 12-factor application for .NET, is really kind of a new concept, at least a new concept at the time. But the benefits of scale and stability of the platform are really tangible when you can achieve that state for a .NET application. A lot of the ways we look at this, is to think about the state of an enterprise, for example, that has invested over the last one, two, or three decades in Windows and .NET technologies. What ends up happening and what we find is that they end up with this kind of miscellany of technologies, everything from old .NET framework apps, .NET 2.0, .NET 3.5, and whatever, and a number of different versions of Windows Server. Windows 2008 is a big one this year, since it's at its end of life of support. But you might end up with a portfolio that includes 2003 and 2012 R2, and some tight coupling between those apps and those servers, and then all these other technologies on the left. But on the right-hand side of the slide, it represents the future, represents everything that Cloud Foundry and the open-source ecosystem has been talking about for years, like Microsoft created .NET Core, we keep talking about microservices, Cloud Native applications, Kubernetes is in this wave right now. But there's this real gap, this chasm between the ecosystem, that's the reality that businesses and enterprises are facing, and the future that's being marketed now, and that we're moving towards in such a rapid pace. In the Cloud Foundry ecosystem, we really believe that Cloud Foundry application runtime, and container runtime is the bridge to get businesses from that world they're in now to the future. So this is what we had to do to get to the state that we're in now. It started in late 2005 and 2006 with a team called Greenhouse, which was really the MVP of Cloud Foundry being able to enable CF push for Windows applications and .NET apps. This truly validated that .NET could run on the platform in this 12-factor state, and it also validated that developers loved the experience. This was Diego Windows release, it shipped as an MSI that you could run on a manually deployed Windows VM and effectively hook it up to a running Cloud Foundry instance, and then CF push your apps by specifying a new runtime stack. So while a great developer experience that Cloud Foundry is known for, it was a really terrible operator experience. So in 2017, enter Bosch Windows, and this is really when Windows entered this realm of immutable infrastructure and infrastructure is code. In enterprises, Windows typically deployed either bare metal or in a VM and quickly turns into a pet, and then quickly turns into a snowflake. This is really a paradigm shift for managing Windows outside of a management infrastructure like Active Directory. So Bosch managing Windows VMs, treating them like cattle, being able to destroy a Windows VM when it becomes unhealthy automatically and resurrect it. It's a totally new way of thinking about how to manage a fleet of Windows servers. Finally, a platform engineer could deploy it and operationalize it and automate that lifecycle in really fundamental ways. So this is where Windows stem cells came about. You could Bosch deploy Cloud Foundry with Windows Diego cells attached to them, and Greenhouse then split, split into the GardenWindows team, many of whose engineers are here in the audience today. Under the Runtime PMC, and we also created a corollary Bosch Windows team under the Bosch PMC. There's this bonus as well. Like now you can CF push with a build pack, which is pretty amazing. So 2018, containers hit the scene. So while Windows 2016 really created the Windows container, well until 2018 until we really got to a state where we thought that the new runtime built on Windows containers was GA. So this is the creation of technology that's really analogous to what Linux looks like. Then finally, what we wanted to talk about today, and what we're going to emphasize today, is all the features and tactics that we've developed over the last year or a year and a half or so to really run Cloud Foundry in the enterprise and run it efficiently and well. So all the highlights that you see in the yellow there will be the focus of this talk. So this is a slide that just shows the breadth of capabilities that Cloud Foundry application runtime with Windows supports now, and some of these actually the Garden and Bosch Windows teams have already presented in previous CF summits when they were created and some of them are new like the SMB mounts and support for trusted certificates. What we're going to talk about today are techniques and tactics that really make this real and really make this enterprise ready. So as with many things with Cloud Foundry, it really starts with the application itself, and so we'll start there. Yeah. So my goal of this part is really show you that a lot of.NET applications can run on Cloud Foundry pretty easily. First of all, you have to step back and look to your portfolio of applications that you have. It's a continuum from, I have really modern.NET Core applications that are going to run on Linux to all the way to the other side of, this is not something I want to continue running and I want to divest on that. Maybe that's a good target for rewriting in.NET Core, but something that still provides value to your enterprise. There's this middle area where you have applications that with some modernization would be good targets things that you're actively developing and continuing to provide business value that would be good targets for Cloud Foundry on Windows. Then there's also some other ones that they're still providing value. Maybe you're not doing active development on them, but might be better targets for Kubernetes because they have or they have dependencies on certain drivers or other esoteric configurations that maybe don't fit well with Cloud Foundry. So one of the first things you have to do when you figure out, hey, I have an application and I want to make this work really well on Cloud Foundry on Windows, is your 12-factor app, which I'm sure you're all familiar with, but specifically for.NET apps, there's three that I highlight here that are very important that you have to pay a special attention to always. Because some of them you get out of the box with Cloud Foundry, and other ones you get out of the box with.NET and IIS or Hostable Web Core. But the dependencies being, I don't have an application that requires something to be pre-installed on my root FS. So anything that you have to any dependencies, the application have have to be been deployable. So that's one of the biggest requirements for our 12-factor app to be hostable on Cloud Foundry. Configuration is something you always end up have to look at. At the bare minimum, you have to update it to work in the new environment, so updating connection strings and maybe even going a step further with Steeltoe and doing config server. Then stateless processes, that's really specifically around session state management, maybe instead of using the in-process session state manager, you change it out to use Redis or something distributed like SQL Server. There are lots of different types of.NET application. So you're wondering, well, I have a web forms app, or I have MVC app, does that work on Cloud Foundry? Yes, absolutely. So I would say, web API and MVC applications are really prime targets to work on Cloud Foundry on Windows runtime. They don't really require a lot of changes most of the time because they tend to be more modern, less dependencies, unit tests usually around them. Those are also possible candidates that you might want to actually modernize without a whole lot of rewriting to work on .NET Core and maybe target Linux with those. Web form apps though, however, those are usually bigger monoliths without tests. Usually, they've been around for almost 20 years sometimes. Those are probably with some slight modernization, good targets to run on Cloud Foundry on Windows versus having to maybe use like strangle the monolith patterns with that and rewrite parts of it as you go along in .NET Core. Windows services, those don't work in Cloud Foundry, but with a little tiny bit of work, you can convert them to a console application, and you can host those in Cloud Foundry using the binary build pack. There's some other ones like Cots applications and things that are stateful. Those might be better targets for Kubernetes. But let's get a little deeper. What's it you actually start and you start pushing your applications out like, what does that look like? What things you actually have to do tactically? CfLogs works just like anything else, the Java app, .NET app doesn't matter. You have CfLogs available to you. However, you do have to make some changes probably to your application. You need to make sure you're writing a standard out or standard error. A lot of times you're already using a logging framework like Log for Net or Siri Log. That means changing out the appenders to target the console or standard out so that they log show up automatically in Cloud Foundry. Some other good things to do is make sure you have a global error handler. So if your application crashes, you can find out why it's crashing. A CfLogs provider is a good way if you need to dynamically change your logging. For instance, you might want to change it to run a debug mode instead of always running in debug because I've heard that's a bad idea. A common requirement for applications tends to be Samba shares, tends to be a whole suite of applications out there. It seemed like they do a lot of passing files around and processing batch processing, ETL type processes. So just recently introduced Samba share support so that you can actually mount a Samba share from your container, your application instance. There's actually a few different ways to do that. One of them is straight up net use or PowerShell. But typically you want to put whatever the mounting is in the profile.batch file that runs when you push the application to ensure it mounts it. Another way to do that is using a steel toe library that will actually mount it and you get the credentials from another party like maybe Cread Hub or something. Now, if you were to run into problems, which I'm sure none of you ever have, but let's say you do run into problems through application, the typical features that you might use in IAS and whatnot also work in Cloud Foundry. So you can turn on dot net tracing. You can configure the tracing profiler or to write out to the console. So that's useful for instance, if you're using some of the system diagnostics tracing or if you're using libraries that have already been instrumented in the dot net framework. So for instance, like making a web request and you're like, well, why doesn't my application work properly when it makes a request to this API? You can turn up the trace profile and dump out and actually see the request and responses and help diagnose issues. Obviously the custom errors mode, so like maybe your application just doesn't start at all. Go in the web config, turn custom errors mode off. And so you see the yellow screen of death. You'll only do that in temporarily, don't do that in production. And then Steeltoe has a management set of end points that would really help you. And I recommend you try them out or at least look at them. They have like tracing, request tracing, health. They have some end points for getting thread dumps, keep dumps from your application. And they also have metrics. So they'll actually emit some of like the garbage collection and other metrics directly to the fire hose. But maybe you need to do a little further. So CFSSH, that works now and you can directly use SSH into a container. First thing you wanna do is make sure you run PowerShell because command.com is not very powerful, let's be honest. Like that's why PowerShell exists. So start PowerShell up, you can use all the power that comes with that. You could also instead of directly SSHing in, you can also do some port forwarding, which might be useful if you wanna use the CFSH transport to connect your Visual Studio debug or to your application instances, actually running in Cloud Foundry on Windows. And so some things you might do once you're in the container, for example, you can curl an endpoint or WGIT, just that use basic parsing. You're gonna probably need to remember to do that because if you don't remember that, you'll get some weird error about you need to accept some sort of, I don't know, some GUI needs to pop up somewhere on server core which doesn't exist because you're not running a GUI. And so that's what that's for. Testnet connection, so there's no net cat or telnet installed by default. So testnet connections away through PowerShell to validate, hey, I can connect to this port. So that's useful if there's firewalls in your way, things like that. You could also view files through catting a file. You can actually see what's been deployed onto the actual application instance and view. Like, hey, what did my continuous delivery system actually emit for my web config and what was actually deployed? Or did I ignore something that I shouldn't have? Great, I always knew that there was a reason why PowerShell was called PowerShell. Yeah, that's in the name. So we wanted to jump into a little bit about how Windows works on a Cloud Foundry application runtime on CFAR. And this probably starts first with the version of Windows that we support on it. Windows Server 2012 R2 was the first version of Windows that we supported. This used a kind of pseudo containerization framework called Iron Frame that was built around job objects and file system ACLs and user management. The current runtime still supports this, but those of you who are running CFAR anywhere, should realize that we have plans on retiring the Windows 2012 R2 stack by the end of this year. So the rest of this should be relevant to show you a little bit about what the future looks like for Windows support on CFAR. Windows Server 2016 introduced Windows Containers as a technology. We didn't support this on CFAR for a couple of reasons. The networking service wasn't really mature yet. The container base image was four and a half gigabytes, which made the file system like five and a half gigabytes, which was pretty hefty, and actually challenged some of the default settings for some of the blob stores inside Cloud Foundry. But Microsoft started shipping versions of Windows every six months, actually feature releases. This new channel they call the semi-annual channel releases, which comes along with your licensing for the long-term servicing channel, Windows 2016 or 2019, assuming you have the right level of support purchased. So this is really what enabled Cloud Foundry to take off with a containerized runtime for Windows. So there's support for Windows Server 1709, and then 1803, which shipped six months later around early May of 2018. And this really reduced the size of the container image to a really small level, and we were able to leverage more and more of the container features as they shipped. But finally, Windows Server 2019, the next long-term servicing release that has a 10-year support policy from Microsoft finally shipped. This is the official Kubernetes-supported release for Windows. It's effectively a more stable version of Windows like as far as the container APIs are concerned. So, and then finally we've just pushed support into CFAR for Windows 2019. So that's gonna be largely what we're talking about moving into the future. But if you're not familiar with CF deployment and how it treats Windows, it's pretty simple. You add an ops file and you can just like deploy Windows Diego cells right alongside the Linux Diego cells. When you CF push, you just say dash s Windows and that gets scheduled to Diego schedules that to the Windows cells. It's all this and the emphasis on some of these slides is really the fact that there's a ton of common infrastructure between the Linux side of the house and the Windows side of the house. We're not actually building all new things. We've just building the smallest that it takes to enable CloudBoundary to push that application and run it in a container on a Diego cell. So you'll note some users in the past that the stack just says Windows. This is a new introduction. The stack is now currently called Windows 2016. So we'll be retiring that stack soon but it really does exactly the same thing. So it's just renaming the stack but it doesn't do anything different. So this is a little bit behind the scenes of what this does. This is the same life cycle that happens on Linux. This describes in the lower left-hand corner the CF push experience. It's one thing to note, you actually build your application in Visual Studio first before you push the application. CF push with the stack. Of course, all of these command line arguments are supported in the manifest file. And then that with the build pack life cycle creates a droplet that then schedules that in a genuine Windows server container. And oh well, the life cycle supports build packs. So there are a few build packs to be aware of when you're working with .NET and Windows apps. First is the HWC build pack which stands for hostable web core. That's effectively all of the framework that's inside IIS that handles web requests. So this is a much lighter weight framework for us than running full IIS. But that's effectively the framework that we hook into with our little web server inside a container to run full .NET framework apps on Windows. There's a binary build pack for Windows as well. This is for running .NET core applications on Windows cells. A lot of enterprises still have dependencies that can only be installed on Windows. I think IBM MQ is one that we've run into recently. So if you publish your application as a self-contained app and push it with the binary build pack, you'll be able to host that .NET core build pack on .NET core app on Windows. And of course supports binaries and console apps as is the name. The .NET core build pack that you find in the cloud foundry of build packs library is for running .NET core apps for Linux. This is generally what we recommend for modernized green field apps on running on cloud foundry. But just recently our teams have implemented multi build pack support for the HWC build pack, which has enabled some APM vendors like New Relic and App Dynamics to build integrations with cloud foundries. So now a developer can push with the HWC build pack and the New Relic build pack and that automatically installs the agent inside the container when that app is scheduled, which is pretty cool. So if we dive a little bit deeper, like all the way down the stack, this is the anatomy of a Diego cell for Windows and it looks surprisingly like the anatomy of a Linux Diego cell. And in fact, it's exactly the same except right at the bottom of the Guardian stack where instead of running RunC, we run an equivalent container plugin that the GardenWindows team authored called WinC, which calls into the same APIs that the Docker daemon calls into on Windows. So we don't run Docker in Windows Diego cells, but we use the same integration shim and call into the same Windows APIs, the host container service and the host network service. So in effect, all of the work that Diego and Garden and Logurgator does and Bosch does for Linux also gets applied to the Windows cells, which is pretty darn cool as far as being an open platform for .NET and other non .NET-y things. Diagram that shows the anatomy of a running container on Windows. This is, there are a couple of things to note here. One is that it's a genuine Windows server container. Another is that the base image comes directly from Microsoft. So you won't find any Cloud Foundry repos that distribute that base image. That always comes from a Microsoft repo. Another is the fact that this is OCI compliant. Another is the highlighted feature here that we'll talk a little bit about later, which is a new feature enabling a lot of enterprises to go to production, which is injecting the platform trusted certificates into the root file system in a way that maintains the security posture of the container. And finally off to the right to go back to that point about the base image coming from Microsoft. There are two releases that you'll see available. One is the online root file system or root FS, which when you deploy Cloud Foundry will automatically pull that container base image from the Microsoft repo and inject it into the root file system and then it'll sit on your Diego cell ready to use. There's also an offline root file system, which actually pivotal ships in our commercial offering of Windows support, which enables you to run a command line command to manually pull that container base image and then create that file system release manually. So the advantages of that are that you can do that in a pipeline or in your CI CD or whatever deployment mechanism that you use kind of outside of the platform if you're working in an air-gapped environment. So, but you can run Windows in an enterprise, in production and we'll go through some of those troubleshooting details now. Yeah, so I guess my point will be that running Windows cells is not unlike running Linux cells. If you're comfortable running Linux cells, you should be able to successfully run Windows cells in Cloud Foundry. So there are stem cells publicly available for the public liaises. You can build your own for if you're using vSphere or OpenStack. So the CF deployment has ops files for, oh, and deploying Windows cells. If you wanna pull those in, is one of those ops files, looks like that. You upload your stem cell, do your botched deploying, include your ops file, I'm gonna fly through these. Some common botched add-ons, so there are botched add-ons and you should be using some of these for example, the Windows utilities release. For now, you should definitely be including the enable of the Bosch SSH agent. I think that's very important. The other ones maybe not as important. The syslog release, that's as important too, if you wanna have your Bosch job logs show up and the fire hoses are as well as your event logs show up too. So those are good ones that you wanna be aware of. The thing to also realize about Bosch add-ons in general is that make sure you're targeting the right OS and you're not trying to run one of these on your Linux cells and vice versa. So one of the new features that we alluded to here was trusted certificates. So that's a very new thing and some very smart people at Pivotal actually figured out a really novel way to actually get these injected into the system trustor. It was actually pretty neat. So it actually, what it does is it actually spins up a container and then it applies this PowerShell certificate. So when you have a certificate, you actually need to have it on disk and in the registry which requires the elevated administrator level privileges to do that. So once that's run, so essentially the container's running not as VCAP but as an elevated user, it runs this diff exporter which creates a new OCI layer and it effectively uses the Hydrator to create a new root FS on each of the cells. So at that point, that's baked into the root FS. So let's talk about cell troubleshooting a little bit. It's, if you're comfortable doing on Linux like I was saying, you should be able to do it on Windows just fine. So remote desktop is an option, I would say probably don't use that but maybe use Bosch because we're here at Cloud Foundry Summit, right? Let's use Bosch for things. Not only should we use Bosch, it's a better experience honestly. It's gonna be consistent between your Linux cells and your Windows cells. So it looks very similar to how you would SSH into a Linux cell, deployment and then the cell, right? And then, oh, one other extra step, PowerShell because you wanna have a real shell once you get SSH then, you can do things like grab event logs. So this example grabs them from the system log, looks for error entries. You can expand upon that and actually get the full message by index there. So that's actually taking one that I found that looked like it might be interesting and actually expanding out to see the full error message. You can also rep for errors or other strings within your log files. So select string kind of effectively the equivalent there. So this would search that particular file, stall log for a particular pattern called error but you can also search through a bunch of log files. So here's a way to do that, very similar. And then it also pipes it into select path which says here's all the files I have found this string in the past and then you can go dig into this further. And then you can also follow tail log. So example here for following a log, the wait command essentially sits there and waits and follows the log as it gets written to it. And then tail says just show me the last five. And then editing text though is not something that you can do from the command prompt by default on Windows. So that's a little unfortunate. Ideally all your changes are gonna be in source control anyway. So you're not actively changing things on a cell but in case you have to, a good way around that is maybe download nano. So you can just W get that and then run that and edit a file just a single binary. Again, we've talked about test net connection again because there's no net cat or telnet. And so in the last minute we're gonna talk about the future of Windows on Cloud Foundry and some of the more promising present. So remember this diagram before that's kind of this decision tree that a customer might use to evaluate the technologies in their .NET and Windows portfolio. Well, the two boxes at the bottom are Linux and Windows VMs in CFAR and CFCR. So you might notice on the far right hand side CFCR has a Windows box. What does that mean? Yes indeed, you can deploy Kubernetes clusters using Cloud Foundry container runtime that have Windows workers in them. It's embedded into KUBO release and KUBO deployment. It deploys a Linux based master with Windows workers alongside which effectively lets you author a Docker container based on Windows and then Kube control apply it to a Kubernetes cluster. So just announced, I think like a week and a half ago that Kubernetes 1.14 has shipped with stable Windows support. So this is the next thing on the CFCR Windows Teams backlog. It's right there in on GitHub. You can deploy it in a similar way that you deploy CFAR with a set of ops files. This one is built just for the current Kubernetes release. I think it's 1.13.5. So there's a couple of extra ops files here. The anatomy of a Windows container on CFCR it's pretty much whatever you put in your Docker file as opposed to something more opinionated like it is in CFAR. A couple of things to note are that there are a lot of Docker base images already available from Microsoft that include things like IIS that run either on Windows Server Core which is the big container image that contains the Win32 subsystem and is able to run full.NET framework or Nano Server which is like a 90 megabyte base image from Microsoft that just is designed to run Windows binaries or self-contained applications. So, and of course the advantages of a Docker file you can do anything you want. You can run things as container admin. You can configure things at will. Of course that comes along with it all the responsibility. So that's actually a disadvantage as well. That's that you can do anything you want in a Docker file. But on our roadmap, cloud native build packs that's been like all the rage today at this Cloud Foundry Summit definitely looking to contribute Windows support into that. Persistence on Bosch for Windows VMs so to enable those persistent workloads on CFCR and Kubernetes deployed with Bosch. Irene Windows, this is now going to be the future of CFAR container scheduling. So we definitely want Windows to be part of that story. And then finally Envoy and Istio routing on Windows. So the Garden Windows team has been contributing upstream to Envoy based on an original Microsoft fork there. And once that support gets fully accepted into the community and it's production ready then that's the gateway to Istio and so we can get some of that great weighted routing experience that Ui demoed earlier at the keynote. So anyway, hopefully we've convinced you that we've been building the future here of Windows on Cloud Foundry. This one? Sure. Go deploy Windows Ls. Any questions, anything, any comments or anything? Yeah, it'll be, we'll publish it on the website. On the Cloud Foundry Summit website, yeah. All right, thanks very much. Remember to reach out to us on Cloud Foundry Slack. You can find us in the Bosch channel with at Bosch-Windows and in the Garden Windows channel on Cloud Foundry Slack. So thanks very much.