 My name is Kunal Kundaje, and I'm a software engineer on the Cloud Platform team at Netflix. Today, I'd like to talk to you about our journey to reboot the Cloud Platform user experience within Netflix and how we're using Polymer as we go about doing that. So let's get started. Here's a quick overview of the topics we'll be covering today. I'll start with a brief introduction to the Cloud Platform Engineering Organization at Netflix and what we do. We'll then go on a quick whirlwind tour of a few different types of apps we've been building using Polymer, five apps in about five minutes. Next, I'll talk about Reboot, our internal component library, and some tooling that we built for a better developer experience. Then we'll look at the road ahead with Polymer 2, and I'll share an incremental approach that we're taking to migrate with minimal pain. And finally, we'll circle back to apps, specifically state management in larger apps, and some experimenting we're doing in that space. So Cloud Platform Engineering, who are we and what do we do at Netflix? Well, here's our team's charter at a high level. We provide a common set of foundational building blocks to Netflix engineers so that they can focus on core business value rather than recreating these infrastructure layers. So some examples of these building blocks include data stores like Cassandra, Dynamite, and Elastic Search as a service, Kafka messaging, and stream processing as a service, Atlas, our internal system for aggregating operational metrics across all of our services, and so on. So all of this sounds like a lot of server-side stuff, right? But all of these services that we provide as an organization become much more usable when we offer cell service apps that empower our engineers to get onboarded quickly and leverage the capabilities of our systems without roadblocks. Insight tools like dashboards and visualizations, together with control planes, also help our engineers detect issues and fix them quickly and effectively without having to switch between multiple tools and command line scripts when they get paged in the middle of the night. And building all of these apps is where we come in. So let's go on a tour of some examples of apps we've built for these various systems. They're all quite different from each other. So this segment should give you a good sense of the variety of app types that can be built using Polymer and Web Components. We'll start with Lumen. Lumen is our dashboard builder for operational insights. And it's widely used by engineers across the company to plot time series data collected and aggregated by Atlas, our metrics data store, so that they can observe meaningful changes in these metrics over periods of time. So here's an example of a time series dashboard that's used by our cloud databases team to track various Cassandra cluster metrics, like IO operations and latencies over time. So Web Components are a great fit for this use case, because we can have a simple public API for these graph components and encapsulate all of the logic to fetch these metrics and render them, and then enable anyone to drop these components into any other internal app that we have, including those that are built in other frameworks like React, Angular, and Ember, which we have within Netflix as well. But Lumen also supports more than just time series data. It also supports the concept of cell types, which are components that visualize the data that's passed into them in different ways. So for example, there are cell types for pie charts, histograms, bubble charts, data tables, what have you. And each of these is written as a custom element using Polymer. So here we can see a single real-time data source that's been connected to six different cell types to visualize the data in different ways. Users can also write custom data mappers, which are just JavaScript functions that take the data that's passed into them, and then convert that to a format that can be understood by any visualization type. So here's an example that shows data from Elastic Search being mapped to different cell types and visualizing that data in different ways. Next, let's take a look at Casper, a custom health check dashboard that we built for our fleet of Cassandra database clusters. So at a high level, Casper visualizes the entire health of all of our Cassandra clusters in the fleet as a tree map showing simple red, yellow, or green states. The sizes of the boxes are proportional to the footprint of the cluster itself. Obviously, the cluster names have been anonymized here. You'll often find this page projected onto a wall-mounted TV right next to where our cloud database team sits in the office. So what makes this dashboard quite unique from a front-end perspective is that the data source is a high-volume firehose of data being streamed in via web sockets from a job that's aggregating all of this health check data from over 10,000 different Cassandra cluster nodes. So in order to not completely lock up our main rendering thread, we process this incoming data in a web worker and post message back when we need to re-render changes. So when a specific cluster goes red, an on-call engineer can look at Casper's cluster detail page to quickly get oriented with the current state of the cluster without having to switch between multiple apps or command-line scripts. So this page describes the topology of the cluster, how many instances it has, what regions it's in, any running maintenance jobs that may be impacting cluster performance. And it even includes charts of some of the most relevant metrics, like read and write operations, and so on. And in fact, the Atlas charts that you see at the bottom here are the same ones that you saw in Lumen earlier. That's the power of web components there. Now you'll notice that this dashboard also sports a dark theme, which is unlike most of our other apps. So this was a good test for the teamability of some of our components. So our polymer elements expose CSS custom properties that Casper could override with its own color scheme to get them all to blend in with its look and feel. So now that you've seen a couple of examples of dashboard-style apps, let's continue our tour with something a little different. So Keystone is our near real-time data pipeline that allows engineers to send events and logs to HiveTables, Elasticsearch clusters, and Kafka topics. This is the collection pipeline that's used to gather data for hundreds of the AB tests that we run, as well as some of our personalization algorithms. And this is Keystone's self-service, a web app for our engineers to manage their data streams. So we use D3 to build out a visual representation of the data stream as a directed graph with colors, motion, and tooltips along the edges, reflecting actual data flow metrics. Engineers can also use this view to make any config updates for any of their outputs or to change the topology of the cluster of the data stream by simply dragging and dropping nodes to rearrange them. So the encapsulation provided by custom elements and Shadow DOM was really beneficial here because we could build a simple graph, this graph component, as a self-contained component with a simple public API, and all of its complex implementation details neatly hidden away. Since all of the SVG nodes are within the element's shadow root, they're protected from the rest of the app's code and CSS styles. Moving on to the Netflix Data Explorer. And as the name suggests, this is a tool that allows our engineers to explore and update their data in our cloud data stores. So here we're looking specifically at the Data Explorer for Dynamite, our key value store that's built on top of Redis. Although the app's architecture allows us to layer in support for other data stores as well. So using the Redis API, Dynamite Explorer allows users to search for data by keys. Now, we needed to scale to handle millions of keys per cluster. So we used a virtualized list, like the awesome iron list element, an infinite scroll style pagination to keep the UI running smoothly without skipping a beat. So you can see that it's handling about like 145 million keys in this case. In addition to simple data types like strings, Dynamite Explorer also supports complex data types, like JSON values, hashes, lists, sets, and sorted sets. And with Data Explorer, our engineers no longer need to figure out which boxes to hop on to and what data store specific commands they need to run, when they simply need to look up or update some data. So you may have noticed that this app heavily makes use of the paper and iron elements along with the material design styling. So having this rich palette of well-built, well-tested components available to us allowed us to rapidly prototype and build out this UI in a very short period of time. So we could spend time also building out and optimizing our Node.js API layer, as well as like adding additional features, like single sign-on, auditing, and so on. And last but not the least is Winston Studio, our app for operational runbook automation. So let's say you're an on-call engineer, and whenever you get paged in the middle of the night for a particular issue, you perform a series of steps from a runbook to diagnose and fix the issue. So Winston is our internal platform that lets you automate those steps in response to alerts. And Winston Studio is the web app that they use to wire up, author, and test these automations. So here's an example of a simple automation that collects logs from a service and emails them to me when a health check failure email fires. So you can edit the Python code for this automation right in the browser itself with syntax highlighting and basic syntax checks. And before promoting it to test or prod, you can actually test this incrementally with a fixed set of input parameters. So this is another unique app compared to some of our other examples, because it's almost like an IDE in the browser itself. We're using Codemirror for the editor component here. And the nice thing about a component-based app architecture is that we can lazy load some of these larger third-party dependencies, only when a user actually gets to this page in the app. So those were just some of the many apps that we've been busy building using Polymer within Netflix. To recap, here are some of the interesting takeaways that emerge from some of these app examples. So custom elements and shadow DOM give us a great encapsulation model, allowing us to build complex components with simple public APIs and shadow routes that shield them from global styles. Themeability can be achieved using CSS custom properties and the add-apply mix-in. And we're also looking forward to the part and theme specs that we talked about earlier. Web workers are a great way to do background work off of the main UI thread. Virtualized lists, like the iron list, for example, combined with paginated APIs can give us fast, fluid performance, even when we're dealing with huge amounts of data. And a component-based app architecture allows us to lazy load just the dependencies we need for the views that a user has requested. All right, so now if you're looking closely, you may have noticed that many of these apps have a lot of things in common. So modern web development across most libraries and frameworks these days is centered around apps composed of components. In the case of Polymer, these components are web components, or more specifically, custom elements. In this section, I'll talk about our approach to building component libraries and some tooling that we built for a better developer experience. So being a small team with lots of apps to build and improve and maintain, we take a more pragmatic approach to building new components. When we can find high-quality, styleable components in the vibrant open-source ecosystem, and there are many, we use them and build on top of them rather than reinventing our own. We then augment these with our own elements that are specific to our internal use cases. And we use CSS custom properties and the app apply mixins from our internal style guide to apply consistent styling across all of these components in terms of colors, typography, spacing, and so on. Now, when it comes to building our own components, we also wanted to have a consistent and streamlined developer experience. This includes things like scaffolding out new elements and iterating on them, generating API docs, demo pages, and making it really easy to do semantic versioning when you need to release new versions of your elements. So a couple of years ago, before the awesome Polymer CLI as we know it today existed, we built the reboot CLI internally. So this is a command-line tool that a developer can install using NPM, and then use it to scaffold up new components with a consistent structure, a common ESLint profile, and a set of common NPM tasks to perform various actions. The NPM run dev task, like you probably expect, just fires up a polyserve dev server so you can begin building and iterating on your new component. But there are a couple of other interesting things that the reboot CLI can do. The first of these is auto-generating API docs for an element's properties and events. So we do this by piping the element definition through a custom Babel plugin that traverses the syntax tree and generates a markdown file, containing the list of property names, their types and descriptions, list of custom events, the element emits, and so on. And this also includes links to the specific line numbers in the source code for all of these things. So having this in a markdown file makes it really easy to look up the API docs for an element either directly in the Git or stash repo where it lives, or in our element catalog where we convert that to HTML. The second is easy and consistent semantic versioning of elements. So when you're ready to release a new version of your element, you simply run npm run release and you tell it if this is a major, minor or patch version bump. Reboot version then automatically figures out what the current version number is, auto increments the appropriate position in the version string, updates packet JSON, creates the Git tag and pushes to origin all in one quick step. So as you can see, we've been using Polymer One for a little while now and we've built a bunch of components and a bunch of apps and now it's time to start migrating all of those over to Polymer Two. So we just started going down that path fairly recently. So I thought I'd share an approach we're taking depending on sort of what the app type is and so on. So to start with, we're making the latest stable version of Polymer One, the baseline across all of our shared elements and the apps that use them. This basically ensures that we can safely use hybrid style elements everywhere. It's a good practice to regularly keep up with the latest Polymer versions anyway. So this step shouldn't involve any big breaking changes. Now we recently started working on a couple of brand new apps. These were perfect candidates for actually starting out with the new Polymer Two library and ES6 classes for the app specific elements. But these new apps also depend on some shared elements from our component catalog, right? So we took this opportunity to convert those shared legacy elements to the hybrid style so that they can work in both Polymer One apps and the new Polymer Two apps that we're building. When we eventually have ES6 class versions of those shared components, we can simply swap those in and now we have a fully migrated Polymer Two app. But what about existing apps, right? Existing apps, I think we can take two different approaches depending on the size, complexity and rate of change. So for small apps and apps that aren't being updated much anymore, we start using Polymer Two and upgrade their app specific elements directly to ES6 classes. Because these apps are small and not changing very frequently, we can skip the intermediate step of converting these to hybrid elements. And again, once when we have the shared elements upgraded to ES6 classes as well, we can simply swap those in and we have a fully migrated app. For larger apps and ones that are still like in active development, things get a little trickier. It's a bit more challenging to do a big bang migration all at once. So for such apps, we started converting their app elements to the hybrid style first while still running the Polymer One.9 version of the library. Once that's done, we can just replace Polymer One with Polymer Two and continue running in hybrid mode. Any new app specific elements that we're writing now will then be written as ES6 classes and we can retroactively start converting some of the older app elements to the ES6 style as well. And again, like the previous two cases, once the shared elements are upgraded as well, we simply swap those in and we have a fully migrated app. So those are just some of the different strategies we can take depending on what type of app we're talking about. And that brings us to our last topic, which is state management. Also happens to be my favorite one probably just because of the Stranger Things reference. So as we were working on a number of apps, especially larger ones, composed of lots of nested components, we started running into some pain points around state and shared states specifically. So let's briefly dive into what the problem was using a couple of examples. So say you've got a parent component and its child that share some common bit of state. A common pattern in such a situation is to make the parent component the source of truth for that state and pass it down to the child as a property. If the child then needs to change the value of that state, it simply emits an event that the parent listens for. The parent then makes a change to that property and passes it back down to the child and everything just works. But consider this example. Here's a deeply nested child that cares about the same bit of state as a parent a few levels up. In this case, the property has to be passed down every intermediate component in the hierarchy, even though these intermediaries don't really care about that state besides just passing it down. So they're just acting as pass-throughs. Same thing with the events bubbling up from the child. And here's another situation. In this one, two sibling components need to share a bit of common state. So now you have to find a common parent ancestor and store the state in that, even though it doesn't actually do anything with that state besides passing it down to the children as properties. This example also illustrates a potential problem when a refactor or design change causes one of these child components to move elsewhere in the visual hierarchy. You now have to move this component there, find a new parent, a new common parent, and then play the property passing game all over again. So it turns out that there's a very popular project Redux that aims to ease some of this pain. It's described as a predictable state container for JavaScript apps. And you heard Kevin talk about this a little earlier today as well. So we've had several teams within Netflix using Redux quite successfully with React, but there's nothing really React-specific about Redux. So we've started experimenting with it in some of our larger Polymer apps as well. So the central concept in Redux is a store, which is an object tree containing all of the shared state for an application. Properties on individual components can be mapped to specific slices of the store so that they're automatically updated when the state changes. Components can also dispatch actions to the store when they need to update some part of the state. So this is great because our shared state now all lives in one place and is only passed to components that actually care about it. So an open source library called Polymer Redux by Christopher Turner makes using Redux with Polymer quite simple. Once your component class extends a mix and create of this library, you can add the state path to any of your property definitions and they'll basically change whenever the store updates that value of the path in the state tree. Components also inherit a dispatch method from the mix-in that they can use to dispatch actions instructing the store to make a state change. So what's next for our experiment with Redux? Well, firstly, we're still evaluating. So Redux may or may not be the right solution for every app type or every use case. So we're still trying to figure out where it makes sense to use it versus not. Second, in apps that are using Redux, most of our shared app state already lives in this Redux store. But there's also this additional state that lives in the URL in the form of path parameters and query parameters. So could we have the Redux store capture some of that as well so the app can now handle all of that state in the same way? Can we keep the URL in the Redux store in sync bi-directionally? So that's something that we've just started exploring recently, just a few weeks ago, in fact. It looks something like this. So similar to how the Polymer Redux library has the state path concept, we can declare like this query parameter here. And then our mix-in now is responsible for keeping the URL always in sync with the Redux store. So our app now needs to... Our app can basically read the state exactly the same way. So diving into the depths of Redux can be a whole talk in and of itself. And we don't really have much time for that today. Props down and events up works really great in many, many cases. And you should stick to that pattern if it's working for you. But if you'd like to learn more about Redux here and how it can be used with Polymer, here are some resources that you can check out. So there's the Redux site itself, of course. There's the Polymer Redux library that I mentioned. And then there's these couple of awesome polycast videos on YouTube by Rob Dodson that actually show you how to use the Polymer Redux library as well as do some of the async stuff with it using thunks. Yeah, and that covers everything I wanted to talk to you about today. Again, my name is Kunal Kundajee and you can find me on Twitter as atkunal. Thank you for being a great audience and have an awesome time at the rest of the summit.