 My name is Blake Miseraini. I have worked at a number of companies you may have heard of, but my current company that I founded is called Backplane, where we're building a globally distributed fleet of smart proxies to do a lot of what we're talking about today. But formerly before that, I was at Heroku, and I was also at CoroS, where I worked on at CD. When I was there, I kind of felt like Martin Yan. How many of you know Martin Yan? Yeah? Okay, no? Okay, so Martin Yan is a Chinese American chef. He had this really awesome show on PBS back in the 80s called Yan Can Cook. And this was during a time when everyone was talking about French cuisine and maybe burgers and fries or grilling or whatever. And then here's this guy on PBS with nothing but a walk and a cleaver making some really amazing dishes and just blowing people's minds. And in doing so, he also taught a lot about cooking with just a few simple tools versus having an entire array or a kitchen stocked with all these interesting toys and weird things you can use. Instead, Martin Yan had a cleaver and a walk, and that was it. And it was really impressive to watch him work as well, because that cleaver, a lot of us Westerners would look at this and think, oh, this is for chopping through bone. This is for just like brute force, everything. But he was able to take this thing and use it as if it weighed nothing. And it was just a really amazing art form to watch him mince garlic with just a quick slap of the wrist and just crank through things or julienne bell peppers. And so for the highly recommend watching some of the older videos on YouTube, this guy is hilarious. And he definitely has a very comedic kind of bit about him. But I love cooking, by the way, so that's probably why I know him most. But anyway, yeah, I kind of like to when I've been doing, I guess, cloud native for like 10 years now since early at Heroku. How many of you have ever used Heroku? OK, great. So you were all cloud native long before. So anyway, so yeah, so some of the interesting problems that I worked on, and I spent almost all my time working on, were load balancing and proxying, service discovery. I worked on a project called Dooser quite a while ago, and then eventually at CD came around and I went to CoreOS and worked on at CD. And distributed consensus was also something that's like a really kind of more of a distant part of my past now. And I don't really put a lot of effort into that anymore. But load balancing and service discovery and what you can do with your data plane is something I focus on a lot these days. And much like Martin Yen, I like to try and find utility in the simple tools that I have and these powerful tools that I have to build more complicated and complex things and try and reduce them down to much more simpler primitives. And so I kind of want to talk my way through or I want to talk you through a little app, a very simple app. And we're going to kind of go through the evolution of a distributed app and then look at how some tools that we kind of almost, I would say, neglect could actually make things a whole lot easier and help us simplify our applications as well. So let's build a to-do list. I chose a to-do list because it's something everybody can relate to, has worked with and is constantly aggravated with on all the like everything that is out there and available today. So let's go through and build a very basic one and then we'll see how complicated and how quickly it can become complicated. So this is our to-do list. We have one list. It's our shopping list. There are items and there are checkboxes. And with these checkboxes, we can check things off. This is the most basic to-do list we could possibly build. How many of you have built one of these? Maybe it was like a fun project when you first, you know, like something to like kind of dip your toes into some new technology or languages, you can build a to-list and then let's add a new feature. Another feature might be that we can have multiple to-do lists. That seems reasonable and like a really great feature to have. There's my life isn't just one long list of things to do. I like to categorize them. So how might we build that? I think to get started, it might look something like this. How many of you would probably just default to this? You have your HTTP server and you have a database. Super basic. Super easy. This is the right way to start. This is a SQL database, by the way. We're not going Mongo or anything yet, OK? And so our call graph, if we were to look at this as like a distributed system, our timeline would probably look like this if we were to implement the marking or the completion of an item. We would have mark done. We'd probably pass in something like, you know, some basic information, but mostly the list name that we want to check the item off and what item in the list it was. But this is a little too simple. So I'm already losing people. It's a to-do app, I know. All right. Just follow me. Bear with me. This gets fun. So let's build an enterprise ready to-do list, OK? How many of you have been to enterpriseready.io? If you're working on anything or selling anything to enterprises, you should need to read that site. It's absolutely great. But one of the first things you're going to need to do is build users and organizations, right? Like no enterprise or any company is likely to use your product without this. So what might that look like? Well, we can still keep our database and we can still keep the simplicity of our architecture, but we're going to need some new tables, right? We're going to need users, organizations, and most likely we'll need roles as well because we're going to want to build some kind of role system into this app. And so let's look at our call graph, our timeline again. So our timeline starts to look like this. First, we need to go get the role. Whenever we want to mark something done, well, we need to make sure that someone can actually mark it done. Do they exist in the organization that that list is in? Can they? Do they have the privileges to mark something done? Can they do any of these things? And once we've figured all that out, then we can mark the item as done, right? So that's two separate calls. That's two calls to the database to do one simple thing, to mark it in the database. But let's go a little bit further. Let's build an enterprise-ready social to-do list. And what does that look like? That looks like this. People can watch you creating lists and checking off items and being productive, right? I can follow a list. I can follow a person. A list can follow a person if maybe, I don't know. We could have audit logs. We can have all sorts of other interesting features to it, right? We're building a sauna now. And so now our architecture might look like this. Any of you have tried to create any kind of social graph or any kind of graph for that matter, anything that goes beyond just basic inside of a SQL database, all knows all too well. They're like, it gets a little complicated and performance can have a bit of an impact at scale and we tend to scale, right? So in order to do this, we're probably gonna want something like a graph database. So this graph database is gonna model the relations that we have between lists and users and likes and followers and et cetera, et cetera, et cetera. But we'll still keep all of our canonical information inside the transactional database so that we have a source of truth and can rebuild that graph if we had to. So now our system starts to look like this. We have to get the roles, right? We still have to get roles. We still have to determine whether or not this person can write to this list, i.e. check something off. We need to then write the mark to the database and then we need to notify everybody in our graph in our immediate following list of what just happened, right? We need to allow them to see that I am being productive. I have done something. So there's a problem with this. Well, actually before that, let's add one more feature. Let's add feature flags, right? So now we have someone who's like, well, I'd really like to be able to bring on five or six users to try this new beta feature out within the organization. So now we're gonna add yet another table. We're gonna add flags. So we have feature flags now, right? And these feature flags, they're pretty basic. It's like on or off. It's some kind of key value pair, right? I mean, people model it differently, but it's all basically the same concept. And then we have to do billing. We plan on charging people for this, right? Right? Okay, come on guys. It's the after lunch slump here. So we have to do billing. And in order to add billing, let's oversimplify billing and say it's one table. And so now you can kind of start to get this idea and you're starting to understand where I might be going here. And that is that this is becoming a little complex. We were building a to-do app, right? Like we weren't trying to build like this whole like social system. In fact, we're reinventing the wheel on nearly every one of these things. So there's another problem with this. If you look at this, A, we have to keep data synced between the transactional database and the graph because the transactional database is our source of truth. So we need to make sure that it's constantly in sync with our graph databases in sync with that. So that's gonna require some other mechanisms as well. I'll leave that as an exercise for you all to take home later. But there's one big key thing here and that is on our track transactional database. Look at that, 75% of our database is not really value add, right? It is not something that we're having to like we want to really be in the business of dealing with. So even so, we might reach out to third party services or whatever. And that's one great way to start. But if you wanna own some of it, then you're gonna have to find another way. And in fact, this also causes another problem. The other problem is that as we grow the application, we start to have other microservices. We start a microservices architecture, right? So maybe we have our site, our marketing site has some knowledge and intimate knowledge of our users and organizations and who's build what and et cetera. And also the API, the API most definitely has almost all more information than any of these other apps. And the admin panel that allows us as administrators and operators of this to-do list, this allows us to be able to manage our users and manage things. And the dashboard, of course, we all have dashboards and that thing needs to start slurping data out of this database. And so all of these services start to get very familiar with this database, almost too familiar. They start to get very tangled. And the disentanglement cost becomes more and more and exponentially more expensive. So what we can do, and then, oh, sorry, to make matters worse, each one of these is written in a different language. I know it seems silly for a small startup, right? But when you're talking with enterprises in large organizations, like this is how it is, right? And I mean, how many of you work for an organization larger than 200 people? So you know what I'm talking about, right? You're not all writing the one perfect language you wanna use, right? There's many different languages. And so this means that in order to do the things that you want with organizations and roles and access management, you wind up having client libraries written in many, several different languages. There's many different versions of them out there within the organization, and things start to get really hairy. And so if anyone wants to change one thing or do one little feature, add one more feature flag or another role or anything or a billing plan, like everyone has to start to become aware of it or at least most of these things have to become aware of it. So this is really complicated. How many of you have experienced this now? How many of you have at least seen it before or have heard horror stories of it, right? So there's one thing that we can use that can help us move away from all of this and help us remove the intimacy that all these applications have to have and the headache of having to upgrade several different libraries across several different languages across the organization. And this is our proxy. So how many of you use a proxy for more than just the basic stuff, right? So the great thing about proxies is that they speak universal wire protocols. Like they're completely language agnostic. They just speak HTTP or memcache or whatever protocol you might be using, GRPC, whatever. They also have some pretty interesting properties about them. They're controlling the flow of traffic. And they're not really, a lot of the more legacy load balancers out there today, don't give you that much control in real time over what's happening, but they can give you some kind of like longer persistent ways of kind of divvying traffic up. And they also know what is what. They have a better understanding of your system and your health of your system and which processes are running and not running and healthy and accepting traffic and not accepting traffic than your etcd cluster. Your etcd cluster is just trying to like keep some things intact, right? But like it's kind of saying this is the way I really hope things would look like. I really want things to look like this, but this is it's not always the case. In fact, most of the time what etcd says is not the actual case of what's actually going on in your system in production. Things are flapping, moving around, dying, coming up, like taking more load than others, etc. And so your proxy, it does more than load balancing. It can do more than load balancing. Your proxy can do more than TLS termination. And your proxy can definitely do more than just doing some quick header rewrites for you to service those old blog post URLs because you moved from one blog to the other last year, right? Your proxy can make macro decisions. It has all of this information. And so why don't we allow our proxies to make more decisions for us that are more mundane and more time consuming for us to do than we do? Why do we always keep trying to do this in the applications? And the other thing that it can do is add context to your requests for you as well. And so I'll get into this in a second. I like to call this enrichment. This is like request enrichment, right? So as a request comes in, there's some amount of information associated with it. And if there is information that the proxy does not have, it can do the same things we would normally do on our application code to get more context around what's happening, such as authentication. But how many of you, there's a really famous request enrichment. It's something that all of you use and are all of you are pretty familiar with, I think. And can anyone name it? It's a header. Holy shit, yes. Yeah, yeah, it's exported for, right? Like that's a request enrichment. It's like giving you some context around what's going on. And like, so when you get that request, you can make a slightly more intelligent decision if you need to make a decision based on where the actual originating IP was. Some load balancers, including backplane plug, can give you country code. They can give you the city code. And those that are smart enough to actually do authentication on your behalf and more of a zero trust networking beyond corp style and identity aware style can give you the provider that was authenticated. They can do the authentication for you and also tell you who they authenticated it with. They can give you the identity of who was authenticated. And what gets even more attractive is when you say that, well, what if I want attributes that I specify about a specific identity? So maybe there's this header called auth attributes and it's just a set of strings, just tags, just arbitrary things that our user defines or are also provider defined. This can begin to get interesting. So if we go back to our to-do list, what if we injected strings for, say, Blake at backplane.io and it made me to say that I'm allowed to mark things off of the shopping list. So our string might look kind of like this, kind of follow this format, right? So here it's just, I invented this for this slide presentation. This is to-do, colon, org, list, action. Like this is our format, right? How many of you have ever worked with IAM roles in AWS? Right, like yeah, that's the same thing, right? Like that's exactly how this works. There is an access proxy that deals with these things. So our code can then look like this. And this is go, so, or pseudo go. So here we can just do a quick branch statement. When we receive a request, we can already have contextual information about that request. We can know who was identified and we don't have any, you can grep our entire code base and there is zero OAuth or OIDC anywhere in that code. Like all that nuanced thing, all having to upgrade the Ruby gems or the packages, the libraries or the Maven packages or whatever, like that's all a thing of the past. I'm not worrying about that anymore because my proxy is able to do it and tell me what it is I should be doing. And so I can just trust the proxy. So let's go back to this timeline again for it to do that. Remember we had git roles. We needed to get the roles for the user that we were authenticating ourselves. We had to go and create, we had to check to see whether they could write. So that's another database call. We had to then market as okay. And then also update our graph. So these things are all step by step in a very linear fashion. Like we're not getting a lot of like, this isn't as ideal as we would like it to be. But what's great now is that we have attributes coming from our proxy that we told the proxy about earlier. We said, hey, Blake at blackplane.io is totally okay. It's fine for him to mark things as done on the shopping list under the organization, backplane, et cetera. So now our timeline looks more like this. We're less complicated, right? So we've eliminated the need to make any calls to the database. Like we're being told what to do. We're told what is what. We don't have to ask anymore. But what also, this also allows us is to add some parallelism now too. We don't have to do the atomic check first to make sure that the person is capable of doing what it is they say they can do. We're not having to ask anyone first and make expensive network calls. Instead, we're told. And so now that we're being told, we can eliminate the need to go to that round trip. And also we know that it's totally okay that both the graph and the transactional database are capable and it is okay to make that right to both. So we don't have to wait. So now we can do something like this. We can just go ahead and do both of those at the exact same time. We don't have to wait for permission. We've already been given permission and we can trust our proxy. So now our call, like our timeline looks more like this. So now we're getting performance out of it as well. Any questions so far? What's that? I'll get there. That's a good question. So if we're able to, one second, I just get, so if we're able to trust our proxy to tell us what organization the user's in, what list and what things they're allowed to do, then we can eliminate that from our transactional database and we can get back to simpler times. Like we're not getting rid of our graph database. We still need that to identify all the relationships between who's watching what and et cetera. But we can eliminate the complexity of users and roles and access and feature flags and et cetera and all this. Like all this can be pushed up into the proxy. So to answer your question, this slide. So now our proxy doesn't need an API now though. So what if your proxy had an API that you could post to and you could just treat it as this kind of black box service that you could say, hey, just update this user. And then all of a sudden, from then on out, like whenever you start to getting requests about from a user, this new context, these tags, these labels that you've identified for them are automatically merged in and enriched within the proxy and then sent to all of your services. Not only is the API gonna be able to receive this, but all the other services written in different languages, running on different instances and different platforms and different environments and in different places, not necessarily having access to all the data and the databases that you, one other service might have access to, can now benefit completely. And if you decide to change identity providers, you decide to change anything about how you want to and identify people or things that are coming in to get data, you can do it without having to update all of your services and all of your applications. No one has to deal with it anymore. These are arbitrary strings that you defined, not some data layer that everyone has to completely rewrite all of their code to talk with that data layer. Does that answer your question? Okay. And again, this is language agnostic and we're reducing the intimacy that we have with our data layer. So let's go back to this again. There's one other benefit we can get from this and this one I just threw in because it was fun. We can now logically shard our database if we wanted to scale out. Instead of logically, like if you want to logically shard based on lists in the current version with the previous one, all the users and lists would have to be duplicated or some other new database would have to come online for everyone to understand, to be able to go get all those permissions and roles and interesting information before they can write to the list database which is being sharded out. But now you don't have to worry about that. We can just shard based on lists and keep items and lists in the same database and the same shard. So it would look like this. And so now we don't, those complicated inner joins that we had and all that intimacy that we had inside of our SQL queries and inside of our client libraries is all gone now and that frees us to be able to do other things and add more performance and scale to our system. What this also gives us is central control. This gives us the ability to control who and what can see what from a specific API that looks like this. You have a plain API. And that plain API is like the all knowing. It is the central source of truth now for anything that you're telling it about and asking it to enrich your requests with. And there's one more thing. So there's a few other things a proxy can do for you. I mean, there's so many and I don't have time for it. That was one of the more fun ones I wanted to talk about. But how many of you do canary deploys today? Like actual shipping of traffic. So I don't know how much time you spent building that system out. But they're not easy to build out. But if you have a smart proxy that has this feature built into it, like it can take care of all of this for you. And if it has an API and you can in real time while it's running, tell it different weights for different versions of your service and different builds of your service. You can have multiple builds running simultaneously. You can ship 1% of it, you know, the request to this cluster and 99 over to this or some variation thereof. And you can test out new and interesting features without having to, you know, do a full up full down where you just redeploy the whole world all at once. And if you have a global fleet of them, then you get other interesting aspects. You get cloud agnostic slash multi-cloud failover. You can now run in multiple different environments and have all of your configuration and information being enriched at the edge. And then once it gets down into the cloud provider, the proxy doesn't know what cloud you're at. It doesn't care. And when you receive that information, you don't need to care where you exist either. You're able to just operate. So you're having less dependency on the cloud, less dependency on cloud specific features or cloud specific services and relying more on your proxy to give you the information you're looking for. So it would look like this, right? So hopefully I've convinced you that proxies can do a whole lot more than what we've previously relied on them to do, load balancing and TLS management. It's kind of that thing that a lot of people set up and they're kind of like, I'll just provision this thing over here or I'll set up nginx or I'll just do this. And then it kind of just eventually turns into this big lump of mess. How many of you have had at least 6,000 lines of an nginx config? That's kind of what happens, right? It's like I'll rewrite rules and everything and that's not an API. That's a config file. So in closing, trust your proxies. Start relying on them more. Start pushing more information to them and make that information and how it came to that information more transparent towards the services that are running within your organization. Developers are much happier. They move faster. And you can offload all this mundane work like feature flagging and billing and IAM roles, A-back, R-back systems. It's a pleasure. Like once you start getting used to doing this, like there's no going back. You're in this really nice little sandbox when you're working on your service, not having to worry about all these things and being able to just kind of just lean in and trust the proxy. And you get your central control and performance as well. You can do all sorts of new interesting performance tuning to your app when it isn't concerned with all of these things that are 75% of your app, right? Like the flags and everything. So with that, thank you and we're hiring. So please, if you're interested in back planning would like to work with us, I would like to talk to you. So thank you. That was it. I can take questions. I have like five minutes left. Anyone? Yes. Yeah, sure. So in that, you're gonna want mutual authentication with your proxy. You wanna know that when you talk to that, when you receive that request, you wanna be able to know that you're getting it from a reputable source. Yeah, so there are concerns in that. Like at back lane, we do a reverse dial. Like, and then you have a token that goes out to the edge and then it does mutual off that way. So you only have to trust local host and if you see local host coming in, like you know that it's backplane and you can just accept this information. Yes, I'll get to you. To what? Sure, so you remember that string we used? The question was how do you implement feature flagging using the proxy? Well, you can use the exact same mechanism I was using for role-based access as well. It's just a string. So just say like to do has blue background instead of green background and then check for that tag when it comes in. It's a much nicer model than like having some SDK that goes out to a database and like is constantly keeping an LRU cache or something like that and like hitting the database constantly. Instead, the proxy keeps the cache and then you can just thread that all the way through which is really nice. Well, you would hit the API for the proxy. So again, I'm saying like give your proxy an API and then you can dynamically update it or you can use the backplane. Yes, oh wait, sorry, someone, yes. Yeah. I want to get to some other questions, but I want to answer yours. Can you come up here? No, I mean after, I can answer afterwards, thanks. Any other questions? I can do one more. Yeah. So like in practice, is that like a service that just makes calls to or are these like engine x modules that you build or what does that look like? So our architecture of backplane is we have a control plane API that we keep a long pole on and all of our edges like are constantly keeping an updated cache. So like the entire data model is like represented in a hot cache at the edge so that it's not having to constantly phone home to like get the information. So which makes it really efficient. All right, that's it. Thank you.