 Hi, thank you very much and special thanks to POS Asia for accepting my proposal to talk here. It's been a bit of a scheduling nightmare for me and special thanks to all of you all because you all actually soldier through like four days of everything. There's I think a crunchy database talk going on in parallel and you all chose to stay and hear me. Thank you so much. There's also I think a blockchain topic on so again very special thanks for that. My name is Yogi and I don't have a clicker so I'll have to go back and forth. I live in Singapore. I work for a company called Pivotal. I'm a senior platform architect there. I've been there for a little over a year and what I'm going to talk about today is driven from my experience with Pivotal and my time in you know various large enterprises that I've actually worked for. This is purely based on on-ground things that I've seen and sort of boiled them down to a few simple recipes that probably might be useful in your daily job or maybe you know somebody that you know may be going through the same sort of motion right. I am active member of some of the local communities so are most of you all local or you all have traveled? How many people are from Singapore? Okay how many of you all are seeing me for the first time? Wow I got to visit more communities I guess now. I used to be manager for GDG SG and the two new ones are sitting right back there. I was doing not much so they threw me out. So abstraction I prefer this being a more interactive session. I don't want you all to go up to sleep. I already see you like you know relaxing but I promise I'll keep it as lively as possible. What do I mean by abstraction? Like we've all used abstraction as developers. How many of you all identify yourself as engineers, developers? Everybody, not you? Partially. What's that like? I really am really interested now. Oh so you do development for yourself on personal projects? Wonderful so you are a developer. Well she managed to be a developer in a bank which is great. I used to work for a bank and I know how tough that is. Probably we go through some of your things in this and mine. So simply put you know abstraction is all about eliminating non-contextual details right to just focus on stuff that matters. When you're building a house when you start with everything you just put like a map of four walls okay here's where I'm going to have a door that's where a window is going to go and you just start with those details. What is the color of the window? What is the color of the door? What walls, what wallpaper is going to be on the walls? Don't care. You start with that basic frame of reference and then you keep adding to it as you go along yeah. So the process of abstraction has actually allowed us to move or break down large complex problems into something more manageable yeah. In essence I kind of went through a lot of definitions online in books and this sort of actually puts what I feel really out there and this was from a gentleman John Gutec. I don't even know who that is but I found that definition to be most relevant to what I'm talking today. So keeping the information of the given context there and removing everything else that is not that probably summarizes the abstraction for me. We are talking about for about extra abstractions very specifically I'm going to focus on large enterprises which have like 1000 plus developers they are managing like you know north of 500 applications some places have actually seen about 2000, 2600 applications being managed by these. The problem of scale really makes these the whole concept of abstraction super super important. One of the key challenges with enterprises is that they have very very huge variety of workloads you know. 10 years ago they embarked on a journey to modernize their mainframe workloads they started actually implementing J2E applications which were very portable but because of either lack of governance or people rolling through projects rolling through jobs these became large monolithic applications. Enterprises have also had message oriented middlewares where you can throw messages at a middleware through some magic it goes through the whole system and ends up on the right system. Sometimes those workflows we don't know. More modern organizations or organization adopting a sort of digital transformation they probably are on the container or the microservices side but it's too few and too far apart even that is challenging for them because especially with the microservices framework you end up because you're building microservices you have these small small applications scattered all over the place no not two of them you pick up any two of them they are completely different from each other how they interact with the system how they throw out their logs how they how you monitor them how do you start them up all those things they are different for different applications and hence the need for abstraction is even more urgent there. That's the second problem which is they have lots of things scattered all over the globe we all bring our you know geographical our cultural biases into the mix as a developer our commands our mess our function names our function flows they differ drastically I'm not saying it's a good or bad thing I'm just saying that it's a different thing it's different things for you to manage and the biggest of them all they have a very very large legacy does anyone recognize what's that can anyone say which model that is because I don't know I tried doing a Google search I couldn't find it's that old so we've actually used abstraction as part of our software development methodologies right we've used object oriented programming where you know a group of senior developers or a people who have more knowledge around the whole domain they put together some high-level imperative some high-level definitions and as developers in terms junior developers new developers pressures they actually start using them and implementing the capabilities case in point servlet API servlet API was introduced by sun back in the day as developers it made our life so much easier that if you wanted a server-side component all you have to do was just extend from the servlet class and package your application and put it in a runtime and voila you have an absolutely performing server-side component which was great until it wasn't functional programming this is still quite relevant people use this quite extensively it has regained a lot of focus these days with Kotlin with a lot of even go with ruby all these programming languages have really really repopularized the whole aspect around functional programming languages in enterprises function functions as a functional programming was not mainstream in large enterprises only like very specific functions would probably end up using functional programming but now more and more I see more and more enterprises actually getting warmed up to the idea this would be the key focus of our talk today which I'm already halfway through the key the the abstraction that I want to actually talk about more is all around the runtimes Cheyenne before me even Sydney they they spoke about for OS and Fedora which is actually great I personally use some of them really allows me to especially the workstation containerization it's a great thing so when it comes to the actual runtime abstraction first of all you have no abstraction bare metals you have rusty servers sitting somewhere in the rear center you have an ssh access to it at the best or maybe if you're lucky or unlucky you will have rdp access into it someone would definitely get the joke but I mean if you're unlucky you will have rdp access because it's running windows ah no you get it so you you actually remotely connect into it and you go in you put your binaries there and you run it you run it as a service you run it as a system d service or in it we choose your poison in that case or or actually I've seen a lot of these actually sitting on the people's desk you know it's like you move office and you are like okay whose server is that I don't know we actually had this situation where we were moving in office and there was there were two large Dell servers which always kept me warm in a hot Singapore day and when we are trying to move like whose servers are these God knows the guy who actually probably bought them is long gone but these servers they are still running I pulled the plug never connected them again nobody has actually asked me so I managed to decommission two servers in a bang kudos to that virtual machines this is a little more modern I would say but again like in IT industry modern is maybe legacy in six months time yeah virtual machine they definitely brought the the whole idea of scaling more rapidly without being constrained by the physical hardware to be able to actually have the level of isolation that is needed yeah just in case you're wondering why I put up put that up it's again my one of my windows jokes it's a BSOD terminal fine who knows what is BSOD really only so few times have really changed blue screen of death that's the error message on windows systems well the color is changed finally I got to put a slash gsod now okay next time and then containers are favorite pets these days right everything has to be in container they are great I use them all the time I really really like them they are really lightweight they start up early they have no sort of overhead per se of having an entire operating system come along for the right but they have their own set of challenges like as a as a developer then you are actually running applications based off of containers you have few things that you have to be aware of right there are external events which is not your own code which makes you actually re publish your containers it could be one of the layers in your container hierarchy that has changed and you need to incorporate those changes right so yes they have super real advantages but there are few things that actually could be painful at scale I'm talking about 2700 application running in containers maybe about over 100,000 containers so it becomes a challenge when you look at it at that scale the last piece of the second last rather piece of abstraction is application this is really really specialized it's still containers at its core it still is a container cloud foundry provides you with this ability to take your application and some configuration or some data which is associated with your application and just pass it to cloud foundry runtime it actually creates the container for you based on the workload so if it's a python application it'll create a container that is absolutely right for running your python application if it has java application same thing ruby dotnet dotnet on windows yes it will actually oh you'll be surprised who uses dotnet on windows uh so all the all these things right so applications actually take away the pain of managing the containers from you and it is the responsibility of the platform you can probably achieve the same degree of flexibility using your own custom pipelines whereby in your in your code you define some metadata and the pipeline will take care of generating the container for you yes you could do that but that pipeline has to be managed by you and imagine that if you have to manage that for about 2700 applications it's tough so application runtimes actually gives you that flexibility of just throwing your code at it and everything below the application and data layer is taken care by the platform serverless functions function as a service lambda all of that is in this space the latest entrant is k native which basically gives you the serverless serverless experience on top of uh Kubernetes so over here you're not even talking about an application you literally sometimes it's just one single file with one function in it you throw throw it to the platform the platform actually takes care of creating containers for you and running those containers at scale and scaling down to zero when there is no workload running on them this is probably something that uh not many people actually put it put it down as an abstraction i actually have seen amazing amount of applications that have started using these specialized runtimes uh make no mistakes these kind of specialized runtimes they've been there for many many years most of them are closed source proprietary you've had tipco you've had you know bpm frameworks from variety of vendors this is spring cloud data flow which is completely open source based on spring framework the way it does things is based on pipelines you have individual components individual applications that are actually stitched together at runtime all sorts of inter process communication is taken away from you as a developer and taken up by the platform so over here if you look at a simple pipeline here you have as an event coming from an http going into the transformation function and getting dumped into a file the best part is in your component which is your http component you don't have to write anything whatsoever to signify that i need to send the data over to transform and on the transform side you don't have to write anything to signify that i'm going to get events from http the connectivity the black lines between the components it's actually externalized it's a runtime uh dependency so you right now out of box the components they support kafka and rabbit mq as the connectivity glue so all you do in your http or transform component is write a single function which takes an object does the work on it and returns it where does that go where it comes from your transform component doesn't care it's actually taken care by the platform to do that so in a summary like i've covered most of these points but basically for physical host we have a very very it's a very very small subset of applications that now requires physical host if you have an application which is written written hobo and needs to be run on a mainframe yeah probably that's what it's going to be but for most of the use cases and anything that was written in let's let's say last 20 years you can pretty much skip this and go to a virtual machine please at the very least virtual machines uh typically uh if you have vendor provided softwares some sort of database servers or any sort of application that has been built you do get ovs or open vm format or ova's or virtual appliances as they call them you get all of that right virtual machines are best abstraction for that kind of workload containers which is on and up and up everybody loves containers everybody ships containers uh even like spring cloud data flows you can actually run it uh using docker compose or even on kubernetes you can run it on top of kubernetes or docker compose and individual components inside that scdf they can scale up and down based on the demand data services are great for containers so if you if you have if you want to host data services like elastic search you want to solar or some sort of even Kafka for that matter if you want to run um you can actually run it on containers on kubernetes it totally works if you want to replatform monoliths so not everybody is ready for 12 factor cloud native applications it requires a lot of investment if you have an application that has been running for like like probably a decade and a half it has so much of logic in it that decomposing it right away is impossible so first step is to move it to a containerized format so that you at least gain some level of operational benefit otherwise if you keep running it on virtual machines well it keeps going down somebody has to actually go walk up to the server put the usb in and start okay let's do it so that's that's a use case applications are very specifically 12 factor cloud native applications perhaps you should look at things like google lab engine or even cloud foundry for that matter for running these kind of applications and they are best for like any sort of api that you're trying to build across your entire internal system or something like that it's it's really great for those kind of things serverless especially like if you have a very volatile workload that is coming in like it can go from say five requests per second to all the way up to 10,000 requests per second within like minutes probably this is something that you should actually be looking at more of course it will require a significant re-architecturing in your systems but that is probably the best runtime and if you want like a just run it kind of goodness right with some level of integration involved specialized runtimes like scdf is actually a great place just to put everything in the picture this is perhaps the money shot so if you want to take a photo please do i'll definitely send out the deck but yeah the key is you want to move as center to this as possible without compromising on like the feature or compatibility or ability of an application you want to choose to move as inside as deep inside the these circles as possible and the more you move inside the less flexible the runtime is more standardizations are in place as you move out your solutions will keep getting a less and less standardized so your complexity your snowflakes in your environment will be very large that's it so if you have any questions i'm sure you do that bad all right the deck would be there i'm reachable on twitter with atyogendra thank you very much for staying back and listening thank you