 So, what I am going to talk about that as I was saying earlier is whatever I have learned and there are two, for the two channels the learning came, one when my company ATIN networks we had our own applications and it was deployed on AWS 25 machines, 25 VMs and we decided actually not we decided our customers asked that they need it in their own environment we were running a SaaS it was okay to have 25 machines we have a DevOps team that used to manage. But when customers said they need it on their environment then asking that okay you take 25 machines was like who does not matter does not happen. So, then we decided that we will go and put everything into Kubernetes and that was very early on at that time Kubernetes was not the defective standard there were multiple like Koroas and Mesosphere and Docker Swarm and everyone by the grace of God we bet on Kubernetes and it came out well so that is good. The other learning that happened afterwards when we started working with our customers and some of them started moving to Kubernetes and then started facing challenges and then we figured out how we can help them and how we can solve those challenges and we work for the customers we work in application delivery area for our own application our own application we do everything. So I am not going to talk about product today what I am going to talk about is how we started and how we moved about the journey rather than the product that talking product does not make sense that does not give learning that makes a sale so that is not the point here. So how we went step by step that is what I am going to talk about I do not think I need to talk about like how microservices movement happened and how applications are now being deployed on Kubernetes environment though we have seen there are still companies not moving to Kubernetes or container environment but still there are good amount of growth that we have observed especially in last one and a half year all private public clouds of course GCP is the maximum as Google has started the Kubernetes project. Now even the smaller clouds like digital ocean there is a conference in India on Friday coming Friday not this one not tomorrow they are doing the entire conference around Kubernetes and all. So we have seen that there is a good increase and all other competitors are now down everybody adopted actually Kubernetes now Mesosphere gives Kubernetes and Amazon also gives Kubernetes and Docker swarm also when the Docker itself gives Kubernetes everybody is compatible. So the two technologies while Docker came up in the container world the Kubernetes come came up in the orchestrator world. So this is what the requirements we have seen from the teams they need efficient operations agility and visibility and control. So now when we were talking then let us what kind of issues we observed this was an e-commerce company of course I cannot take the name so because they were e-commerce they come and they process payment and they come under PCI compliance. So there are some set of services that they have PCI compliant and then they have some set of services they are not really say for example in e-commerce when they just put the product out they need not be PCI compliant. So what they did they set up two different Kubernetes clusters. So one for the PCI compliant services and other for non-compliant services and compliance enforces them that whatever traffic flows that has to be audited and monitored properly and access control should be there. But because Kubernetes does not have the access control by default inside they put two clusters put an application gateway this is their regular application gateway or like data center firewall and entire traffic is flowing through that. So all the microservices are within Kubernetes but still traffic is going out and coming back in and they are maintaining two separate clusters. Next case this was a FinTech company. So they did little intelligently they did not create two separate clusters. They created one cluster but they created two name spaces within the cluster. But traffic flow was the same because there was no other so now it is much more visible that traffic is going out of Kubernetes and coming back in Kubernetes in two name spaces and all their in both the cases the developers are instructed that any time they want to have traffic between these two they need to write separate code for going out and coming in etc. Plus in this case they also were not aware that what kind of traffic is flowing. So for that they started collecting some information from that application gateway that what traffic is flowing and what is happening. So some minimal analytics they got there not like too much application level but some analytics they started getting there. The third one this was a media service company they had actually so many microservices the number of micro I have shown only four but they actually had I think 200 close to that number I do not remember the exact number. So their problem was the sidecar model came in sidecar proxy so each microservice will have one proxy their challenge was when they do this way for 200 services or 250 services they are doing 250 proxies. And each they had they what they did they did big servers big nodes each node is running multiple proxy one for each. So their resource requirement went very high and the overall cost for the project went high and they were not really comfortable. So they were like give us a solution that does not go this way so that our resource requirement is low and we have less number of proxies to operate rather than manage this adds to the cost of operations also like effort of operations. So that is what their challenge was. And these are some other things some people like some people are like we have seen some cases here also yes it is learning curve and everything is working fine then why should we move or some people start learning and then some people still say a lot of learning internal external network isolation that is like pretty standard challenge and Kubernetes provide Q proxy that Daniel is going to talk about. But Q proxy works with IP tables and then gives the layer 3 people require little more sophisticated traffic routing layer 7 traffic routing and that was generally not possible. Actually there are CNIP plugins also that other than Q proxy that map IP addresses in and out but again the same problem it is that network layer 7 thing is not possible and no access control between microservices you already talked about application layer visibility we already talked about the company did outside and of course cost of operation that we talked about. So now let me talk about how the whole generally so first you put the CNIP plugin you put an external load balancer use traditional some people started doing this use traditional hardware based and then like L7 happens there and then it comes internally maybe Q proxy is a two layer load balancer something like that or some people went and found some proxy load balancer some vendors actually deployed that can be deployed in within Kubernetes node as well. So now traffic is coming in and proxy is within Kubernetes node and distributing but this is layer 7 proxy say for example NGNX has this layer 7 proxy or HA proxy has this layer 7 proxy 8 and also have it in proxy but that is okay. So other challenge that came because Kubernetes ensures availability it is a good thing. So when a pod is not behaving well Kubernetes will kill it and bring another one that is a very good thing but IP address changes and any external or internal need to know that IP address. So Kubernetes defined ingress but did not implement it left for vendors to implement some people some vendors like NGNX etcetera they implemented this ingress within their load balancer and some people did it outside. So both both the ways but some service that monitors the infrastructure and communicates to the application traffic management area was required and that was built. Then requirement come for security and especially in this case for north south traffic security traffic that is coming from outside. So the typical answer was that you put one more layer one layer you are doing load balancing other put other layer for security but again the movement we start putting layers the complication starts growing because it is not about just one simple box one simple node that we are dealing with we need to scale as well. So we will come to a scaling but before that the simplification of architecture is that simply these two get merged something is still remaining. So if there is a possibility to merge then it is a much better solution rather than putting multiple layers in and then come came the consideration of scale and again in here I am showing just little scale just two nodes but we all know that it does not stop at two nodes it has to go longer and when the server side scales the load balancing side also need to scale or security side also need to scale. So again both cases are there external as well as the internal but if it is external then somebody need to put that manually and there is a possibility to scale automatically when load balancer is internal because Kubernetes provides us demon set. So you add a node Kubernetes with automatically launch instance for you and you will be happy. But one more component now is required the management component because otherwise this whatever monitoring that is happening it has to communicate to multiple entities now and then again it becomes inefficient. So if the central management comes into picture again it is not about just two here two looks very simple but scale when we are going into enterprise scale we are not talking just two it is and number is changing all the time because of scale up and scale down. So in traditional environment maybe not a scale up a scale down too much but as we are using in like AWS kind of environment scaling up and down very simple. So that can or any cloud environment for that matter GCP it is much simpler. So in that case the central management also came into picture and then resource requirement as we talked about there is a sidecar model but there is another hub spoke model when one node sorry one proxy per node can do the job. Traditional hub spoke there is nothing new about it a traditional hub spoke model where you just put one proxy and it can serve all the services there. But the requirement in that case is that the proxy should be able to identify the segment that we call micro segmentation should be able to segment the traffic based on certain conditions and apply policies accordingly should not happen that over traffic came from service one and it went to service two that does not make sense. So proxy need to have that ability I think I talked about it earlier only. So ADC for the scaling purpose is demon set that was that is the first kind of solution that we figured out. Oh sorry wrong word load balancer. So they are they are sometimes used interchangeably proxy load balancer and ADC they are sometimes used as interchangeably I may be also doing it but yeah in this case proxy or load balancer that is again a container. So external in case of external it cannot be done that is why that that use case is gone because if you really want to automatically scale then it has to be within Kubernetes then only it will scale automatically with Kubernetes and Kubernetes provides this as a demon set. So as you add a node you need not worry about putting proxy in their Kubernetes if you define as demon set it will do it automatically that is one thing. The other thing as we talked about the central controller but just central control is not enough because say if a scale happened but configuration did not come to the proxy then again it is useless. So the central controller and the proxy has to work with in tandem so that as soon as proxy comes in it automatically talks to the central controller gets the configuration it starts working then only it missens what black yeah ignore. And again that piece that monitors the infrastructure that keep looking whether IP addresses are changing and what is happening and configuration is changing and updating that piece is also required. I think one more thing I am not sure let me talk about it here maybe there is a slide later on maybe not. The other observation that not observation actually people explicitly asked that we will not go outside Kubernetes to configure something it is our comfort level that is one but more than that people want to keep actually companies want to keep not people themselves but companies want to keep code as sorry infrastructure as code and that is a movement we are all part of DevOps movement. People want to keep infrastructure as code and then check into the SCM system like Git is the popular one now and then entire deployment will happen via CI CD pipeline no manual work. So because of that no matter what central control is there central manager no configuration in that central manager all configuration has to happen within Kubernetes in form of whatever YAML or JSON supported in form of Kubernetes objects and then the whole thing has to flow automatically. Kubernetes played a very good role here. So one earlier Kubernetes just defined ingress for the monitoring purpose but other thing that it did within ingress framework it allowed for annotations. So if you see the YAML or JSON there is a annotation section and then there is a services section in annotation section it is nothing is defined actually by Kubernetes. This is for you to extend the way you want. So you can now what we can do is we can bring any proxy with any amount of advanced configurations and any kind of policy support and still extend the ingress configuration to support all that. If you define your own annotation it is simple key value pair it is little more than key value pair because Kubernetes says you put a FQDN and then key colon and then value. So but it is ultimately key value pair. So you can extend the Kubernetes environment itself and Kubernetes configuration itself for the for whatever you want to define. In fact there is another way you can extend Kubernetes API as well but again like you can define custom objects etc but maybe that may be required for even higher cases but and that becomes little complicated because you are introducing your own objects there. This way the annotation way is very simple because you are not defining any objects. The users are not forced to use APIs because new objects introduced that can be accessed via APIs only. It can still be handled everything within the configuration and within the already defined framework. So that part was very good like on part of Kubernetes for doing this. Now comes the access control. So this is actually we wanted all these e-commerce companies or like whoever looking for compliance has to do and this example is like UI should not directly talk to database very simple requirement but there is no you cannot define such policy in Kubernetes because there is nobody to implement that. But when we have this custom proxy extended one that has all access control features built in then this is possible. But again the challenge is the same typically this kind of access policies are based on IP address. That is the identification when the traffic is flowing. But IP address in Kubernetes environment keep on changing that we all know. So now the way to do it is define policies in terms of service labels. Service label Kubernetes give fix and behind that service label IP addresses keep changing. So even if new pod comes in the new IP address is mapped to the same service label or if even if the scale up or scale down happens the change happens behind the service label all the time. So because of that the service labels can be used for defining the policy and it is not just the access control policy I am just taking the example of access control policy here. But any policy for that matter can be defined in terms of service label. So after this all that complicated architecture and complicated traffic flow is not required. The other one more security aspect again traffic flow security there is a lot of security containers that is like the code how the code is and how the image is created whether any vulnerable code is in there or whether any vulnerable library is used etc. Before creating the image but everything is fine even after that when traffic is flowing the traffic should be encrypted as standard SSL. But in this case let's see this services in this diagram I have shown like two different nodes but the service can be anywhere at some point the service can be on the same node and then somebody adds node or some other change happens in the environment Kubernetes can move that service to other node and that can happen all the time and that is the whole microservice concept. If the service on the same node there is no need for this all encryption decryption because it is on the same node nobody can generally intercept unless somebody hacks into the node itself. But when Kubernetes can move it to somewhere else or following the microservices architecture even the ops people can move it to some other place then the traffic is flowing on the wire and then it can be captured it has to be encrypted. In that case what we figured out that if we try to put all this intelligence and logic into the microservices themselves there is a lot of work involved. So the proxy that is outside can be intelligently provide that without again without changing any of this configuration or code for these services. So in the previous case we have seen that we have deployed a proxy and it intelligently not intelligently transparently just captures intercepts if not captures it intercepts all the traffic and then applies policies. In this case also it intercepts all the traffic and decides whether it needs to be encrypted or decrypted or nothing needs to be done. So if definition microservice is on the different node it needs to be encrypted. If it is on the same node nothing needs to be done. If it is an encrypted traffic and destination service on my node then I need to decrypt it. So all this intelligence if it is built then developers need not worry about it. They just deploy the traffic in the same manner sorry the services in the same manner and then the ops team who is deploying the application they can take care of all these policies. Next I will talk about some security sorry this visibility and analytics aspect this is not very specific to Kubernetes but in general. But I thought of including this because one analytics is a very waft term and different people say different things. So different when I was say for example in KubeCon last year December and multiple booth were there they were saying yes yes we do analytics. Then what what do we do really then ok we tell how many containers are running and when container IP is changing and those kind of things like container level. That is good good information and when container goes down when container comes up and all that. So good information but it helps also and not like it does not help but it helps at certain level it what it does not do and one more some cases were where people have created a good UI where it shows what traffic is flowing there. So it coming on the service one then going some traffic going to service two some traffic going to service three from service three some traffic go to service four some traffic go to service five and so on. Again that is important because one company I worked with their operation team didn't even know that. So they actually asked for it and then it makes sense for them. But let us go one more level higher and then all information is good but does it help really troubleshoot something when when it comes to that or does it help preventing a problem. If that happens then it makes much more sense. So it is like we are walking eyes open and what we are getting is is helping us what we are seeing is helping us and helping us in a different manner. Helping us prevent something so I can see the some pothole is there and I can prevent myself go on the side that then it makes more sense rather than when I am in the pothole then yeah pothole was there. So that is why I put this staircase. So we took a little approach and then figured out that we start showing analytics we start capturing specific information from from the proxy itself from the load balancer itself very specific to the application rather than at the container level and for container level actually we have the promethias and all that so nothing needs to be done separately whatever ecosystem is providing which is simply can use that. And then we start showing that in the context in the same context client information client context application information application context and so on. And we talk in the in we display things in a fashion so that we are able to figure out anomalies the traffic anomalies. So for example suddenly the traffic spike comes in that mean there is some problem either some problem either is a very good news or very bad news either suddenly people start liking our application so very good news we should know and we should improve or somebody is trying to really attack and then also we should worry about it and go block. So we need to go next level and figure out then who is sending traffic is everybody sending traffic one or two or three or five people sending traffic so that we can differentiate we can get an idea whether something somebody is trying to attack or somebody is it is a genuine interest in the application. But we thought of going one level further so that people the operate the whosoever is operating this application can have much more confidence in whatever he is thinking. So for example it should not be a false positive false negative where oh I thought it is a attack but it is actually not and I ended up blocking a genuine user then it really heard rather than help. So go to the individual transaction and look how things are happening is it coming from the real browser is are people you going through the same sorry a valid path on the application rather than say for example if somebody is doing just banging on the same URL again and again or it is going proper URLs and when somebody is moving whether it he has a proper refer or it is just a blank refer or some he has put some script that is just banging multiple URLs and so on. So all kind of differentiators look whether it is attack or it is a genuine interest and accordingly we can act then the whole thing start making sense. This is again this was asked by the users when application becomes slow and the report comes out application is behaving slow then typically in enterprise there is an infrastructure team there is a network team there is application team they start fighting whose problem is this? It is application problem infrastructure problem or network problem or nobody's problem sometimes but they do not know it is nobody's problem. So because of that we created a separate page and I am just using some small shots from there. So the chart on that side I am showing just three one is that the one that shows a triangle that shows whether application server how application servers are behaving. So if one application server behaves wrong then you see this kind of thing all servers are okay but one is behaved something wrong and the client also if you see the other side bar is the client two clients are experiencing delay but all other clients are fine and actually in this case even you are this case has all three problems one URL is behaving bad it is slow but all other URLs are fine. So this case in fact we have three all three problems and we can we can quickly very quickly identify which server has a problem which URL has a problem and which client has a problem but actually I have one more screenshot that that is not here. In that case there was nobody's problem it was actually one geographies one more graph was their geography and this one geography was the response times were very high and in that case when when people searched like what happened to that geography so they found that there was internet cable cut that internet pipeline that goes to that country there was some problem with that pipeline so entire country was running on low bandwidth and because of that everybody in that country was having latency issues and that is pretty obvious and teams fighting here that whose problem is this application team or infrastructure team or whatever team is nobody's problem somebody else's problem. So in that case when when this company figured that out and responded to the users who complain not everybody complains but some people are proactive and talk to the company that okay you are absolutely not behaving properly they responded to them that okay it is not our problem you can see that all your applications are slow right now because some this this incident happened and they provide references. So what result as a result what happened the customers who were like furious and saying okay we will leave your we will stop using your application and so on they finally well they as okay fine we are good that we are okay because you gave us the information. So overall it it does not just become the the team efficiency part and team does not fight each other but it also becomes a good user experience not not just the UI user experience it is that okay you are a as a as a as a application owner the company is able to respond to their users that that what really problem is and and just win the faith from the customer rather than just so it is not just about technology sometimes it is also about the relationship between user and application providers. So that brings me to the last one we talked about security we talked about application sorry we talked about application layer visibility we talked about load balancing so many things but the bottom line is that everything should be done very very simply. This brings me to one more item again back in KubeCon this year there was a round table conference that I was part of and everybody on the table expressed yes security is important but typically whenever this CISO team comes and then says implement security is very complicated they do not it just hinders in the process they do not allow us to respond to the customers and I do not know how to do that and many times we have seen in that round table and otherwise as well we need to do security because of compliance reasons not because we want to secure because compliance ask for it so what happens of security is implemented only as much as required to check mark the compliance or check mark from the CISO that yes we have done but actually things are not secure and then data breach happens and then company goes into I don't want to use words but that's okay different different scenarios so if it as a developer or as a as an engineer it's our responsibility and I'm including all of us here that we work on security or we work on any aspect of application delivery or any aspect of deployment or anything the whole thing should be simple and should be easily consumable rather than being complicated there was a time when complicated solutions used to work but now users are demanding and any users any level of users it can be end user who are using their mobile phones it can be internal users of some internal application or internal framework or anything and especially that this the whole Docker and Kubernetes it is taking it is flourishing it everybody is adopting because it is simple to manage it allows to do our work faster and easier and that's why that's happening so those security etc is little lacking there and the ecosystem players like all of us are create trying to create that but we should keep in mind that whatever we build it has to be simple and yes sir okay so engine x ingress I actually touched upon that I think I talked about engine x as well so at this point what engine x did engine x of course didn't have this external hardware based or engine x has always the smaller one so it goes that area what engine x did instead of implementing this monitoring separately they built it on the load balancer itself so in case of engine x it is the same proxy that does that takes the traffic and is the same proxy that monitors the environment also so what happens when engine x scales so at this point it is fine but when it comes to this area so now scale happens and engine x there are multiple instances of engine x running and everybody is monitoring vis a vis one component monitoring and communicating to everyone else that is the only different approach I believe and engine x create engine x open source didn't have that central controller but engine x as a company trying try to do engine x controller as well how much it went ahead I'm not really sure but they try because they try to cater this entire thing into the open source area and they built they built ahead they built together and because of that each instance and especially when it becomes like larger deployment then each instance is monitoring so that goes to the not too much that's okay means I don't I don't have the data on like if it adds to anything or not I don't have that data is why that's what we did sir so by the way I didn't talk about what it and did it and does so 8n actually took engine x core about to three years back actually we started in 2013 that was a different company not 8n a different company that got acquired by 8n later on so we took engine x core and then developed some modules on top of it and it started doing so now when 8n provides that service the proxy is headless so all configuration is through central controller and that configuration goes automatically in and the we wrote the modules for collecting the specific information ingress is yes ingress API is from for the Kubernetes part but this we did actually before Kubernetes Kubernetes portion came later but we created a separate proxy that was initially for the cloud native environments and then we ported it to container native environments as well so those modules we have built and from from that the whole thing works again because you ask this question building the building the module that that gets the information of course some changes are required at the within the engine x as well but doing that is not really very difficult the difficult part is actually reverse so engine x itself is is works like instance by instance but when you want to deploy multiple engine x they are single they are not a cluster so so the operator or the like application owner or operations team whosoever is managing is they need to go to each instance and configure it but if you if we want to give a single cluster that automatically gets configuration basic configuration is good if you just say traffic rule simple ok this traffic goes here this traffic goes there simple but when it comes to like limits when I say limits then if there is there is a particular user should be allowed to send only 100 requests now this has nothing to do with the niggas by the way see there are there are certain policies irrespective of environment say for example if you want to mitigate DDoS attack or you want to limit traffic coming from single source then you say okay this source is allowed only 100 requests per second or per minute whatever the thing is now if if the requests are coming all on the single instance it is simple simple to limit you put 100 be happy but if you have deployed like 10 or forget 10 like four instance only still like mathematics is very simple okay or 25 each but you really do 25 each then will it work properly because somebody can send traffic here here oh total is still less than 100 I sent like 50 on 1 and 30 on 2 then what should I be blocked should I not be blocked ideally it should not be right so this kind of policy is bringing those in is become really challenging that was the challenging part getting the data out was not really challenging part whether of course we need to put some hooks in there and then collect and getting the data out in the efficient manner was the little challenging so we need to create because if you put too much of too many taps into the data plane then the data plane goes slow so efficiency of the entire system goes down so because of that we we put the minimal hooks and then collected locally and then club did and then sent to the to the central controller thank you for the questions but it was really good one yeah okay so I got the time out so maybe we can continue talking and thank you everyone for listening yeah thank you