 So, hi everyone, I'm Dipali Trivedi and I work as software architect for Constant Contact. Just to give you some overview of the things I did, when I was in college school, right? I worked on few internal applications. When I came out of that and I joined professional career, I worked on few startups, few unsuccessful one and few very successful one, right? And got chance to work on many enterprise applications. And during all these years, what I realized is that if you are developing a product or an application that is for hundreds of users or thousands of users, your architecture is very different versus the product that will be public-facing, internet-facing and that will have millions of users. So, if you are working on a product that has thousands of millions of users and have many transactions per second, it needs security, it has mobile app, your architecture will be very different from the application that is more internal. And that is what we are going to talk about today. I mean, that is the microservice architecture. Should we take up, understand microservice architecture, take up all the complexity and deal with all maybe issues with that if you are working on a highly scalable platform or highly transactional platform? So, just on the agenda, we will discuss the more traditional way of designing an application, right? Maybe if you wrote an application or product 10 years back, that is mostly monolithic architecture. We'll compare and contrast that with microservice architecture and discuss the advantages and disadvantages of both. We'll talk about scaling, because scaling is the key if you are working on SaaS product and especially a successful one. We will talk about interest service communication, one design pattern that I highly recommend if you are using microservice architecture and the transition, right? So, if you are working on monolithic architecture, are you stuck with that forever or there is a way you can really go to the newer way of writing your application? So, monolithic architecture and even though this looks very surreal, it's not really that fun if you are working on monolithic architecture in my experience. So, just my first job out of the college, right? And it was a SaaS product. It was a mid-size, it had few clients but it was not really that big of a success. It was in a mid, three, four, five years old product. It had 60 developers and all these developers were working on the same code base. It had many modules but in the end, all these developers were deploying that code like taking all their changes, deploying that as a huge ball of code and put it in a container. And when I joined that company and of course, being the first job, I was super excited and I said, today I'll take up a defect or I'll take up a work and I'll fix it, right? It took me a week to set up my project because I had to, it has EJB, it had multiple container, it had all this code base, I had to, it took some time and it was painful to really get to a point where I feel that I can work on any part of the product, no worries. So, that is the typical monolithic architecture and I mean, there are three layers mainly and for a long time, actually we always thought that application architecture should have three layers, right, MVC. One, first is the browser which will make a call to load balancer. All your code, your product module, customer module, payment module, all these modules are part of this one single deployable year, I am from Java so I'll talk about Java related thumbs and technologies. So, all these modules will be part of that year and deployed on the container. Browser will make a call to load balancer that will fan out to different server and then all these modules will talk to database and serve the request. Most of the time our product is all about write, read and delete sometimes and create. So, that is the typical monolithic architecture if you are working on e-commerce, just very simple example there. What are the issues with monolithic architecture? So, first is a very steep learning curve. If you are a developer and you join a team that is working on monolithic architecture, it will take you a long time because the scope of your code base is very, very vast. You have many modules and maybe for some time you will work on one side of the product but to really understand everything that you are working on, it will take you some time. So, developer is overwhelmed when he will start working on a monolithic architecture. Slow development and IDE. I mean, anyways when you start IDE it makes your laptop slow, right? And if you are going to import 10, 15 projects then it is going to make it really slow. And many times you are maybe just working on one project so even this big monolithic architecture might have some modules inside that. All these projects will have dependency on other projects and your project is dependency of some other projects. So, in the end you end up getting many projects in your IDE and that will make it slower. If you make a change it will end up compiling all these classes, it will make it slow. Think of a build time. And again, if I go back to the job that I was talking about, the build time was 18 minutes. And if I run test it takes couple of hours. Regression was because the scope of testing was really huge and I wanted to make sure that I do not really mess up anything. Just to tell you I drank a lot of coffee during that job because I couldn't really sit there and watch my computer running all this test and build. The biggest problem I see with monolithic architecture is Big Bang, right? If you heard about continuous integration and all this cool platform deploying to production multiple times in a day, you can't do any of that because just your regression take couple of hours, right? Now think of running all other tests and all this developer making changes at the same time to the same code base you cannot really take just your changes and go to production and deploy that. So ideally in most of the monolithic architecture product development, all these developers will work for a couple of weeks or four weeks and then they will schedule a time for deployment. And everyone is scared of deployment. I don't want to be the support person for this deployment because there will be issues, right? There are 60 developers wrote code without knowing each and every side of the product. So it is a Big Bang, you hope for the best. And if you say issues, you spend days figuring out. So your time to market is really slow. Long-term commitment to the technology stack. You feel like, so today if I'm going to start a product I'll think of the best technologies and tools available today, right? So most of the monolithic architecture chose the technology that made a lot of sense at that time. But after a few months there will be new framework, new library, and in a few couple of years there will be entirely different way of doing the same thing. I mean, technology space is so innovative. You always have some other way or better ways to do the same stuff in a couple of years. You cannot really adopt new technology if you are actually a monolithic architecture. So if you go back to the previous diagram, if you're in database, if I'm using Oracle or if I'm using DB2, and I think for my use case, MongoDB makes a lot of sense and it has sharding, it will have high scalability, I can still query my database. You cannot really go and adopt MongoDB because if you want to do that, you have to change product module, customer module, payment module, authorization module, order module, all this module at the same time, change it to MongoDB, test everything at the same time and deploy everything at the same time. That means you have to tell your business that for a couple of months, we'll not do anything for you. We'll just go to MongoDB. Honestly, that is possible, but we never have that luxury of time when we are working on really a business use case and features. So most of the time, a team working on the monolithic architecture feel like one of these people. Development team. So if you talk to a product owner or if you talk to a business side of people, they always complain. We are not going fast enough. We are not delivering a feature fast enough because there is definitely many competitors out there who are developing the same thing that you are developing. And you want to be the first one going to market. There is a huge advantage of that. So many times, even company has money, lot of money to really hire new developers and go faster. But if you are working on a monolithic architecture, you cannot really do that very effectively. Because when you add more developers, first of all, those developers need more time to start adding value. Second is that if you have more developers, still they are all working on the same product. So instead of 60 developers, if you have 120 developers, it's a bigger mess, more coordination. So even though you double your velocity, your productivity is not doubled. So really, it's hard to see the advantage of hiring way to many developers if you are still stuck on this big architecture and big code base. So what can we do about it? What is a better way of doing it? And how all these other modern platforms are writing their product? So that is an answer is microservice. You should have microservices architecture and you should read microservice. So even before we go to the architecture, what is microservice? And the biggest comparison I can give it to is a LEGO block. Think of each microservice as a LEGO block. So each LEGO block is an independent unit. And it can, so when you combine multiple LEGO blocks, you can make different structures. So that is the advantage of that. And that makes it agile. So similarly, each microservice is an independent unit that you can develop, deploy, test independently. But when you combine all these microservices and then use it in a particular format or particular sequence, then it will serve your use cases or it will serve your product. And that is how you should see microservice architecture. You are actually building some structure using LEGO blocks. Scalability. So we are doing, you can be adopting microservices architecture for other reasons, but the main reason I see is scalability. And I'm using XYZ scaling from this art of scalability book, which is a very good book to read if you are working on a scalable product. There are three ways to scale your product. Well, first is XX6 scaling. So this is the traditional way of scaling our application. We have been using that for many, many years, right? What we do with this one is that maybe you are starting to product this huge year file that you are deploying on a container and you have a cluster of 10 nodes. If you see some success and you see 10X scalability or growth for your business, instead of 10 nodes, you put 100 nodes out there. So you are just replicating your code base on multiple servers and that is XX6 scaling. What is a problem with that is that all these modules are still talking to the same database and that becomes your bottleneck because now all these modules are running a query that will join across tables and eventually when you have a lot of data because we never delete data, right? Eventually that will be a bottleneck. So what is the next level of scaling? That is why XX scaling. So if you divide each of your functionality in a smaller pieces and make a separate service for each of that, you can deploy each of these services on different cluster. So here in XX6 scaling, if you are seeing more load on SSO, so you want to scale SSO 10 times, but product catalog maybe is the same. People are using product catalog at the same scale and you don't want to scale product catalog service. So you can't do anything here but if you have divided your SSO as a separate service and product as a separate service, you can add more load or more nodes on SSO service and then you can still keep the same scaling for product. So you can scale actually different models at different scale if you have divided your application into a smaller unit, right? And that is more of a dividing that into microservices. But the first problem remains the same because if all these services are talking to the same database, same cluster of the database nodes, then you are still having the same problem because eventually you will see issues at database layer. And that's why this is a modern way of designing your application is XX scaling and that is data partitioning. So ideally each of your services should have its own database. So SSO should have its own database, product should have its own database. So ideally you divide your database layer also into multiple units and that is the combination of Y-axis and Z-axis makes microservice architecture. And that's why it makes it extremely scalable because you can use all three methods when you want to scale microservices architecture. Just to give you the example here and I'm going to take the same e-commerce example so that you can relate to that. So if you have to take that monolithic e-commerce system and write it as a microservice architecture this is how you can divide your services first of all. So you will have authorization service, customer service, product service, audit service, payment service. So there are many ways you can divide your application and this is, it needs some experience to really figure out how you divide your application but the methodology that I use is domain driven design application. Domain driven design. So what I do is that I draw the domain diagram for the product and I take each domain entity and make a service out of it, right? So that really gives you some idea how can you divide your service but there are people who take verbs and then try to go by verbs. Some people take nouns and go by that but I try to really do the domain design and then take each entity as a service here. What is the, what happens on the UI layer, right? And for a long time people generally focus more on backend and how to scale the backend but now given different types of UI and with mobile app just there are just way too many UI for each product, right? So what can you do for the UI? So if you have a UI that has 25 screens you do not really have to divide each screen in a separate module, right? But you can think of similar use cases and combine that and write that as a separate UI. For example, here you can have customer UI, consumer UI, maybe catalog UI, which is a more reusable UI component that you can use in multiple times. And here all this UI will call services in different combination. Maybe my arrows might not make sense but just to show you how you can divide your UI into different applications. So here, and of course in the end you can show all this UI components in a, you can show all this UI components in a uniform UI experience, right? You do not have, they don't need to log in again to go to all this UI. You can run those in iFrame or this UI or some other ways so that it still looks similar and you can use the same look and fill CSS stuff like that so that it doesn't look that they are different applications. But the advantage here is that you can have different teams working on different UI part of applications here and then you can make progress on all these things, right? And the developers working on customer UI has to just know about the use cases of that UI and not product catalog and all the services that it is calling. So somewhere you are asking each developer to just focus on their particular part instead of doing everything. So one biggest point I want to mention here is that and many people who, I'm sure like many of you must be using web services, right? And people think that if I'm using web services, REST services or SOAP services, I'm using microservices architecture. That is not true. You can still have monolithic architecture and have all your functionality exposed as microservices or services or REST services, right? But you, if each of your service has a separate database assigned to that, then only you are using microservices architecture and this is why I want to just show it here that here each of the service has its own database and ideally you should never do joins or there should be never more than one service talking to your database and that is the best practices and guidelines for designing microservice architecture. So when you design your microservice architecture, there are few points you might want to think about, how small or how big should be your service? If you have too few, so if you have a huge codebase, 4,500 in classes and if you have just three web services, three microservices and maybe one or two UI app, then you are not doing microservice architecture because you are doing small monolithic architectures. Each of your services is still too big and you will not see the advantage of that. If you go crazy and if you have 2,000 microservices, then that is too many because in the end you have to maintain those, test those, deploy those and manage the performance because in the end your data is divided into 1,000, 2,000 services and how do you even read across those things? So you should use the right amount of microservices, just not too many, not too few. So now you realize that if you, you are working on monolithic architecture and if you are going with actually microservices architecture, instead of one unit, you are going to have multiple deployable units. So instead of one this big year that you have to test and develop and deploy, you are going to have maybe 30, 40 deployable units. So now the key to success, so if you are going to manually deploy all those units, you are signing up for failure for sure. So before even you divide your services or divide your product into microservices, you should have continuous integration and continuous delivery in-place. So what is continuous integration? As a developer, when I check in my code, ideally that should kick off a Jenkins build or some other kind of build that should run all the tests, unit test and regression test and then once it passes, it should go to the next level of environment. Where it should test all the integration tests and once that will pass, it should go to the next level of environment and then it should run all the performance tests and then once everything pass, it should go to production. So ideally the time developer checks in, there should not be any manual test before you go to production and ideally you should be able to write some code and then go to production without really talking to other teams and making sure everyone is fine with that without even coordinating because everything should be backward compatible and it should not break any existing functionality. And to really do that, you need a lot of tests. So developer should write tests, tester should QE should write tests, integration team should write tests, performance team should write tests and ideally that is how you can deploy your code. I mean once you do that for few months and you get better at that, you can actually deploy your code multiple times in a day without any issue and every deployment again is an uptime deployment so you do not have to worry about downtime or anything like that. So ideally you build, you deploy, you test and then you release and you just continuously doing that without having any overhead and that is theoretical, you will have some overhead but having minimal overhead and then only you can be successful if you are really using continuous integration here. Advantages, so what are the advantages? Simple is always better and there are many people thinks that microservice architecture is not really simple but if you think of view of a developer, it is very simple. If you are a developer and you are just working on one team that has few microservices, four or five and then you have one UI maybe and taking care of two, three databases, I think your world is very simple and it's easier to understand, it's faster to deploy and build and it's very easy to test because the scope of testing is through. So it's actually possible for you to know all the use cases of that service and then add some code that makes sense and doesn't break anything. So it works really well for the developer. Adoption of new technologies and framework. So think of a scenario that you wrote a web service or a web service few years back and that has JSP, Java, EJB in middle tier and it has maybe DB2. Today you have a new initiative and you want to spin up a new service. You know that maybe EJB doesn't make sense and JS based way too heavy. You want to use Node.js, you want to use Spring and you want to use Cassandra. You can do it because all these units are independent units. They will not share any database or code base here and they will just talk to each other via REST, call or SOAP, call or AMQP which is technology agnostic. So behind the scene you can use any technology that you want to use. So just giving you example and you should not try for this because this will be disaster if you really use different databases and technology for each of your service. But you have flexibility to really do that and that is the really good thing for the developer. They are empowered to use the right technology if that makes sense and really spin up a new service. On the scaling of development teams. So think of a new, there is a new initiative you want to businesses or we have money, we can hire more people. You can really spin up a new separate team that will write new services or you can add those people to the existing team and really scale your development team linearly. Today we want technology that can scale linearly. We want development team that can scale linearly. It's a linear scalability. Ideally you should have a small team. So I say two PISA roles, your team should be as big as it can be fed with two PISAs. That says some, it translates to number of services, microservices and UI component that it should be owning. I mean, if you have one team that is taking care of 20 microservices, then you are not really, you will not see the advantage of microservice architecture in terms of development velocity. So these are a few of the advantages. Just the next point here is that service to service communication, right? And there are most of the time companies who has worked on monolithic architecture for a long time will go for it with synchronous communication between services. But I just want to specify two types and give the exposure that ideally, if possible, you should use both at the right places. So what is the synchronous communication? It's like Joe is calling Mary and Mary should be available to take up the call at that time and answer Joe and then only Joe can call someone else, right? So if Mary is doing something else, Mary will not pick up call for one minute, Joe is really waiting there and not able to do anything else at that time. And that is a more like traditional way of calling services. That is more of a request reply mechanism using HTTPS or it can be actually restless on SOAP XML or anything like that. It is simple. It is way simple to design or develop that. It is way simpler to really show your error messages and stuff like that, but it's tightly coupled because so just translate that to the services, right? Service A has to call service B and it's waiting for some response there or some information there. And if service B is slow or it's down, then service A will wait for some time until it will time out and that will make service A slower. So ideally, if you have synchronous communication in your enterprise, every service communication is synchronous, then one slow service can take everything down or make everything slow and you do not want that. And that's why every organization using microservices should explore asynchronous communication. So what is asynchronous communication? If Joe is calling Mary instead of calling that, he'll just text it. And now it is who even calls, right? Everyone texts. Yeah, he'll just text it and Mary might be doing something else but when she'll have time, she'll reply and say, oh yeah, you needed that. This is the number and yeah, go ahead. But meanwhile, Joe can be texting other people and he can be doing other work. So he's not just waiting for Mary to reply and that is an asynchronous way of communication, right? So here, if service A wants to talk to service B, it will just post a message and then service B will pick it up whenever it will have time and meanwhile, service A can do anything. If service B is dead, then we'll talk to the failure mechanism but service A is not down. Service B can be down but service A will not slow down or will not be down, right? And that's the advantage here. So ideally one failure in your organization will not take everything down. You will see some issues but you are still up and running and that is the biggest advantage here. So and there are many, many frameworks and many technologies that can support that. So you can use RebitMQ, you can use ActiveMQ, Node.js, Kafka. There are way too many frameworks that you can choose from and really support this model. If I stand here and tell you that microservice architecture is awesome and once you use it, all your problems are solved and that is a lie because there is no architecture that is perfect, right? Every architecture has pros and cons and you should be aware of the cons as well when I tell you this is awesome and you should use it, right? So the disadvantage is that it's a distributed database and people who has worked on monolithic architecture for years takes long time to really accept distributed database because they are so used to doing a jointed database layer, doing all this logic database layer, just doing all that service layer seems complex and slow and not fun, I would say. So it has distributed databases and that means that you really do not have database transaction. So if you are going to write to service A, write to service B and write to service C and if C is down, you can't write to service C, you are stuck with this orphan records in service A and service B because there is no foreign key constraint against all these services and in so other ways to compensate and stuff like that but in microservices, you just deal with those orphan records when you get chance, right? So that is the biggest disadvantage here. Service versioning and maintenance. So if you are using microservices architecture, you should understand the backward compatible changes, non-backward compatible changes and then design your versioning strategy accordingly. You can use minor version, measure version, you can use database version, you can use any other mechanism. It's a topic, it's on its own, how to do versioning and maintenance. Chet UI, right? So I read somewhere when you load a Netflix page, it talks to at least 7,200 services in backend. So it's a very, very Chet UI, right? When each of your UI pages will have multiple calls and the problem here is that so think of a e-commerce page which will show you order history page for yourself, right? All the things that you buy. So it will show you order data, it will show you product data, right? And it will show you some payment data that you use this credit card, start, start whatever in 5, 6, 3, 5 or something like that. So actually it is reading from multiple services and show it so that page needs read from multiple services. Now who should do that combination or aggregation? Should you write a new service that will really combine everything? Should you write a new layer that will combine everything? Or should you let a UI do the combination, right? An aggregation. And if you have 5, 6 UI for your product, you do not want a UI to do that, right? Because then you have to do that in every UI and that is against reusability. So, and that goes back to the next thing I want to talk here is the API gateway pattern, right? So the platforms that use microservices architecture often go with API gateway pattern. So the problem I mentioned here, right? That there is a web application that has one page that needs data from multiple services and either the web application can do it, layer can do it or there can be a new service here. And what I'm proposing here, what I'm not proposing, the people who use microservice architecture and use API gateway propose, they try to new layer for that. So that is called API gateway layer. Some people, we here at constant contact call it platform layer. You can call it any layer that you want to. But the idea here is that this is a single layer that will talk to services, manage the common functions and then combine data or sometimes it's just a pass-through. It doesn't have to every time combine data. There can be some UI that just show your UI user setting, right? And it doesn't have to combine your data. So it will just be the pass-through and then it will manage the complexity. So what are the advantages of using this API gateway pattern? So first advantage is that, so just to go back to the previous one, right? So here you have this multiple, you have web application mobile app and third-party UI or anything like that. Earlier when we did not have mobile application in the picture, we could have all our UI just living in our data center, right? We can just deploy that locally and run it from there. But now with mobile applications, you really need all your API to be public-facing. It should be accessible by internet. Most of the web application today is pure JavaScript app, just living in CDN. So you need all your services to be public-facing. Do you want to open up all these services to public-facing and really deal with the security or you want to have this common layer that will handle authentication and authorization? And that's where this layer is helpful. So that's where I start with the biggest advantage here is common layer that will handle authentication and authorization. Second is that if you have used Java or X or any other reactive programming, this is a reverse way of doing your programming with API gateway, you can do it because if your service layer or data will change, API gateway will be observing that and getting the notification and then you push that notification to the UI. So instead of UI making the call, now everyone wants the reverse way of pushing your notification to UI and you can use that, enable that if you are using API gateway. Third is that as I said, you should not have just synchronous microservices. You should have HTTP-based microservices, REST-based microservices, and now synchronous like MQP-based microservices. So now if you have combination of all these services, do you really want UI to handle all this complexity and remember that for this service, I should use this kind of client and for other services, I should use that kind of client. Instead of that, you can have API layer handling that complexity and remembering that for this service, I should do a MQP-based client versus HTTP-based client. Service registry, service registry is not must have most of the time if you are deploying in your data center because there is a fixed number of nodes located with service and then each service has the specific load balance or assigned to it and you always call through that, right? So there is no dynamic discovery of the service needed, but if you are in public cloud and which might have failover and then your service can move to some other zone and there can be many dynamic addresses for your services. At that time, you really need service registry integration that can discover your services, service dynamically and then use that service at that time. So that integration can live with API gateway. Partial failure handling and this is like my favorite topic. So how do you design your product that can handle failure, right? So if I don't know if any one of you come from SOA background, but in SOA we used to say that each service should have 100% up type and which was, which is very theoretical. It is not possible. You will always have downtime, right? And you should design your interactions. You should design your product to handle those failures. So the example here is that if your product service is down, right? And for example, if you have recommendation service, you have recommendation for the product that user can buy. And if that service is down, instead of not showing anything or showing any error or message or anything like that, you can just show top 10 product, right? That is very successful and you can, so ideally you should, it's not just the technical design. It's a functional and technical design to think of partial failure and design your product and interaction to handle that. And those complexity can live in API Gateway. So API Gateway can call the recommendation service. If it is down, it can call product service and get the top 10 product from that one. So those complexity can be handled there. So is microservice is right for every use case, right? If you are working on anything, is it right for you? I would not think so. So if you are working on a startup, you have five developers or four developers, limited fund and limited use cases and you have no idea whether this will be the next big thing or this will just, someone will throw out, right? I mean, at the hard time, you might not want to really take up the complexity of microservice architecture. You can develop that in a more traditional way, monolithic architecture, which will be faster initially, right? With five developers, it will be really fast. And once you see the success when you get the next round of funding, you get feedback from the customer that this is awesome, then you can re-architect your product and really write with microservices architecture. But if you don't do it then, you can't really do it with three or four or five version of your products. So do it as early as possible. When you think that this is something I'm going to maintain for five to seven years, that is the time you really divide your application into microservices and go with that. Second here is that the biggest push we saw for microservices architecture was because of mobile development, right? Monolithic architecture or more traditional way of architecture was focused on one UI, one business layer and one database. But that is not true with multiple UI. And if you really have multiple UI, you are going to have multiple mobile app and stuff like that, then microservice architecture will give you a lot of advantages because you can really decide whether you push some of the business rules on services side or you push it on UI side, how to reuse those things and how to have public facing API actually for your product. How to transition? So if you are working on a monolithic architecture and if you think, oh, I mean, I don't know when we are going to write second product or third product or when are we going to do the new version of our product? How can I use microservices gradually, right? So you can do it. It's only the mindset change. You should be able to really educate your team and really start with that. So one way of doing it is that any new feature that you write or any new domain entity that you add, you should write that as a microservice. It should have its own database here and then you should have some blue code which will make this integration work. So your monolithic architecture can call these services whenever it needs some data from that entity. So eventually, as you have more functionality and more domain services, you will have more services on this side instead of keep adding more stuff on this side. And once you have more experience with microservice architecture, you can actually take smaller part from this big ball and put it as a service. And then eventually, this can shrink and this can grow. And then still you have to maintain your monolithic architecture but you will see fewer issues or you will start seeing the advantages of it up. And developers can start using new technology on this side and really see advantages for that. So, and it is complex and many companies don't do it because writing business layer this way is easy but migrating data is the problem, right? And that's why many, most of the companies end up rewriting their product if they have time and money for that. But there is a way you can make it work and start seeing the advantages of microservice architecture. And that is my last slide. Thank you and any questions?