 Good morning everyone. So everyone had their breakfast and ready for an exciting day of talks I hope. So what I'll discuss today is how some of our experiences of what we learned as well as the tools and techniques that you can use when you're developing in a large monorepo. So I am Jay. I work as a senior manager in Microsoft. So I'm from Bangalore. So it's a lot hotter here and a lot less traffic as well. So I work as my team builds a lot of search experiences for a product called Microsoft Search. So Microsoft Search is like a unique and cohesive search experience which you build across all Microsoft apps for all your work based content which can be like all Microsoft funded as well as the non Microsoft apps that you use. So that's the app that I have. So essentially like yeah we work on a search box which gives you sessions and then a bunch of search results and as the complexity and based on the query as the confidence and we have more insights into what the results should be. We also add a little more complex stuff in the search results. So one of the things that we consider search as is a high value experience. So high value experiences are something that we consider that are like most important user actions that are done across all the office apps and across devices as well. So these are some things that are used across the Microsoft Office suite. The other part that's yeah so the other part that we consider really important is coherence. So whenever we build these experiences we want to ensure that we have like a familiarity of the interactions as well as the experience across all these applications. So it could be like across outlook across teams, across Word, Excel, PowerPoint as well right. So across all these apps it should feel similar as well as it should have native device based interactions for example if on a mobile device you have long press and things like different interactions like that you should honor that as well in the app in the applications. So instead of search I'll probably explain with a slightly relatable experience called people card. So people card is something that we started like a digital listing card across Microsoft apps. So it used to contain like contact information where you set who you report to what's your structure and stuff. It slowly kind of improved to a little more detailed information about your work profile and your LinkedIn profile as well. And but over time we added a lot more things to this people card right. So you get insights of okay if I have to set up a meeting like what is a more convenient time between the two of us like you have contact details your org structure your recent documents or like conversations between the two of you and like there's a lot more complexity that gets added. So given that the people card has in itself has got a lot more complexity like I said and imagine this as like a common experience across all the applications like word outlook teams and like SharePoint and all these applications as well as on each device as well right. So for this one of the things that you really need to when you're developing these experiences one of the things that you need to think about is agility right. So when you're thinking of agility I want to I would like to put it into two loops I call one as an inner loop and an outer loop. So in the inner loop is like what a developer does every day like most first write code you test it then you raise a pull request someone reviews it you get it merged into the master branch and then once it's merged in master branch then you have like you release two different rings in the production and then you get feedback from the users kind of add some more features and next steps and write code again right. So this kind of seems okay so this is okay doable but think of it when you have to do this across six platforms right. So when you're doing it across Android iOS, UWP, Win32, web and Mac, macOS. So a lot of times we kind of tend to have like separate silos where or separate reports where all these components are developed and the source code is different the teams that the people that work on them are also different right. So given that it's very hard to get a product agility importantly where like you can't release all of them simultaneously you can't have you can't share a lot of code you have to redo a lot of stuff as well so that's the reason we looked at moving it to once a single source code repository and kind of have like a community of developers who can add these components for all these endpoints right. So just to summarize so if you have like a people cut of this like this across all applications like Outlook, OneDrive, Word, Excel Powerpoint, BooksharePoint teams and then across six platforms right. So when you're thinking of getting a good agility as well as ensuring there are the experiences are coherent you have to remember one thing that no one size fits everyone right. So which means that you can't have like one solution which can work across all these endpoints okay. So to simplify that one of the most important things that you have to do is breaking things into small packages breaking your code into small packages. So that's the first thing that you have to think of doing for example for the people card I would like to I mean I would break it down into a people core package then a people API package auth package a data layer and a bunch of components and then a thin layer for each of the endpoints like Android, iOS, Mac OS and so right. So so few things that you get a benefit out of when you're breaking into smaller packages are like reuse can be made easier when each package is they have like a single responsibility right. So you each package knows what it needs to do right. The other thing that it also does it is it encourages developers to have like additional thinking and develop a little more and thoughtfulness in how you expose the interfaces between the packages and how you ensure it's a unit and itself right. It also encourages more of composition and reuse patterns as well right and of course like if it's smaller it can be published easily made changes you need to worry about the entire universe but ensure your block is you know getting published easily and released well and that's why I can also manage releases easily right. So given all this when you have break your code down into multiple repos this is what we call it as a mono repo where you have like a repo which can publish multiple packages. So the main essence that mono repos bring is singularity in code right. So what is a mono repo it is in general to in short a repository with multiple packages right. So why multiple packages I think I've already touched upon this but yeah like other repositories think of say team sap wants to use the org chart module in some internal modules in teams which is not related to people right. So I can just reuse that org chart package as a module in teams right. So that is where it actually makes a lot of sense it also ensures that it behaves as a code reuse unit means like you can share a lot of tooling you can actually you know share a lot of things so that whatever you build is coherent enough right. The other reasons why you want to put in one in the same repository are definitely like developer agility where like as a developer you can actually move faster because you can reuse what other people have written as well right. A lot of code consistency can be maintained. So if you have the right linters and like some of the best practices followed you can have and if you put efforts to ensure it's homogeneous across the repo any developer can easily figure out what's the code right and you can unify a lot of process like build deploy and so on as well right. Code discoverability CI CD and so on as well. So this is something that helped us a lot moving to a mono repo structure because think of like in Microsoft in general there are about I think about 100,000 developers across the company and you have like so many applications and there are about 200 packages that we publish in that mono repo and more than 400 developers who are working on that right. So that way it actually helps a lot when you kind of simplify your code into smaller packages and kind of reuse a lot of tooling release process a lot of other things that you get in a single way right. So one of the major things that you need when you're moving to a mono repo is like you need lots of good tooling right. So some good tooling around package management the whole package lifecycle how you publish it how can you make it common and easy for every developer. Cross package orchestrators so when I by this I mean like something that you want to run across all packages simultaneously or like uniformly rather. So that's another thing that you need because now instead of a single repo now you have a lot of smaller repos right like when you think when like before we used to have like one repo per package but now if you're plugging in a lot more packages in the same repo you need a lot more cross package stuff right. Of course a lot of common things like linters testing tools build system infrastructure. So the bottom three and like more like that's something that you can add by yourself but two key things that you definitely should have in a mono repo and consider thinking about it are like package management and cross package orchestrators. So we'll look at these two in detail in this talk. So let's start with package management right. So earlier you had like a one single JavaScript package now when you have a repo you have like multiple packages in the same repo right. So like I said in our repo there are about 200 packages that get published and then if you look at the external dependencies that it brings it's a whole lot more right. So you have to deal with this entirety of number of packages for even to start development right. So let's try to understand how packages work in a node development scenario right. So you write the package configuration in packages and file everything understands this right. So you install it with your favorite tool npm or and what this does is it fetches your package content from somewhere and puts it in your hard disk right. Now when you think of node in execution whenever you require a module right node essentially looks wants to look for a file which can give what is the module and it will essentially resolve it right. So if you kind of separate these two concepts in mind we can kind of understand how node modules also like resolves your modules. So like whenever you say require react first node tries to look at your current package node modules folder if it's not found there it'll do one up and try to see in packages folder and then like and so on and so forth until it can go to the root folder right. So this this is how node tries to resolve for a module like availability of a file on the disk in this fashion right. So like you have like package one and package two in two repositories. Now we are trying to put it in a single mono right and these have their own dependencies like for in this example package one has a bc dependencies and then say package two has a dependency on b you can see that b version one is a common dependency between the two. So when we try to do an npm install it will try to install two copies of the same version of b right. So now let's if you understood like since we saw how node module resolves the structure like how it resolves a package if you kind of move this node modules folder to the parent folder which is of the mono repo in the node modules folder you can add a bc and then of course like since there's a different version of b for c1.0 like it still maintains that order. So this way you are kind of dedoping the installation of b right. So there's only one copy of it that gets installed. In a mono repo the other things that you also would want to do is to create sim links of the packages within the mono repo. Say later on I want to add a package three which can take a dependency on package one you need to install it again it's already present. So given so this concept is called package hoisting. So when you try to move the package installation to a parent folder kind of hoist the package installation right. So a bunch of tools help you do achieve this right. You have rush js which is something Microsoft open source a while back and you also have yarn workspaces a lot of us use yarn so yarn workspaces is an easy understandable thing. We also have you can also use pnpm workspaces if you're using pnpm as your package manager then there's also another popular open source library called learner which you can use for package management as well. So I'll just walk through one of them which we use in our mono repo but a lot of concepts are similar right. So we use yarn workspaces so here to use yarn workspaces you have to do a couple of small changes to your package json. You have to ensure that this you add a new package json at your mono repo folder right. So it's not inside the per package package json you add a package json to the mono repo folder right. So in this essentially what all you need to do is you have to ensure that you set private to true right and you add another object called workspaces here you can give a list of what packages you want to be managed in this workspace right. So if you want you can give a list of packages or you can also use a regex. So this is generally how most of us do if you want to keep adding new packages to that folder you just give a regex path right. Optionally you can also specify okay so for some packages you don't want to hoist it you want to manage the dependencies internally you can still achieve that where you can add a no hoist key with the packages where you don't want to hoist packages right. And the other thing the other changes you have to run yarn from the root of the mono repo right. So when you run yarn now the package installation it came down from about 20 minutes so every day like when you do a git pull from master and then you try to do a yarn it used to take a much longer time now it reduced to six minutes we are working to bring that further lower right. But yeah this is something that like using workspaces can really help you and understanding this concept of hoisting packages will actually help you a lot in ensuring you don't have to you know reinstall a lot of duplicate copies. So the other part that we talked about was cross package orchestration right. So in general for a package itself you have a lot of things to do this like a subset of what I could fit into the viewport of the things that we do on a package right. So we have like a bunch of pre-built things you can do you have some generator commands you have some linters you have some test commands that you want to run and so on right of course watch as well. So given all this you want to kind of have this if you're a developer and you're breaking your larger package into smaller packages which means like you want to have a same way of running the same set of commands that you can run across packages right. This means that you need to have like a homogeneous way to run these commands or your day to day activities across packages not just your set of packages the packages that you own but also like you can easily do the same for other people scored as well which which has been added to the monorail. So for cross package orchestration again we have a bunch of tools so Russian learner feature here Russian learner feature here again so these help you to kind of run tasks across all these packages in a uniform manner. You can also use gulp shell scripts simple shell scripts or we also recently added just scripts which is like where if you have a lot of custom scripts already which you want to add you can use them as well but yeah like Russian learner kind of have a lot of out of the box things that you can directly use others you can just tune it if you want a lot more fine tuning for your repo you can use the others as well right. So we use learner in our mono repo so I'll just talk through it through a couple of slides. So for using learner you need to add a config file called learner json in the mono repose root folder right. So few things that you need to add are okay across what all package paths should I run this command should learn or run this command so you can give the list of packages where you want to run the learner commands. You also specify which npm client you use so it could be yarn it could be npm it could be npm right and you also usually set use workspaces to if you're using a yarn workspace right so and then it's as simple as doing learner run and you give one script to run so learner has a lot of optimizations where it looks at the package dependencies and tries to see if it can parallelize execution of these steps for example if there are three packages which are dependent on each other probably they it cannot run parallely whereas for other packages it can simultaneously run the same script right so that way it saves a lot of time of course it also occupies more memory but given for developer productivity and if you have like decent machines there's something that really helps us a lot right. So the other thing that it also does is of course like since it's one source for running commands you can optimize a lot in your developer workflow as well. So apart from running commands we also use learner also for a few other things one of the things is versioning so when you have to manage package versions it might become nightmare where you have like interdependent packages in the same monorepo say package B depends on package A and then you are changing package A but forgot to update the dependency of package B or like managing dependencies or versions of packages across all these right. So it gives a nice nifty way to handle that so you have a bunch of commands like learner publish you can use to create new release versions so you can customize how the release versions look like and so on then you can also do a learner diff and you can give it for a specific package also so it can give like if you do a learner diff on the root of monorepo it gives whatever has changed since the last released version right so what all packages have changed so you get a nice change log of sorts as well as like the differences in the code the diff so learner change gives the change log so using all this you can actually run like manage versioning a lot easier so because you might so we kind of have weekly releases to production so we publish a new package version usually every week but the smaller packages might be done through the week right so we kind of use this to manage versioning so it becomes a lot easier to manage all of that the other important part which I really like is so in our monorepo we have like different high value experiences like one of them was search which I work on then we also have people experience on the same repo you have like another feed processing feed app as well so we have like a bunch of and a lot more apps so for all of this developing when you're when you have to develop these and build these experience like push this experiences in multiple apps like outlook where there are like a big area of apps that I want to push these experiences to write so but as a developer I can't you know run all of those and test against those when I'm actually writing code so we actually use something like a test dev app we have like a test app where you just build these components test them properly and then release a version which is integrated into these applications right so learner has a nice way to scope commands to the dependencies of a certain package so I can do like when I'm say let me call this dev app for search as search dev app and I have a package for that so when I do learner runs scope search dev app it will figure out what are the dependencies for it and only runs learner and you can do learner watch across all of those dependent packages for the search dev app right so I need to just run one command using which I so it kind of when I run learner I want to watch so I so when I do learner and watch for and with a scope of search dev app here it kind of runs this npm npm run watch on all the dependence all the dependencies of search dev app like nc search form search dispatcher a bunch of search data layer search logger and so on right so you have all the packages that search dev app uses the watch is run on all of them and this something that is figured automatically by learner right so this way like adding new packages you can easily keep running the watch on the parent like the dev app package and you can still if you make any changes you can still develop a lot more efficiently right so these are some of the wins that learner brings in so it actually brought down our build times by about five times as well because there's a good amount of optimization in how it optimizes for running the process it has a good community adoption as well across like there are a lot of open source repos which use learner but yeah summing it up you want to ensure that you pick the right tools for your mono repo based on the complexity of the packages right so in our mono repo we have about 200 packages so we actually invest a lot more in tooling so that it it can make life easier for 400 plus developers but if it's a smaller scale maybe you don't need so much so pick the right tools for your repo for linting unit test functional proof testing you can use the tools that you have combine it with learner or other like other package management things for making this more useful if you're using type script then of course like handle a type script compilation also as an extra step so for the content that i used in the stock there are some of the links that i've helpful links which give you more information on these tools that i've talked about and yeah so that's pretty much it thanks a lot for sitting through my talk so any questions there is a question there uh you yeah you talked about no heist for stopping auto heisting in yarn packages in workspaces available only for the workspaces now or is it available for public as well sorry could you i couldn't hear the last part clearly right is it available public workspaces as well now yeah yeah it is available for workspaces so the no heist configuration is something that you are putting for yarn workspaces right so it's actually a workspace configuration that you can have uh any other questions okay if there are uh if you have more questions feel free to catch me i don't know just one second one question so you showed screenshot of people and is it only the monorepo for people then or you have like many other functional topics built into say 200 yeah so what so uh correct yeah so as i said like our monorepo consists experiences for uh people search uh like the search box and search results components uh as well as we also have for uh a bunch of other things like feed app we have a microsoft search in bing app as well so we have multiple other uh high value experiences which are used across uh like all of microsoft apps uh so it's not just one monorepo for people because even within these we use reuse a lot of components uh like uh like the data layer that we all of this talk to we kind of try to build like one platform uh api using which we can get all this uh information because you want to use a lot of intelligent signals across all these apps from one layer uh which can abstract you into other downstream platforms essentially what i wanted to understand was where do you draw the line as to what should not go in this monorepo but it belongs uh so as i said like for our i mean it's just for our monorepo right ours is one of the monorepos in microsoft so our monorepos we have uh like these high value experiences only uh but we also have a lot of uh monorepos for the applications itself like outlook has its own monorepo uh sharepoint has its own monorepo so uh again like depending on since these are like all these experiences are published as packages we don't have like a deployment into a server of sorts for these uh high value experiences but for app specific uh monorepos you need that deployment level optimization as well and so yeah uh any other questions uh you can catch me anytime during the day uh so thanks a lot uh and i think over to the next talk have a great conscience and uh i really want to thank all the organizers uh as well uh and the sponsors for this event thank you jai thank you