 Thanks. This one is working. Yeah. Hello. Yeah. So everybody able to hear me? Okay. So yeah, today's talk is, so Mike is working, everything is fine. People are settled. It's a good start, right? So today's talk is about having a good start. We all love having a good start. A bit about myself. I'm a Python developer. When I get stuck on dependency issues, like I'm not able to just do my work, that is the interesting stuff, developing things. I get stuck involving like installation issues. So this is how to get rid of those issues. So we will need five different components, which we will be using to build our dev ENV, which are Docker, Vagrant, virtual box, shell script, and some good luck. I will talk about what is good luck here. And this talk is going to be about more on the wise and less about the house because we are all smart. Everything is on the internet. You can get most of the info like how Docker is, Docker works and Linux countries, et cetera. Okay. So you are running your company, you are managing a team, or you are a developer doing something. It starts from the developer's disk, right? So developer time is very costly for the enterprise. You can easily relate to this fact that hourly cost of developers is equivalent to the monthly cost of servers, right? You hire a developer for $20 an hour or $50 an hour and you get AWS EC2 instance. You are going to... It's going to cost you about $50 for a month. So your developers are costly. Less productive developers increase time to the market. Obviously. So for some reason, your developers are very productive. They have a lot of issues installing stuff, dependency issues, not able to get things running for a new developer in the team. Yeah, in the new team. And all these reasons, all these cases, are potential bad... for bad production. You plan to launch your product in February 2015 and it all deals up to, say, December. And you lost to your competitors. So productivity is very important for your team. I don't know if you are able to make much of this slide at the back. I will just make out the components. So how does the developer workflow look like? So we have a developer. He does his local test. This cycle is like... This thing is very frequent. And then it goes for... And then to the production. So developer to production, this is the least frequently executed thing. And developers' local test, this scenario, is the highest frequency. And one of the stats says 95% of developer time is spent in the development environment, which is very true. So we need to optimize here. This is our interest. We are going to optimize this thing, which is the development environment. So how should a development environment look like? Ideally, it should be production-like. Sure, for sure. Because if you are very... My stuff is running on my laptop, my friend's laptop, but it's not running under production. Where the end users are going to face the app, it's of no use. It should be very quick to set up. Obviously, it takes like four... For my Devi and me to set up my app so that I can start working, or the new developer starts working on it. It's not useful, right? Reputable and upgradable. Quick to set up is one thing, but like other team members are going to push their code each day, other day, and we need to have a... to make the setup repeatable and upgradable as fast as possible, as easy as possible, and simple and easy to use. If you ask me, you can use Chef, but for that, you will need to know a new programming language. I don't like that. It should be something like, by default, everybody should be knowing it. It should be very easy, the learning curve should be very low. They shouldn't require to learn a new language or a new DSL or things like that. So simple and easy. If there is a way, you write a line. And a very complex app with a lot of components, multi-tier application architecture magic. So that's cool. That's a feasible solution. That's the wanted thing here, right? So say a modern architecture, what it looks like. It has, say, one company, EngineX, you have your Flask application, and with Cassandra, Datastore, some XYG server, maybe T-server, or whatever reason, I don't know. RabbitMQ, Elastic... All these cool stuff, right? And then you want to just recreate these things. You have RabbitMQ cluster, but at the least, you want to recreate these things, these components in your development environment. One way is to have one VM, like one virtual box VM running for each one of these components. This is one way. But if you have six components and you are planning to run six VMs on your laptop, I don't know, it's a very good thing, very good thought. Just today, Michael Ford talked about the dependency hell. You have two applications. One is Flask, and it depends on some version 1.0, and other Flask application. For some reason, you are using one more application, and that is using 1.2. And you need to have these applications in the same system. So this is the problem. So the solution we are going to talk about is all of these issues. So yeah, here's the talker. So what is Docker? Docker, most of you, if you are not much into systems level, Linux countries, and those kind of stuff, very easily you can think of it like a lightweight virtual machine. That's it. You can take it like this, and you can work on it. I don't say that don't dig on it like details on the address, but you can use it by just knowing a very light virtual machine. Other than that, the main big win using Docker is it's pretty fast. I mean, starting a VM takes you two minutes to five minutes, maybe. Starting a Docker container takes you milliseconds, or in seconds, two seconds. So it's very fast. So yeah, instead of running VMs, we will be running containers. So how are we going to do that? Okay, some introduction about Docker. So Docker has this kind of images and containers. If you are familiar with AWS, so they have this concept of AWS, AMIs, right? AMIs are the images, run the instance, the running instance, EC2 instance, that gives you a running operating system. So in a similar way, Docker has Docker image, which is the OS image, and when we run it, they call it Docker container, a running container, right? Yeah. About Wegrant, the five components I talked about, what is Wegrant? So Wegrant is a provisioning tool. You can use Wegrant to provision an EC2 instance, remote server EC2 instance. You can use Wegrant to provision a local box, virtual box. You can use Wegrant. With Ansible or Chef, or these kinds of other tools to do a lot of interesting stuff. But we are using Wegrant with a bunch of shell scripts to build our Debian V. So the problem I'm trying to solve here is I'm a developer and it's my preference. I love writing my code on a Mac. And my friend, maybe he loves writing his code on a Windows machine for some reason, and another one he loves writing his code on Open2. And we all join the same company and if we would have a way that this thing could be feasible, I don't know. So is it feasible? Sure thing. We are going to implement that. What we are going to do is this is my laptop and I'm sharing with the machine running inside the virtual box thing. And the virtual machine will be hosting different components of the multi-tire application stack, different components. Each of these components is a running Docker container. In the next XYZ server, these components don't need the application code. But yeah, the Flask app, this needs the application code. So if this Flask application is able to view the code which is residing here on the host operating system, the laptop, not the VM, some day, I need this thing visible to this Flask app. We think the feedback loop which what I'm talking about is a developer needs to have a fast feedback loop. If I change something in my template, like I'm just writing an HTML template, and I want to see how it goes, so I must have a way to see it immediately. I don't want to push to GitHub and then see things on a test server, things like that. So I change my code here on my laptop in a platform-agnostic way. May I Mac? Open to anything. And the Flask app here sees it, it changes the application right away and I can see the app running changes on my laptop. How is it going to be possible? So we are going to use two key one is shared volumes and another is put forward. So... Okay. Could you help me please? I want to show the editor. Show the screen. I'll be doing multiple screens. You need any other ones? I have open to. Okay, leave it. Oh, I got it. How it's coming? Okay, on the other screen, right? On the right screen, right? Now you see that? Yeah. But you are not seeing it? Yeah. So I can do that. You just... Yeah. Got it. Yeah. It's like your virtual screen is here. Okay. So in our Docker, sorry, Wegrind file, we are specifying two things. One is the port forwarding, which is saying, let me get the app which is running on the virtual machine at port 80 and make it visible on my laptop, say Mac or Windows on port 8080. Okay. So anything running on your virtual machine under the virtual box at port 80 would become visible on port 8080. And the Wegrind file has the concept of like provisioning script. So we are providing, okay, this is the provision.ssh script and that basically Wegrind app start our dev envy. It runs that particular script at the startup. So it's something kind of the user data concept which AWS is having. Okay. So here I go with my development setup. I just wrote a one-liner and it should start my full app running with all the components like I am running elastic search and what not. Okay. Simple search application. Okay. It started. So you can see the app running here. Yeah. This is my Mac. The things the application is running under Linux. Yeah. And now I will show you I change the code on my reader and it should be right over visible. The change should be right over visible. On my I am the application where I am running. So I just change this template instead of my application. I write our application. Okay. So as you can see the title there is my application and it changes to our I mean the change is right over visible. So good thing about this is applications applications can become very complex and when you have your when you have in your team a developer who is a front-end engineer and he doesn't like to think about you know back-end systems it may be very interesting to some of people like back-end some people like designing things. They are CSS developers or HTML5 developers and they want to focus their skills on the front-end. So it's not right for us to just bog them down with back-end details. So however complex your application is the developers come the front-end engineer comes up he does the CSS changes without worrying about how this application is getting deployed or how complex is the architecture there. He can simply work straight away. So this increases the productivity for a lot of people who don't want to get into much details of every component. I remember like three years back three years back I joined one company and it got them like it took them four to five days to just set up a devian be like a full application running for me where I could start my work. So this is a problem many of I mean it's very kind of everywhere I see this. So the devian be should be very fast to set up and it shouldn't be getting into our way just like I mean the problem with the front-end is everywhere is okay back to the presentation. So what is minimal impedance mismatch which is the thing is like we want a high fidelity you know resolution to to the production on our devian be as say if your production in principle we could have just done our best one flash cap the flash cap to our developers they work on it and assuming that everything would run fine when they deploy it to production with all the you know moving parts like a Sandra and what not but yeah that way it doesn't work like things could go and a lot of possibilities there so it's actually imperative that we want all of the components as close as possible in our devian be so that is I mean minimizing the impedance mismatch between the production and your development environment so how is this how are we going to achieve so docker what is dockerfile exactly so docker has this concept of dockerfile docker in itself dockerfile is not very useful I have like I have been using docker for a year now version 0.7 and I did a lot of like devian be a range of applications from internet of computational genomics dockerfile in itself is just a dsl you know which has some directives it is just like a shell script so if dockerfile was that awesome then why shell is not so actually dockerfile is not it has just some directives to let out the docker devian or the docker software work the problem with dockerfile is you run your dockerfile to build the images it may work today and it may not work tomorrow and just today I had this problem I was in odd one and trying to build using my dockerfile the net was not running some issues I had to go and then get back my image the image the image is the immutable thing so it is not getting destroyed or it is not getting corrupted problem with dockerfile is shell script going to depend on a lot of third parties exactly like a shell script so how does a dockerfile look basically yeah it is okay I got it enable mirroring yeah it is doing a lot of problem to me just today yeah thanks so this is how a dockerfile so we have this line it is building on a base image which is like from dockerfile java whatever this is a base Ubuntu image which has already java installed so right here like it depends on somebody else image else code right so you cannot depend on that and you know this dockerfile is working today and one month from now you try again building it and the person who made this image some reason he changed everything right and you have you have written the same image name and things don't work so you cannot rely on dockerfiles the only thing which you can rely is the docker images so what that means is you build your image you have this image you use the same image for your development and for your production no change at all and that image is basically your full environment runtime environment including the OS including libraries and including including the app code and everything everything basically that's in your docker image and next time you want to build one first build one have a successful build test it try it everything and if it's working properly then replace then you can replace your old image this is the only thing that's going to work so you cannot rely on dockerfile for sure ok and how are we going to like for the devian v how are we going to solve the problem of having a snapshot of a database inside our devian v so what other projects I worked on was auto.com and they had this like 60 to 80 GB of data and we didn't want to pull all of it into our devian v we wanted some lighter version of it I mean a subset of it so one way was to just randomly select you know some sample points and have it and get it every 24 hours so that we get the latest snapshot but what happened is that some of the app app scenarios which depended on some specific data points was not working we were not able to test it so that is where the selection strategy comes you need to select a good strategy so that every component of the app which you are going to work on has the data you can see all the features working it shouldn't be something like okay like you have a real state app for some reason they have done their section for daily Bangalore and Chennai and your data the data snapshot you took it was just for Delhi and Bangalore but not for Chennai and you never tested it because the data was not there and there were issues with the Chennai thing so you won't be able to catch it so it's very important when you are building the snapshot for your have a good strategy that spans to all the scenarios which the app features needs about host names so of the production we don't usually work with ips we work with host names so I have a MongoDB cluster so I will be using it as MongoDB dot you know immuno.com or something like that with a URL and on the dev envy when we are talking about minimizing the impedance mismatch between production and develop environment we want to have similar thing here on the dev envy itself so how are we going to do that yeah so docker has in fact when you release the docker they have many features on editing the ips host file dynamically but the thing is yeah this is the solution so on the local there basically it overrides the DNS and when go to auto.com on your local system it connects you to the local container not the real application so when it's in the dev envy it works exactly like when it is working in the production okay all of the source code is here on my github account and we have seen a pycon 2014 here thank you and I'm ready for qa docker is built upon elixies right docker didn't invent elixies so docker is just giving developers a good user interface for working with elixies right at the end of the day when you're using a docker right yeah it's process isolation more than a virtualization technique and when you're dealing with a complex web app set of web apps and you're trying to script in databases and stuff like that then how do you deploy code synchronously across these dockers in a fashion and also I mean is there any particular reason you're trying to build your own docker file for stuff like elastic search that comes out of the box with its own docker images yeah so yeah I didn't build that docker file I put it here just an example how the docker file looks I am using the exact from the you know of the chef docker file which is available from the docker hub docker index but when you have a very I mean when you are building a custom application you are not like you have elastic search config file a custom file right so for that you need to change I mean you can inherit from the base docker I mean elastic search of the shelf but you will need to change some of the stuff so about the deployment is like okay I missed one of the concepts is the registry right so registry so like some people have this preference they don't want to share their code I mean the companies don't want to share their code with other cloud providers maybe docker hub or something I know right you're talking about a registry where I can push essentially I would say docker is get for deployment now if I am using a certain docker philosophy at least what the docker deployment philosophy dictates or what has emerged today as the way to do large scale deployments is that you use environment variables your docker files pick up environment variables from the host and those can be passed along into the into the docker containerization or LMCT file whatever you use then why would you want to expose disk data binds and then modify them through another bind over there I mean it just causes sometimes you have AUFS lockups on docker like docker does not give you the full flexibility of butter FS that Alexi would give you but they are I think they are having I mean you have the option to use different drivers right I mean that's the default one right you can use many other drivers with docker because of the isolation it gives at the OS file system level but I don't know I mean they are not drivers I am not able to understand your question actually what is your question exactly why are you using dockers in places where you would rather use an LXC use environment variables to modify your docker your docker files write on that we don't use config files for managing docker but config files are like for example so you are saying like I have to change the config for elastic search I put it in an environment variable right yeah you can do that share it and ask it to pick up the config file if you are using something which doesn't yeah I think you are right there that's how it looks like a write approach how can we configure different subnets in using variant is that something we can do I have not tried doing a lot of stuff and subnets on those stuff using variant so I don't know that much ok thank you Anubhav we thoroughly enjoyed your presentation