 specific, although this conference is held in Taiwan but not too much time in speaker. So today I'm going to give a talk about server infrastructure for Rails in 2016. This title looks a bit of boring to be honest. In 2016 you know every time I put an ear into my big title I feel like well okay so in next year that this slide is no longer useful. But since I have some chance to talk with some Ruby and Rails friends about their work and how they feel when they know Rails and many people complain about when they finish their Rails code and want to put it online and feel quite confused and really panic about putting their Rails code to real-world servers. So we have a conclusion that the running Rails is fun but running Rails is hard comparing to... I even have a conversation with my friends that I say hey come on you write a lot of Rails project but why you don't use Rails in work and say because my boss didn't know, my boss asked me about why we could just upload it and it will run automatically like just you know set up an Apache server plus PHP you know just put the PHP files via FTP and it will run automatically. But anyway so in this talk I want to give you some new concept about Rails server infrastructure and how you can improve it your server infrastructure and make your Rails runs faster. So let me introduce myself. I'm Richard Lee. I work for iCook. iCook is just yet another recipe sharing website in this world using Rails but we basically focus on a time in this market so maybe you cook one day and you will probably find our recipe online and I am developer so I write Rails code since 10 years ago and I also do some infrastructure stuff so that's why I'm giving this talk and yeah you can find me on Twitter and GitHub that's my username let's connect and if you have any feedback of this talk definitely please mention me online or send me email or whatever let me know. So this is our website we are recipe sharing platform and user upload a recipe using web using mobile native app and our major our main revenue comes from some collaboration between brand customer so you can see there is some brands on the right side of our homepage but most recipes are still social community driven so what an important job that in our company is to create all this kind of recipe and put it into different category maybe this is a hot summer not so hard actually but people may want to have some ice cream, mango ice, something like that and so we need to put it into the feature session and that users can assess it and see it when they enter our website. Okay so this is my company and we have some challenge just like every other company does running a business is really hard despite all this technical challenge our main business challenge is that because we have different kind of small apps we have the main website is a social recipe sharing platform but at the same time we have the e-commerce website we have the video website that because for cooking you can you can you can understand that if you only follow the text and image it's hard to see the idea how you could put for for instance you want to put an ingredient when should it be added to your pot or something so in video using video format is better to understand so we have a separate website that is called iCook TV and we have an e-commerce website called iCook market we have several this kind of small website and we try to separate it into several rails project instead of putting into a big model of select big apps so one challenge we have is that we have main developers that they develop a lot of small projects you know they want to put it online quick and fast and also development environment becomes more complex just you can see in the last talk there is MPN start you know you don't have MPN stuff all this kind of know that there is stuff sometimes I think is a bit of confusing for example how many of you have tried any you know MPN or Bower thing that for example if you want to install jQuery via Bower first or four you need to install Node.js and you use MPN to install Bower then use Bower to install jQuery that sounds quite stupid but anyway in real world you might probably need that because several package packages are only available via certain package management system so yeah so not not not mentioned put it online even for our development environment when I on board a new developer to our project I would say oh first of all you need to install Ruby specification and you need to install Node.js and set up your database and set up lots of things and some common libraries like image magic deep XML of course need to be installed to our production server so all this kind of thing make Rails a really complex program to run on your server so today's agenda first I will introduce how to containerize your app many of you might probably hear about some cool stuff Docker you know it's a Docker error I think it's containerize yeah we even have container submit in Taiwan so it's quite popular now and I will let you know how you can you know take your app to make it more container friendly and then I will share our experience while we are doing this process that you know setting up container itself is a hard thing but make it automatically and then make it automatic automatic and building every time you push or commit and it build the image is another thing and some kind of hard and have some we will share I will share my experience with it then I will show how we do centralized logging and monitoring because we have several small apps so it's not possible that we you know just logging into every server and check the log file that doesn't make sense at all and then finally I will show you some recent improvement for Rails and maybe in Ruby ecosystem talking about straight environment why you should adopt a straight environment make your server perform more faster so first thing is containerize your app so talking about container many of you might just you know it's like equivalent to Docker but not necessarily true I think all of the Ruby or Rails developer I met use Heroku before because using Heroku is quite simple we just add a git remote and git push and it will deploy and down you you have the Rails app running on Heroku's cloud and that actually also in containers so I would say if you are starting a new project for Rails and you might probably use Heroku for your first deployment and yeah you are already computerizing your app so not necessarily for Docker but also for Heroku and many other container technologies so is Docker stable yet this is a common question that when I chat with my friends and they say I hear that Docker is some problem or whatever but in my opinion we have switched to a fully Docker environment we run all our server we run all our apps on Docker for I think it's like six months six months and no too much trouble most trouble and most bugs are actually encountered and solved by others so thanks for their contribution so I could I could certainly I can certainly say that yes you can use Docker in production now is a good time to get started yeah many companies use it in production now not only I could but also other you know big world company and you can see that from AWS to Google cloud or even Heroku they provide certain support for Docker so I would say it's widely adopt and more stable technology right now so we can keep it trying and yeah they provide a really simple installation for Mac and Linux right now definitely for Linux it's easy because it's based on some Linux kernel piece to make container work but on Mac you might probably hear that as we set up Docker on Mac is so troublesome but they recently released a new project called Docker for Mac provide a simple package that you can install it run it us in regular Mac app on your laptop and it will immediately provide you a darker common line interface that you can run any image on it so it's super simple now no no no more virtual box no more bigger and all this kind of stuff no more so that's a good thing and yeah as I say a major pause and I spend a lot of time to support if you use Heroku I will recommend you to search something about Heroku's Docker support and you can actually to rebuild like like Heroku like environment on your laptop and put it online to production and of course tons of tutorials online but I need to say that when you when you see any tutorial online do look at this Docker version because Docker changes a lot in recent years so like Docker 1.8 is a major updates and bringing many new features so if you read actually when I was writing these slides I also found that in the next chapter I will talk about our experience with building Docker image on continuous integration server but those problems are already solved so do check the version when you read any tutorials online so I'm not sure how many of you have experience with Docker so I still try to introduce Docker in you know one-on-one way to keep it simple so actually Docker is just like building you know we usually say that writing Rails code to build web address quite simple because you have a lot of a chance available online if you want user authentication you can choose between device choose between all logic many good libraries and Docker share Docker has a really strong community that they share many pre-built images it's called base image for example Docker official provides Ruby image that has it's a I think it's a Ubuntu image with Ruby building and some common libraries and you can actually find some really good base image made by the fusion or the creator of passenger server they release a series of good base image that you can use and so you can choose from many pre-built base image then after you choose a pre-built base image then you add your file and run some Linus comments on it like you want to get install in match magic you want to install Node.js you want to install whatever you like and you can add your Rails project into it and then do bundle install do something else then finally just run it there again Docker run and you just get up and running so it's more much more like compared to the old techniques to build a virtual machine image it's sure many similarity between building a virtual I mean virtual like previously you might probably build images for AWS, build image for virtual box those kind of virtualization thing and this container is a bit of different for virtualization usually you build an image for whole system but for container usually build for one process say for example that this container the only the only single important problem of this container is to run my Rails app so I choose from a good base image then I add my Rails project and specify what you need to be to be done say for example if you're bundling store before you can run your Rails server and specify those command in your Docker file and run it that's it so let me show you a simple conflict I'm not quite sure if it's clear enough if you couldn't see it please raise your hand and so in the first line I say from Ruby 2.3.1 that means that I use this base image which has Ruby 2.3 pre-installed on the Docker image then and I do some command like I get update and run build essential run libpg run load.js add something I need and then I create the directory and then I add my train file to bundling store after bundling store I add the rest of my project file into it then finally run it around bundle Rails server so that's it that's a simple minimal Docker setup you can actually use it our our real-world case looks quite similar to this one we added something more but general speaking just choose the pre-build image then add your project file and install something that you need and run bundle recommend now oh I forgot it you need to maybe do SS3 compile okay then finally specify what it specify the command that you need to run so finally this Docker image it's done so lots of tutorial online so I will skip that for some details but just give you a rough concept and how it looks like but thanks oh always when you read something about some tutorials online everything seems so simple and that means there is a real problem behind so if you read I'm not quite sure how many of you have to read this this 12 factor apps which proposed by hero crew school founder and say stating that how a good web app in modern era should be and they have some certain principle for example that you need to use environments configuration that you means that you're you don't write all this configuration into your file system but instead you can treat config using environment variable or there are several principle that they recommend to preview a good web app so in container era let this same principle still apply for example a good container she looks like first self-contained for example if you have assets definitely of course most rails app has SS pipeline and you have some JavaScript you have some stylesheet and instead of uploading to somewhere else like you can probably using regions like SS sync to upload all this style sheet pre-compiled to S3 or CDN you should actually put it into the container because that when you have some trouble with your app you can just switch and rolling back using the legacy version of your container and put it online you don't have you know two different versions online you have so try to put all the assets everything into the single image then second you need to run one process per container say for example in common rails app you might probably have background worker using exit job using sidekick or using other background processor then one way to do that is that you can run your Docker container with two processes in it one is real server one is sidekick or instead I would suggest that you to run the send image but it's two container send image two instance and one is for rail server one is for your sidekick process okay so what one process per container then also store config in environments just like you might probably have some experience with hero crew there is a simple command code EMV files that you can set environment variables when you are run any Docker container and also if you want to do some admin process on hero crew who might probably use hero crew run console to open up a rails console hero crew server and in Docker there is no I mean you can write some show script but basically it's still like Docker run and do something and we use simple show script called ECS run show ECS means elastic container service it's a service provided by AWS because we put all our Docker images on ECS but anyway you can run it using in a separate admin process and fortunately enough that hero crew folks make many contribution to rails file so rails file is actually container ready they enables they add a several environment variable to tweak default behavior for example that after rails 4.1 you no longer need a database.yaml you can actually put your database configuration in an environment variable called database URL and I highly recommend it and also there is an environment variable called rails serve static file just as I say before a good container should be self content so you you might probably also want to serve a static file from right from your rails web process and in rails file you can just simply add this environment variable to set it to true and it will your rails project will start serve static files and also another good and another good thing in rails file is there is another environment variable called rails log to standard output because in Docker or in hero crew if you want to collect log from your container the only I won't say the only but the recommended way is you just write your log into standard output and your container scheduler system will capture all this dog and put it somewhere else I will cover it later in centralized login part so rails file is container ready so too long don't worry if you haven't had a chance to use Docker in production if you don't really plan to use it in the near future but you still I still recommend you to make your app container ready if you have a config file at this moment you can probably want to switch it switch it to use environment variable okay so then so okay I talked a lot of a good thing about Docker and show you how you can use Docker to make a 12-factor app that the real-world issue is how I could build image in my CI or a CD flow means that wherever I push a commit someone should help me to build the Docker image and actually building Docker image takes some time takes so much time so our first try and first try because we used circle CI so in circle CI and we would just simply add another comment to do Docker image building but unfortunately before we add Docker image building to circle CI testing it's only take less than five minutes to finish all our aspect tasks but after that it's a 15 mini and even more to build the Docker image and you know that's not horrible in our case we want we want we want our CI text wrong within 10 minutes so what's what's happened because Docker has a lot of cash mechanics and it should be fast if you try it on your laptop the first view might probably take much longer time but the second view or third view and if you don't change too much file they will just use the cash file and it will be super super fast so what's happening so why docker cache fails on CI service public CI service because that for Docker image cache work they actually takes file metadata into account to to to compute a hash to compute digest to see if file change or modified so it takes certain metadata and one significant one is MTI which means the file when the file was modified it and it doesn't track MTI so it's actually new to me that when you get called a new project you look after its daytime of those file you will see that they get just doesn't ignore all these days or time or when these files modify all this attribute so when you commit and get a file to get in wouldn't track this kind of attribute but however this is fixed in Docker 1.8 that in Docker 1.8 to compute a file digest to compute a hash in no longer take MTI into account so there's no longer a problem but however that for certain package management system especially one in JavaScript world they sometimes when you do MPN install twice or three times or whatever every time you do MPN installed in it changes some files a bit you do boring store in change the file a bit so it will break the Docker's cache also package management system doesn't work really well in this kind of cache environment and sprocket sprocket also has some cache and if you use certain Ruby genes and it will generate just like just like those package management system does that it will randomly generate some cache file or whatever that will easily to break your Docker cache so how we survive so bring bring your own CI server actually personally I if there is a pay service a hosted solution I will usually usually choose a hosted solution like circle CI or Travis CI but however in this case because in circle CI or in other public CI service between view to view your file I actually pull and download every time when a build starts so which means that the file system is not persistent so but if you roll out your own CI server definitely you will share the same file system between view to view so there's no too much cache issues I think all the files are persistent in your server so don't story short we use junkie plus jinkies junkie is a project open source by GitHub it's just like it's like an API wrapper for jinkies for those who have tried jinkies before you will probably found it really hard to set up and to tweak project setting and jinkies yeah to solve this problem so github folks they created this project called junkie and junkie provide chatroom interface that to install you install a script in your Hubard and you can say Hubard CI build something Hubard CI do something Hubard CI anyway so and more importantly for per project setting that you can add just like using circle CI or Travis CI the configuration of the project building is put into repository so usually I will put a script file in my Rails project and specify how what steps need to be take taken to build Docker image and they provide a good user interface like this you can see how your Docker image build process looks like so yeah so for this part I would say building in Docker image on your laptop is easy but building it on CI service might be hard so do care do take care of those crash issues otherwise you will spend lots of time building oh by the way you might probably hear that Docker official provide a service called Docker Hub and Docker Hub there is automatically building solution but we definitely try that before and that doesn't work they have no cash at all so when I take too much time I've been building an image especially for Rails like if you build assets without any any cash it takes usually takes three to five minutes to build your assets so that's well so through all your own CI service is highly recommended the next off I will talk about centralized logging and monitoring actually more about logging so since we are in a container era so probably I will have multiple server around different kind of apps so I need to have a way to pipe all this log string into the same single pace and so I'm going to talk about Flindy Flindy is Ruby project and we have some native extension for performance and so for Ruby for Ruby guys just like you and me I probably be quite comfortable with Flindy's config you can even write your own plugins I create one for my for certain use case so I can say that it only takes me one day or two days to to survey and try to write my own plugin is quite easy so what it actually does is that before Flindy you might probably see different kind of logs from different era you might probably have web assets logs from maybe you have some operation logs from AWS from hero crew you use different part of service and you want to put it into several place they want to put it into Hadoop you want to put into S3 and it will look like a mask and using Flindy it's just simple as UNIX pipe I really like UNIX file pipe that you can pipe one comment or another comment and this philosophy just look at Flindy and so in Flindy you have a lot of input plugins that collect logs from several different place and then in Flindy itself you can do some filtering do some modification maybe you want to do some passing stuff or filtering out certain logs maybe it's too sensitive or something and then put it into several different output but you might probably want to store data in your MongoDB cluster store data into Amazon okay so it's just make everything looks much better huh so and more importantly that Flindy works so well with Docker that in Docker 1.8 there is a building there's a building logging driver for Flindy which means that as I mentioned before that a good container should put to show output its logs via standard output and using Docker 1.8 it could actually collect your log from your container then put it into your Flindy cluster and do all this pipe piping that put in and downstream your logs to somewhere you like so how we make our log from Rails to Flindy and first of all I will recommend you to use gems like a log rage or log stash which make your log beautifully format into a single line JSON format if you haven't hear of it you haven't tried it I will suggest you to check it out what it does is actually usually in the default Rails production log a single request will generate like five or six lines of logs and one talking about its HTTP request what is passed and what which controller and what views was rendered okay so but in using log rage or log stash this kind of solution could make you your log in combined into a single line so every one request has a one line of log then you output it to standard output using Rails 5 you just tweak any variable to put it to to make it right to standard output then after that when you run a Docker command you can use an option called the log driver equals Flindy then you probably want to tweak some Flindy setting like for example you want to take tag in Flindy is like name your string name your log okay so how did you name this log in Flindy so you add some flint tag and run the image and it will automatically collect your log from standard output of your Rails project to Flindy cluster so that's it that's quite simple to collect logs from Rails to Flindy and in Flindy and you can actually find many configuration files online they have good books for for how to put your log data from from Rails to Elasticsearch or from other pays and I will share our experience the recommended approach is to first of all to pipe all your logs into Flindy if using Docker that will be quite simple because just make your Docker image to write everything into standard output and do the configuration that make your container to output to standard output and after that you collect your logs into Flindy via foot in the driver then second you might probably add some medieval valuable outputs for those who first time do this kind of thing just like me in maybe one years before the the most common problem is that which kind of output should I use should I put it into S3 should I put it into whatever somewhere else but I would say the most simple one is you choose three three kinds of output first you want to do archiving because you want to store all your log in a place that you can you know maybe one years out of the one year like few months later you want to go back and find out what happening at that time so first you might probably store it into S3 then you want to have a query interface which is easier for developers or for your operation people that could find some logs from that then you might probably use Elasticsearch plus Kibana or that's a common and popular solution and for data analytics you for in our case we want to know what users search because search search recipe is a quite common use case in our company in our website many people they search what they want to cook everyday so I want to have a place to do data analytics and you might probably use AWS Redshift or we use treasuredata treasuredata is the company who actually created Flindy and all this open source project and they provide really good hosted Hadoop service with good Presto and all this query interface and pretty nice so this is how our log system looks like and we use Flindy from copy data from Docker and pipe via different footing Flindy server and finally piping to treasuredata's query interface and there's a several rows there are actually access logs from our Rails application so that's how you can build your simple centralized working system so the last thing I want to talk about is server performance tuning this is a huge topic so I wouldn't talk too much because every app has different use case and scenario but the general rules of some of in 2016 is that you use a threaded server it's real safe it's real straight safe yet I would say yes because it thinks rails 2, rails 3, rails 4 every time a major rails upgrade someone will probably ask okay so it's real straight safe yet and I would say yes actually we have using a threaded server for four years and use background process workers like psychic or other threaded solution for your rails app for years and everything runs smoothly not too much trouble for trace but so you can actually see some major performance boost using threaded server for example like psychic versus delayed job or rescue you might probably hear about the job and rescue but they use a single one process model which only handle one task per process but using psychic you can handle 25 or even more jobs in a single process that's much more efficient and for web server there are several solution one is Puma one another is passenger passenger in recent release they have good straight straight more support and if you use unicorn you might probably want to use Rimbos Rimbos is like a fork of unicorn that adds sound straight into your unicorn web server so there's a good article called changing Ruby analyzing Ruby Gen states for 2015 and you can see for app server that Puma actually more and more users are using Puma or think both of them are more concurrent can handle more requests at the same time and for background workers and the psychic just fly right up so I would say a threaded environment is quite stable in nowadays and also for hero crew previously maybe in the first few years of hero crew they recommend you to use unicorn but in last year they start to recommend user to use Puma and not to mention in real-time the default web server becomes Puma and drops web brick and use Puma so for the reason I would say for instance if you stop you create a new real-time project and you run real server by default you use Puma and they also provide default Puma to be config config okay it's a table so I provide a default Puma configuration file and by default it has you use a straight mode so my default you have five straight in the same process so for those who is inquire understand what's the difference between straight and process and general idea is that using multiple strings means that they share the same memory and using several process means they don't share the same memory so for certain cases that you might you might not want your different worker different web request handling using the same data otherwise there might be data corruption or something but if your program is designed properly you try to avoid this kind of you know share memory usage and that will be fine so the problem is actually not rails if rails if rails is straight safe yeah but the problem is that does your app straight safe yet so common trading issue when we migrating our service from unicorn from rescue to Puma to psychic or first of all you try to avoid using class variable or global variable because all these variables say it's global so it will be assets from different straight and if you change your value and do something with it and your app server might probably explode then another common issue is that many gens are not actually not straight safe I just named a few but usually can find a what could run for that for example in our match many people use our magic for image manipulation but instead you can use mini magic and AWS SDK provides a configuration that you can eager load all the class to avoid low straight problems alright so that's summary what you need what I want to tell you is certain some trends for real server infrastructure in this year so first is that you should try to containerize your apps no matter you are planning to use Docker or not if you use hero crew you already run your rails app in a container environment second be careful about CI cache issues know this is more specific to circle CI and my use case but generally speaking if you want to use any container do care to take care of CI cache issues then third you need to build your minimal viable centralized locking solution if you use others if no matter you use Flynn D or use other tools this is pretty important and you should build it up you should build up your own centralized locking system from day one this is really important thing then adopt straight okay so there's many terrible story how they when they migrate to straight server and things broke and explode but that's not not that terrible okay it's already quite stable yet so just use it so finally thank you and oh we are hiring so if you are finding a job just come and chat with me okay