 So this is one that I talked about in the previous JSPurlpen, so if you have heard it I think you'll be pretty familiar to you. So who am I? I'm a co-founded actress with a small certainty. Mostly I do Ruby and Node.js. We wrote this thing called Node Coolbox, which is basically a good way to find the package managers. Me and my colleagues support a bunch of open source projects. I love Node SQL, I think I'll talk about it in the later slides, but mostly because I don't understand SQL. Yeah, that's pretty much what I am. Some of these patterns are what I learned from largest Node project. We do some work with big Node applications as well as small ones like Node Coolbox. So yeah, just before I lose context on this, I lose context very quickly. So just stop me and ask me the questions and I'll hopefully answer it if it's really hard. Like I think Ruby will ask and I'll probably pass it. So really quickly, one of the things I wanted to check was how many of you guys do Node like that? Like a thing that you do every day. How many of you do JavaScript? That's pretty much, pretty good. So very little server-side, how many server-side would I want? JavaScript, Ruby, whatever. So I think most people understand like the website of JavaScript. I don't understand that stuff, but I do a lot of server-side stuff. So for me, when the question is around why JavaScript on the server-side, the answer is really obvious. All the apps that I have on, everything that you do is IO. Like whether you're hitting the database, whether you're actually doing file systems, you like big configs and stuff, whether you use DNS. Does anybody know Express? When you do Require Express, it's actually a talking. It's probably one of those weird things in Node, which is actually blocking. There is actually a Require Passing, which works as well, which is a thing. So as a general thumb rule, I mean, I've seen this in a lot of Ruby projects. Some Java projects, basically your CPU is doing 10, 15% most times, but it's actually blocked on IO. And you can't really do it anymore. Like you can't make more requests per second because even though it's showing 10%, there's a lot of stuff it's doing in the background. So in a sense, like a typical web app, when I say typical because there are lots of different ways to write web apps, one of the big ones I work on has 10, 15% throughput on the CPU. And we have to add more passengers or more boxes to maybe support it because we just can't scale that one anymore. So in a sense, you're basically seeing 90% of the capacity of your web app being wasted because of IO. That is a big question. Why won't threads scale? Well, good question. Threads do scale, but there are lots of issues with threading itself. You have logs to access shared data and things like that. Most of the frameworks will abstract that for you. You want to know your response on the thread. But you still have to understand what's being shared across. For example, Ruby has a really bad... One of the things I'll keep comparing to essentially Ruby because Ruby is where I come from. I do a lot more work in Ruby than JavaScript. Ruby has a really bad implementation of threading. So it has a very basic issue with Ruby as in the global Internet log. Maybe I'm going a little far, but... So it's not like if you actually create five threads, you'll get five threads. It's never been the case. I mean, I don't know. There may be a language problem, but I don't think any other language or any other framework has sorted out. So Reactor is a pattern. Reactor is one way of solving this problem. If you want real scaling, you go to Google, so I don't deny that. So yeah, that basically starts. It's almost using 10% of your train power. If you want to be doing high-performance web apps, if you're okay with wasting 90% of your machines for just doing some IEO, then it's fine. But if you want to do these performance apps, when I say performance, it doesn't mean IEO intensive, right? Like, then, block the problem. But it's more web apps sort of thing. I think Reactor and non-locking IEO is basically the only, almost the only way to work. So this is one of the feeds that... Do people recognize IEO? He's one of the big node guys. He maintains MPM, which is like RubyJams for Node. So basically he says, LinkedIn basically went from 15 or whatever. You can read this. Basically, you get an 8x throughput by just switching to Node. There's a lot of part around this thing, but it may not be reproducible on your machines or your applications. But basically, you could get that. That sort of says you, in the same range of like 10%, 10%, 12% CPUs, 5%. I know that LinkedIn was using Ruby, so they switched from Rails to... Sort of thing. So let me see what the landscape looks like at least on the reactor side of things. So Ruby has even between... Python has twisted JavaScript as well. How many of you are Python guys? Okay, so you're more familiar with Twisted. I think Twisted probably is the inspiration for even machine. At least even machine. I guess so, let's extend Node. But I know even machine and even machine. Yeah, I understand even machine. Has anybody done even machine code? Not a lot. So the problem with even machine, and this is another philosophical debate is sort of why not even machine and why Node.js? Do I have to switch right into this shitty JavaScript all the time? So problem with Ruby is not written in ground up to non-blocking. So there's a ton of chance that you use by default. Like my favorite is West Client, and I just can't use it in a reactor loop. Like in any even machine. And you have to actually think through your whole application before you say, oh yeah, I'm going to write a reactor loop into it. So not all channels are blocking, non-blocking. So you can't obviously leverage your reactor loop if you have blocking bit. Basically if you have like everything else that runs in 0.1 second and just one bit that calls out to some server on the Internet takes two minutes, two seconds, it's going to take 2.1 seconds for each state. Like each loop will be 2.1 seconds. Which basically you're not doing much in that time cycle. So to me, I think it's a Node's obvious choice. I think a lot of people will disagree with some of the observations I made and feel free to reject them and I won't mind it at all. There's a big problem I need to fix to Node though. This is like a classic car. As you can see this is XKT. Basically it compares learning curves for like bunch of editors. Like Emacs is like the thing on the right which has a very weird learning curve. So I said let me try creating one for like the application. So Ruby is like, it's very general and there are like two or three plateaus you get and then you are a Ruby or a brains guru. C++ is always like uphill battle. Java is easy. It's really a few things. You think you understand everything and then you like how to stop talent and you write this code and half the stuff doesn't work to start deep again and you basically lost it. Maybe I'm at the cycle where it sort of peaks right at the end but I think a lot of people get to that when you if you're doing a Java script. It's really, really complicated. I mean I agree. So let me take a deeper sort of talk about this thing called Node. I think if you haven't participated I think it's really cool. Take part in this yearly event. Basically you have 48 hours to do something to build an application and yeah. And you're rated against like hundreds of, I think when we participated it was 300 teams worldwide and you're rated against like some of the best people around. So to be participated in that we had a decent application called active node because basically what it does is tracks your application performance almost like a new LX world graph application. This was our second node application so we did a lot of things wrong in this. I mean we just basically messed the whole code but yeah basically what it does is real-time monitoring gives you all the usage pattern what times things are taking. So it would have been nice to show what active node does right now but it still under wraps. Hopefully the next time I'll be more proactive with that. Most of the code is from active node let me jump into the real stuff now like it's just all cool around. So how many people agree with this statement? Basically it says in JavaScript I just go crazy if it works I just check it in. Don't be shy I mean it's okay. I think this is a classic JavaScript thing I don't think anybody can say this in any other language it's really not intuitive I mean there's so many things I mean we were talking with this thing called WTRTJS so many weirdnesses with JavaScript that it's not fast. The other thing that not JavaScript but I think eventful programming needs to like spaghetti like lots of callbacks callbacks callbacks and it just leads to really bad code. This guy is like a colleague of mine we worked together at some point. Okay so it's not intuitive many things are summarized what problems node has for example it's JavaScript it's weird like it has its own this thing it's really hard to master this thing is also really hard I think a lot of people have this problem especially I came from C++ Ruby background and I had to unlearn so many things to get to get to the patterns around the same programming I think it's a lot of unlearning and maybe it has a different mindset as well to start thinking about node npm module issues has anybody had node module issues or npm issues? So node really moves fast I mean like in the two weeks we've had like two or three years at least maybe not in a month at least we have three reasons so there are bugs in node itself npm is like it's very easy to start with but there are so many things that you don't know about npm that yeah it's funny modules like in the 48 hours of our active node development we had five full requests five different node modules it's like really buggy at this point at least it's really buggy deployment is really really hard to get it right where do you guys supply I have some people develop node applications where do you guys supply so you've custom built your own that's interesting we should talk Hello Peezy I was a node seller but now we move to self posting the JS website is running on Node.js so okay so let me jump to the big dimensions I think this part wasn't there just to start with the very big thing there are two types of functions I think this is the most fundamental thing with Node there are two types of functions there are callbacks you just keep calling callbacks callbacks and they all look exactly like that function error takes the results yeah callbacks should handle errors all the time if it can and there are actions actions represent a sync operation so if I want to get stuff from the database I do an action last argument of the action is always going to be a callback and it should not range the exception I think all these like I might say there is a lot of taking that sort of ball behind it exceptions so by the way if you like exceptions in JavaScript anyway yeah exceptions are a sure way to kill your application so don't use exception in JavaScript it's weird because of the whole stack weirdness it's got so that's the reason why Node always says use earn and reserve it brings it like the first thing in the in the callback I didn't follow how it breaks up the problem is that it's actually not basically in Ruby code there is no concept of where I work for example right like you have to put a global handle in this case there is a concept of where you should stick to it and most people will not catch exceptions in there like if you are a what you lost for example you shouldn't be throwing exceptions because your application the one that is hosting it will not be catching it or at least the convention is not to catch it sure let me let me give like a skeleton of this thing so that's the action look it look it is the action basically it's trying to trying to get a DNS basically map IP address to a location like a city your company a given IP to say oh you are new in Bangalore so it takes an IP and takes a callback right does this weird things and then it has this thing that says oh call this if there is an error pass the error back if there is nothing just return the result is that what you say what do you want to do so that's the I mean you see this everywhere in the node this is like a standard template I mean if you just have a text made shortcut just shortcut it so I think like the biggest complaint that most people have is like oh I don't understand this stuff and if you just follow some conventions like this I think it's a big man so this is this part slightly controversial everybody use JavaScript or JavaScript oh not many so I'll be a little controversial and say if you unless you are like really fanatic about JavaScript just use JavaScript don't bother about the JavaScript it's too weird anyway it has all the supposedly the good parts and none of the bad the one thing that I really like about JavaScript it generates really neat code for the JavaScript that you write so even if you just read the JavaScript that it generates like the nice wrapping it up so I think I've learned a lot by just looking at the generated JavaScript and if you know JavaScript I'm sorry so I think to even use JavaScript you need to know a little bit of JavaScript I think it's like a cache 22 but yeah I'm assuming that people know some JavaScript and then they just want to really quickly write code and use JavaScript so this is like I've got complaints about JavaScript it's local versus global scope it picks up some of the bad ideas from JavaScript and continues them again I don't remember the exact argument but I've been told by the guy who wrote a classic framework from my he's got this thing about saying that there's one little thing in JavaScript that is wrong and that the author does not need to fix could basically make JavaScript not usable in the long term I don't understand what it means but I know that this could be smart enough that he's not making that so I haven't had that like JavaScript you always have that local, global like somebody requesting it and all these weird issues I haven't had to tell whatever JavaScript I've written I haven't had like these weird issues so some issue that the projective for in my particular situation the projective is the way to use that function by default it assumes the new scope on the left side it will actually come from the so if you assign yourself if you want to increment then it creates a new variable so then one place one particular different specification where you have a variable in scope that you can't access so it has some of the bad parts of JavaScript let me change my question so this is a classic JavaScript pattern I basically write all the code in JavaScript and have several .gears manually compiled into JavaScript and then I do require copy script and then just just load every other copy script file from inside that so I don't have to compile all the other chain classes separately so just to require the copy script I mean it uses the npm method the node hooks to compile it into I mean so this is a common problem you'll have with copy script I mean I think everybody complains about it I guess the only solution is to write decent code and not have these errors but nobody loves that so it's really hard to debug and stuff and like compile and like fix the JavaScript and fix the copy script and stuff there are things that I don't use actually let me just be clear it's not like I completely say go to copy script I'll be writing copy script there's stuff like compensions and loops I think I'd prefer underscore than copy script I also don't use classes that much and maybe just that functional style of programming but I don't know it's just I don't use the copy script like I was talking with Vishnu like the double arrow I hardly ever use so let me move on to the next one I think programming given a graphical and sort of this is exactly how I implemented the very first time and this is almost like we wanted to track so the gem that we wrote basically would be gem or module that we wrote would be installed on a server and would track some of the things that are going on in the server and periodically post it to our very post metrics application and we wrote it in a very weird way we found a channel that was awesome it finds all the things that we wanted so by the way I'm going to overload you with code hopefully you can see it this will be copy script so hopefully it will be easier but anyway you just need to see how it looks probably you don't want to go into that detail basically this is exactly how I wrote the first time this actually works I think it's probably I mean if you see that it will be there basically it's callbacks inside callbacks inside callbacks so first you get the version you get the environment then you get prefix then you get npm version and so on you just contact it sequentially because I want to make one call to the server at the end it makes sense right am I making any sense yes yes so right at the end all of this is just one call because I thought the idea of the callback would be that you make a call make a return and then pass from the value somewhere else yes yes so bear with me this is not like idiosyncratic copy script basically the idea is I want to get all the information about that machine and do a callback right at the end there's a callback right at the end here collect all the data and post it back and obviously I mean I don't have to say it like this far so let me do the second implementation it's much nicer basically I collect all of these and they run the stuff execute all of them and then callback yeah is there a mistake is there it's wrong it's like the first mistake no new makes yeah so basically so if you see I may call these calls but immediately after that I just callback the data has got nothing data can be right the callback has to happen right after all of the calls have come back I have to collect all the responses from each of the calls and then say this is the data this is slightly certain but I think most people who don't know it will not realize there is a big problem with it so what happens is the very first time I saw the data was empty and suddenly like almost like second and a half later I see all the data half later so it's like really starting to debug as well because they are working on the shared data of it right okay so basically I have and the idea is basically now I am using a counter to say yeah just hold with me for a minute I think this is hard but I so basically I am using a counter like there are like say 8 calls I make and I track each one and say oh this was decremented let me this is still not 1 and hold that call back till all the calls have been connected by the way the underscore means the underscore like yeah making sense so so you have to sequence all of them right at the end of the completion of all of the calls basically you have the ability to make the call back if you want it correctly and that's what I am trying to do here so yeah this relies on counter and this assumes that you don't have to worry about environment has to be called before get CPUs or get platform is called before they are all running in parallel and I just care about all of them ending right and sorry there is this thing called next take most of the reactor loops give you so you could also use next but this is like this is how I wanted to show that it works so basically with Sling programming there are a bunch of things that you need to worry about some of them have to run in C like for example I have to do a count before I populate something else sort of requirement you have to know when all of it is and the previous example was similar like unless all the calls have been finished you can't do anything till that point you have to block not block as in the Ruby style of blocking but yield to the reactor loops the other thing that you find hard these sort of things is like what happens when there is an error like I'm running five things in parallel in a sense in parallel what happens one of them blows up do I do I block do I kill the whole operation do I just ignore that result so there are like many start implementing something like that it gets complicated so the common mistakes you have is not follow conventions which is like follow error callback errors and actions taking callbacks really bad handling like exceptions are not handling because I think this is controversial as well is trying to I think the whole direction Ruby community has taken is using fibers to make sequential code look non sequential code looks sequential there are promises in JavaScript which you can use sort of oh this happens this happens this happens only when this promises there are fibers also in JavaScript I posted that they are more complicated than just using a simple flow control library so your mileage may vary I prefer just using a simple flow control than fibers or problems I did understand the reason that you do not like fibers but in Ruby fibers are major to the language to Ruby there are facts people have put them into VA to make fibers work so that's why when it breaks it breaks unpredictable things I think promises are similar as well the thing is to understand like the whole idea at least for me is to understand how it works and then use it or not use it if I don't even understand the concept of promise I find it hard to digest it what I think promises are usually just a library in JavaScript but there is a known fibers runtime and they technically could write same code actually a sync but looks synchronous also that does itself in a tricky one I want to know what is a sync what is not a sync by mixing sync and anything stuff together I find that issue in Ruby I want to be clear for this is a sync there is a call guard and it will complete at some point when you mix a sync and sync together at least for me and maybe for a lot of starter node developer it will be just confusing is it complete now or complete at some point in time so anyway to understand the plumbing there are ways to write your own code but don't write it there are like I think like at least 40 libraries that do flow control so you can check that out I think there are like more async as well so one that I recommend if you use sync sync is pretty popular the other one I use is very similar there are some things that you can't do with sync and there are some things that you can't do with async so I generally recommend to sync so basically let me go back show that code on how it looks is it here? basically you wrap it to the sync this thing and say run all these parallel like all the methods that are there in parallel wait for the results to come and call back really simple it is really similar to the stack API yes I think it is based on stack it is the same author who wrote the stack yeah okay sure so I need to run I think a little bit faster than this no I can't okay so it is really simple parallel execution wait for everything to complete execute the sequential operation and then give the error right so it gives you a for all these three things you can even chain it to an extent that's the best thing about sync so so the other breaking of the modules itself is a very well known pattern at least for the javascript developer or jQuery author I think it is called event emitter it is called something else in basically events in jQuery almost everything is an event emitter sort of sort of an event happens you just emit that and somebody else will catch it almost like the jQuery find trigger mechanism right most objects are a lot sorry most of the code mode objects are event emitter so you basically can say on error do this on something do this it is the same as trigger and bind mechanism in jQuery so the way we modularize applications in jQuery or parts of applications in jQuery is exactly how you do it in node as well just use triggers to part mechanism as well so one thing if you want to take away from this talk is that it's not hard to modularize in javascript or node it's just the techniques that we do in object oriented languages or like in Ruby for example is not the same as what you do in javascript or node anyway quick one so the other thing that we found is we prefer async architectures than just synchronous so reddit boxup is how many use reddits have used reddits so reddit boxup I really like it because it's just completely asynchronous when you start programming with node you sort of start going away from synchronous sort of operation we use first application where we sort of every user logging in shows up on the client browser use websockets to basically hook up to the reddit monitor basically it hooks up to the reddit monitor all the events are just piped back to the browser it's really simple com series has underscore changes which you can use I always find it better than just making adopt queries like it's a problem on Heroku I think I mean if you stick to Postgres on Heroku I get just like you have to call back the whole change sort of happens it's nasty so let me breeze through this basically common jazz is a way to sort of bind the whole lot of node applications together so understand it basically there are a couple of ways of using require one is to basically expose simple simple sort of methods and other is to expose like classes itself or classes prototypes so you choose the right one if you're exposing a lot of methods just expose a class which which you can pass it along to the other day ok so this is also slightly controversial I had at least when we were doing websockets we had lots of memory issues I think so every two days so this was this used to be running right and people would come and log in or do stuff and every two days we just it just dies because it would run out of memory on NOD so when we practice we basically found there was Socket.io that was working even Socket.io causes problems on the web browser node.js probably has some leaks Socket.io I think definitely somebody else said probably fixed on 0.8 or the later versions I can't be sure quickly running through debugging in NOD like do you actually debug what do you do so there's I think this thing called node inspector please check it out I think it's really nice because you can actually step in sometimes people have to do this but obviously as you can see it's JavaScript rather than copy so you won't you'll never understand the mapping between copy scripts and JavaScript pretty much done just once I think it's about deploy already this is where I think I probably need to talk to you guys like each platform that we've deployed the only thing we've not done is we're pushing it and well we've actually tried it but I think it still nodes premature enough to not run it as a full-fledged HTTP server so either you project the engine and redirect calls back but when you do that socket is great so at least I haven't gotten the way to find a good way to connect it's still it's still hassle it's not part of the engine's distribution so that means maintenance is your problem so yeah that's one option or redirecting socket I have to do something else just socket I have to do it but it's both and they've already been solved by node and giant node and stuff so I think we should just deploy to some known deployment I think the two things I recommend Node.jitsu because it supports 0.6 and Node because they have really awesome support every time we ask them to do something they just either afraid of that so highly recommend Node.jitsu I think I'm close to that thanks guys for boring you guys I think the code was a lot of thank you so any questions? I do this but I still I've read a lot of blogs I came here in this talk just to understand why should I put myself to that JavaScript to you know I'm not code square I'm not handling 500,000 check-ins every day or more than that I'm not trello why should I put myself to this I'm very open to I've read a lot of blogs I just don't have a solution why should I use Node.jitsu I think you know what if you're running a really smallish application I will know if someone is code square so I think eventually it is about how much you want to the request per second I also work in a really large for a really large application that has like close to 10 million request per seconds and the only way we can add even though I know like the servers large boxes that we have it's just adding new boxes like it's taking 15% CPU on that machine I know we have to like roll something else better or something else to get some data like there is no reason for me to not utilize that capacity so what what do you use it for what do you process in whatever the code that you have written to process the 10 million request what are you processing we can take this offline but basically the point the only thing I'm saying is if you want to scale an application and if it's I have gone mostly then I think the only two alternatives are line or Node.jitsu like you can obviously if you start we like Ruby stuff so we use Event Machine internally but it's still I think Event Machine is really hard compared to Node.jitsu so we scale a few things Event Machine rather than Node.jitsu but anyway we can take that any other question