blog

Node.js – This Should Be Easy (and it is, eventually)

by

Software screen capture

It’s been on my TODO list ever since I first heard about it. Everyone I spoke to seemed enthused about it – or else viciously opposed to it, which is almost as strong an encouragement to try it for myself, just to see what all the fuss is about. I already knew Javascript – the original transition from my background in C and Java had been a bit rough, but I’d been over that bridge some years back, and node was Javascript, so no problem, right?

 

Ah, blissful ignorance. When last I had a break between projects, I resolved to scratch that ‘Node.js’ line off my list. Importantly, I had only one week to get to a point of semi-demi-competence, which I figured (hoped) was doable, given that I already knew the language. Probably, if it had been the ‘write something useful in Malbolge‘ line I was trying to cross off, I would’ve at least expected to be frustrated. This is going to be a long, rambling sort of post from frustration to (something approaching) enlightenment – fair warning.

 

Getting Started

Not that Node is particularly difficult to get started with – quite the contrary. There’s a preponderance of ‘getting started with node’ tutorials out there that will walk you through serving a web page with the http module. You’ll see something like:

[sourcecode language=”javascript”]
var http = require(‘http’);
http.createServer(function(req, res){
res.writeHead(200, {‘Content-Type’:’text/plain’});
res.end(‘See? This is easy!’);
}).listen(8080);
[/sourcecode]

Wow. That really does look easy. Its no wonder this is the snippet you’ll see repeated time and again – say you’re used to installing and configuring Apache, and here you are serving a web page with a few chained function calls. Nice.

 

So… how do I listen for various urls on my server (regexp on req)? How about reading files to serve them (sync/async calls)? Authentication? Databases? Hmm, this getting started tutorial just stops after that cool snippet. Well, there must be another that doesn’t… somewhere….

 

Batteries Not Included

I think the real problem may have been that I’m spoiled. I’ve gotten used to the ‘batteries included’ experience. Working with limited time frames means that I need that batteries included experience if I want to deliver a given project on time and on budget. Node doesn’t offer that – at least, not at first blush.

 

Maybe it was also my expectation as to what Node offers; and which, in all fairness, all those getting started tutorials biased me towards – that Node is for serving web sites. If it really were, then it should have more of the functionality one would expect from a modern web server and framework baked in.

 

So what is Node, then? JavaScript outside of the browser, and a set of libraries to accomplish some (many?) of the things you might want to do outside of the browser. Full stop. It took me about a day of reading tutorials and looking through the Node api to accept this. Gah, all I want to do is serve some html. What is all this? I thought this was supposed to be easy!

 

Getting Started, Again

After sitting down more seriously with the api and quick-stepping my way through Hands-On Node by Pedro Teixiera (which is a nice expansion on the api docs), I was able to come to grips with the fact that those getting started tutorials weren’t really starting in the right place, at least not for my needs. They were showing off something cool and easy, making a point about how different Node was.

 

They weren’t telling me that Node was not an environment solely dedicated to serving up web pages and did, in fact, need some knowledge of how the environment and libraries worked in order to get things done. That it wasn’t a web framework – those could be built on top of Node, but Node wouldn’t do it for me. It seems so simple and obvious in retrospect, but all the hype I’d heard had been about Node serving web pages. When I have read, since, posts indicating disappointment with Node, I suspect they are experiencing the same let-down that I did.

 

So, let’s start again. Node is not for serving web pages. It can do that, however, and that’s what I want it to do right now. How do we go about that?

 

NPM and package.json

Package management doesn’t seem like a fun place to start. It doesn’t involve writing any code (json doesn’t count). It’s the right place to start, though.

 

New installs of Node are easy, even on Windows, and it comes pre-packaged with NPM. Node Package Manager isn’t for publishing your modules (although it can do that). It’s for getting all your dependencies easily, playing with different modules until you have the right mix. After my first npm install, a little bit of my enthusiasm returned.

 

After you’ve installed Node (using one of the Node Installers), let’s say you create a directory like ‘nodetest’. Next step is to write a file called ‘package.json’. It will look something like this:

[sourcecode language=”javascript”]
{
"name":"nodetest",
"description":"Node Test App",
"version":"0.0.1",
"private":true,
"dependencies":{
"express":"3.1.x",
"express-plates":"0.1.x",
"JQDeferred":"*"
},
"devDependencies":{
"vows":"0.7.x"
},
"bundledDependencies":[
"express-plates"
]
"engines":{
"node":">=0.8"
}
}
[/sourcecode]

An excellent resource for figuring out what each line is and/or does is http://package.json.nodejitsu.com/. Overall though, its not too difficult to work out – details about your app, what version of node it works on, and its dependencies. The magic is that, when you save the file and, with some trepidation, type in npm install and it works… ah, now we’re getting somewhere.

 

Choices, choices

So, what am I putting in those dependency objects? In some ways, Node feels like a world built on the iOS catch-phrase, ‘there’s an app for that’. If you need something done, there’s likely a module for that. In this case, we need a web framework.

 

Searching Google and Stack Overflow will reveal there are any number of web frameworks for Node, with a variety of paradigms, each with their staunch proponents. I liked the look of Matador. Flatiron and its associated packages caught my eye as well.

 

But as a beginner getting started, what I really need is a good idea of how things work – some examples for things I might want to do with my node server, and a large community to query when I run into problems. Enter Express. The community support for Express seems to be the largest (and many other web frameworks are built on Express, such as the aforementioned Matador, so it wouldn’t hurt to understand how to work with it even if you intend to take advantage of another layer over top of it), and more importantly, the github repo has a directory full of examples that cover many of the common usage needs.

 

Its hard to overstate how helpful those examples were when I felt like I was scrabbling for purchase at the roots of the Node mountain.

 

Modules: require, exports

So, now I know I’m going to use Express to serve my pages (and take advantage of all those examples, and possibly the upcoming ‘Node.js in Action’ book, to figure out how to do it). How do I get this Express thing hooked up and ready to go, with the rest of my logic?

 

Node implements the CommonJS recommendations for module importing/exporting. That ‘require’ you saw at the beginning of the simple http server example is how we import modules. Here’s another neat thing – once you’ve installed your dependencies with npm, all you need to do is:

[sourcecode language=”javascript”]
var express = require(‘express’);
[/sourcecode]

There, you’ve imported express and all its functionality into whatever module you’re working with. I won’t reproduce the examples I’ve linked to here – but once you’ve walked through a few, suddenly it does seem surprisingly easy to serve arbitrary content, like you wanted to do when you began. Throw in a template engine like Jade, or (my preference) use the familiar DOM-style manipulation through something like Plates or Cheerio, and we can manipulate our markup before sending it on to the client. It’s all coming together.

 

Other Stuff

Part of the enlightenment that came with fully realizing Node wasn’t about serving web pages, was realizing that Node didn’t have to be about just serving web pages (stunning, I know). It could fill the role that Python fills (or Java, when I need to pull out the big guns or my rough and ready knowledge of Python fails me) – useful scripts, one-off automation for a large job, etc. Throw in modules like colors and optimist, and it might actually become my preferred option (not yet, but it might be someday).

 

The fact that Node is useful for other stuff shouldn’t have been such a revelation. But for me, it was, and it restored a bit more of the shine that had been slowly re-accumulating as I discovered just how to go about getting things done with Node.

 

Further Thoughts

I’m not convinced Node is the best solution in all cases for all things (I think most of us have realized it’s not). I’m not even convinced its the best solution for serving web pages (there are definitely cases for its model of handling concurrency, and cases against – one must choose the tool appropriate to the task). There’s some resistance to Node that comes from it being shackled to the reputation of JavaScript (which has been oft-maligned undeservedly, but understandably – a topic for another post, perhaps).

 

That said, if Node is on your TODO list (because you’re interested, or because it might be the right tool for the job at hand), I have good news: you’re right to be interested (it’s interesting); it just might be the right tool for the job.

 

But for goodness’ sake, don’t start with that http example and think that this is the fast track to web serving! This is going to be easy! It only leads to tears, recrimination, and needing to read the Node api through twice.

 

+ more

Accurate Timing

Accurate Timing

In many tasks we need to do something at given intervals of time. The most obvious ways may not give you the best results. Time? Meh. The most basic tasks that don't have what you might call CPU-scale time requirements can be handled with the usual language and...

read more