Project Plumbing with Plumbum (Part III)

I’ve written before about Plumbum, first about it’s basic capabilities and then about it’s ability to do things remotely over a network. To paraphrase the great Bill Cosby, “I told you those stories to tell you this one”. In addition to all it’s other features, Plumbum also provides a complete framework for building command line applications in a clean, declarative way.

(more…)

JavaScript Isolation with Nodeenv

I do a lot of Python work, and I’m a huge fan of virtualenv, which allows you to set up a completely isolated Python environment. This is hugely helpful to keep various dependencies from multiple projects from clobbering one another. It’s so helpful, that when I work in pretty much any other language, I miss having the ability to isolate things so nicely.

Since a lot of the Python work I do is also Web work, I frequently end up doing a lot of Javascript as well. The (relatively) recent appearance of node.js and various development tools based on it has been a great boon, but whenever I’ve had multiple projects using different versions of the same tool, I’ve craved a way to keep my Javascript environments as nicely separated as my Python environments. If only there was something like virtualenv, but for node.js…

In fact there are several ways such things, but my favorite by far is a project called nodeenv. It’s written in Python, which might seem a bit strange until you realize that it’s written specifically to integrate nicely with virtualenv.

Here’s how you use it. Or at least how I use it.

First create a virtualenv. Here I’m using virtualenvwrapper, but of course you can use vanilla virtualenv if you prefer:

$ mkproject nodeenv-test
New python executable in nodeenv-test/bin/python
Installing setuptools............done.
Installing pip...............done.
Creating /home/funsize/code/nodeenv-test
Setting project for nodeenv-test to /home/funsize/code/nodeenv-test

This will create a project directory, create a virtualenv, install pip into the virtualenv, activate it, and switch my current working directory to the project directory just created.

Now we install nodeenv into the virtualenv:

$ pip install nodeenv

This installs a command called, unsurprisingly, “nodeenv”, which we now run with the -p flag, which tells it to integrate with the current python virtualenv:

$ nodenv -p
 * Install node.js (0.10.26) ... done.
 * Appending nodeenv settings to /home/funsize/.virtualenvs/nodeenv-test/bin/activate

Everything will probably work fine at this point, but just to be safe, I like to reset the virtualenv by deactivating and reactivating it:

$ deactivate
$ workon nodeenv-test

(Again, I’m using virtualenvwrapper, which may or may not be to your taste).

At this point we are working in an isolated environment for both Python and Node.js. I can install libraries for Python using pip, or for Node using npm -g and they will not interact with my system Python libraries or Node libraries in any way. Some of you may recall that I’m a big fan of bower, so I can install it like so:

$ npm install -g bower

Bower has a bunch of dependencies, which npm takes care of for us, and all of those get installed into our isolated environment. They can’t interfere with any of our other projects in any way. Bliss!

Ok, so it’s not quite all sunshine and roses. nodeenv needs a bunch of dependencies to work, though nothing too outlandish (stuff like a C compiler, curl, etc.). This isn’t too onerous, and anyone who can install software packages on their machine should be able to get it working on pretty much any Unix flavor without too much trouble. I have not, much to my disappointment, been able to get it to work on Windows as of yet, even with all of the relevant dependencies installed. It appears to have something to do with the way Windows handles command parsing and/or escaping. If anyone out there has gotten it running on Windows, please let me know in the comments. With those caveats, though, nodeenv really is pretty fantastic.

Now this is the kind of thing that you may not really appreciate until you use it, so I encourage all of you out there to give this a shot. It really is worth your time. Enjoy!

Accelerated, Lower Frustration Web Development with BrowserSync

With apologies to Mr. Poe:

I was fuming in frustration, sitting clicking at my station,
I'll spare you some narration, but I was getting pretty sore.
"Surely there must be some method, whereby changes are detected,
And perhaps then be injected into my page's DOM!", I swore.
"Some way to edit CSS without it being such a chore,
And click Refresh nevermore."

So I then took up the challenge, and began a quest for knowledge
on this strange mysterious thing for which I'd searched some times before.
But amidst auto-refresh plugins, and bookmarklets by the dozen,
Nothing did I find which would my page's state restore.
Then I chanced upon a newsfeed and this here did it implore:
"BrowserSync, you will adore."

-- not quite Edgar Allen Poe

BrowserSync is a nifty development tool for editing and/or testing websites. It’s written in JavaScript, on top of node.js, so you install it using npm (perhaps inside a nodeenv ). After installing it, you start it up something like this: (more…)

Project Plumbing with Plumbum (Part I)

Bash scripting is hard, let’s go plumbing

Pipes!

Image via http://www.flickr.com/photos/ctweed/

Consider the following scenario:

Let’s say you’re working on a software project. Maybe it’s a web service, maybe a GUI app, whatever. Doesn’t matter. As usual, you discover there’s some tedious task that needs doing repeatedly, so you decide to automate it. Since it’s pretty much the easiest thing you can think of, you crank out a quick bash[1] script, which seems to handle things for the moment.

Later, you find some more similar tasks, so you crank out some more bash scripts. And then some more. Then you realize that you’re repeating yourself an awful lot, so you try factoring out some stuff, calling some scripts from other scripts. Eventually you realize that you have dozens of bash scripts calling each other in various combinations, scripts depending on other scripts three and four deep, parameter passing that makes your eyes melt from all the “$1″s, dogs and cats living together, mass hysteria!

But what else can you do? I mean, these scripts are basically just calling a bunch of command line tools, so we have to use a shell language to automate them, right?

Nope. Totally wrong.

Plumbum is a Python module originally written by Tomer Filiba which adopts the motto “Never write shell scripts again”. You can use it to completely replace those pesky, unwieldy shell scripts with nice, clean, reusable Python.

(more…)