The node.js Community is Quietly Changing the Face of Open Source « caines.ca

If you consider the number of packages on pypi, npm, and rubygems, and the release dates of python (1991), ruby (1995) and node.js (2009), it looks a lot like those communities are getting new packages at the following rates, per year:

python:  29,720 packages / 22 years = 1351 packages per year

ruby:      54,385 packages / 18 years =    3022 packages per year

node.js 26,966 packages / 4 years =   6742 packages per year

It’s important to note that I’m only showing the other languages as a measuring stick.  There are probably a lot of reasons for the disparity here (including my imprecise math), and I’m not trying to say anything like “node.js > ruby > python”  but obviously new node.js packages are being published to npm at a staggering rate.   I just checked npm (on a Sunday night) and 20 new versions of packages have been published to npm in the last hour alone.

This is a pretty amazing feat for node.js.  How is this possible?

Batteries NOT included

If you’ve needed an HTTP client in Python, you’ve probably asked yourself “should I use urllib, urllib2, or httplib for this?” like somanypeople before you.  Well the answer is probably that you should use requests.  It’s just a really simple, intuitive HTTP client library that wraps up a lot of weirdness and bugs from the standard libraries, but it’s not a standard library like the others.

The “Batteries included” philosophy of Python was definitely the right approach during the mid 90’s and one of the reasons that I loved Python so much; this was a time before modern package management, and before it was easy to find and install community-created libraries.  Nowadays though I think it’s counter-productive.  Developers in the community rarely want to bother trying to compete with the standard library, so people are less likely to try to write libraries that improve upon it.

Developers are less likely to use non-standard libraries in their projects too.  I’ve heard many instance of people suffering through using an inferior standard library just to have “no dependencies”.  But in this day and age of cheap massive storage and modern package management, there are very few reasons for a policy of “no dependencies”.

Conversely, the node.js core developers seem to actually want to minimize the standard library as much as possible.  They have repeatedly removed features from the standard library to free them up to be implemented by the community instead.  This allows for the most variety and lets the best implementation win (but only until someone makes a better one, of course!).

Imagine how liberating this is for standard library maintainers too.  The node.js standard library is comparatively tiny, freeing the core developers to just deal with the core necessities.

The Tiny Module Aesthetic

Just like the 140 character limit on twitter makes people “blog” more, the node.js community has a culture of publishing tiny modules that makes people more comfortable with publishing smaller packages, and so it happens vastly more often.

While I don’t really consider myself “a node.js developer”, I’ve found myself publishing over a dozen packages in the last year myself, just because I’m working on a node.js project at work, and I publish any reusable package from that work that I create.  My project ends up factored better, and I end up with a bunch of building blocks I can re-use elsewhere (and then I do!).  Sometimes random people from the interwebs even fix my bugs before I notice them!

This approach is not unusual at all either, at least in the node.js community.  The community is lead by developers that have each published literally hundreds of packages.  I don’t even know how thisispossible.

Of course there’s absolutely nothing to stop other languages from following suit, so it seems to me like community culture is the deciding factor.

Massive monolithic frameworks like Ruby on Rails tend to not take hold in the node.js community mindshare, I think due in part to this “tiny module aesthetic”, and it’s all the better for it.  It seems to me that monolithic frameworks like Rails don’t have a clear notion of what does not belong in Rails, and so they tend to expand indefinitely to include everything an application could need.  This leads to people trying to assert that Rails is the best solution for all problems, and having a cooling effect on innovation within the web application space in the Ruby community.

The tiny module aesthetic also leads to an ecosystem of extreme re-use.  When a package doesn’t do more than is absolutely necessary, it’s very easy to understand and to integrate into other applications.  Conversely, while massive monolithic frameworks tend to appear in a lot of end-user applications, they rarely never end up being dependencies in other packages themselves.  It seems unfortunate to me that so much great software should be so difficult to re-use.

git and github.com monoculture

To be honest, few things are more boring to me in software development than version control, and I find the complexity of git to be a bit silly and mostly unnecessary for me, but I’ve never heard of a node.js developer that doesn’t use it, and there’s huge value in that kind of monoculture.  It means that they all speak with a common language and there’s no impedance when you want to contribute to someone else’s projects.

Github.com has a similar effect in lowering the barriers for cross-pollination between developers.  I’ve only very rarely seen a node.js project that wasn’t on github.  This means I immediately know exactly where to find source code if I want to contribute.

The pluses and minuses of a github monoculture go far beyond this, but there’s probably enough fodder there for an entirely different post.

Permissive Licensing

Node.js packages tend to use extremely permissive licensing like MIT and BSD licensing.  In fact, the default license for a package created by `npm init` is BSD.  I think this is another sign of the times.

Very few people care anymore if someone else forks and doesn’t contribute back.  There’s very little to gain in not contributing back anyway because of the cost of maintaining your own fork.

This is important because no one wants to have to deal with dependencies that make them think about legal repercussions, or give them extra responsibilities.  Licenses like the GPL end up having a cooling effect on software-reuse for this exact reason.