Christopher Allan Webber

If your project uses NPM (or its dependencies do), it probably can't be packaged by distros

Christopher Allan Webber at

Yesterday two things happened in #mediagoblin:

  • One community member remarked in surprise that npm was listed as a requirement in MediaGoblin's documentation since the 0.8.0 release. Surprising indeed! This is because of the attempt to remove our committing of javascript libraries into MediaGoblin's extlib directory. The real goal of this was twofold: towards better practice so that upgrading javascript/css/etc dependencies was easier (we are using Bower on top of NPM), and to make packaging in distros easier (by avoiding leaving places where browser-client-side dependencies can be symlink'ed in rather than bundling these things ourselves).
  • The second thing that happened was I expressed in frustration how much I dislike NPM, towards which a friend in the channel exclaimed surprise. NPM is their favorite language package manager, how could I dislike it?

The irony that binds the former and the latter together is, despite our usage of NPM/Bower to simplify deployment, anything that uses NPM as a package manager can probably never be packaged by distros. This was the crux of my blogpost "Let's Package jQuery: A Javascript Packaging Dystopian Novella": because each dependency bundles its own dependencies, you end up with each dependency having its own versions of dependencies, and aside from being a bloated nightmare in its own way, you may have two versions of say uglify-js which are totally incompatible, and both of these are running side by side. This is a serious problem for several reasons:

  • On a traditional distribution like Debian or Fedora, this is an impossible situation, since traditional distributions cannot easily accomodate multiple versions of the same package.
  • Even in fancy-pants new-age functional distros like NixOS or GuixSD where such a thing is possible, managing that dependency chain is mind-bendingly complex, and almost nobody is going to want to do it.
  • The culture of npm is to break things down into many super tiny packages, which is fine in npm-land, but exhaustingly complex for those trying to package free software distributions.
  • This also introduces potentially serious security issues: even if you are using all the latest versions of your dependencies, your dependencies' dependencies may be massively out of date, and it's hard to keep track of that. You may be presenting your users with dangerous security vulnerabilities and may never notice.

Despite all this, npm is popular with many of its end users. It's easy to understand why: the decisions made above make it very easy to get up and running with npm, though not necessarily easy to maintain keeping things up to date in the long run, and when pushing out deployments while eschewing the system package manager, things appear to "just work". And if you're working for a startup which is using NPM for its proprietary "top level product" which only goes out to its own personal sever deployments, maybe this doesn't matter.

For those of us concerned with building applications we want to push to end users for advancing network freedom though, this might be a serious problem.

(The similarity to this problem to "application-container systems as static compiling on a distro-sized level" is left as an exercise for the reader!)

Elena ``of Valhalla'', Deunan, olm-e, Stephen Michael Kellat and 6 others likes this.

Olivier Mehani, Stephen Michael Kellat, David Thompson shared this.

I think that if Stanley Kubrick were still alive, he would definitely want to make the film version of "A Javascript Packaging Dystopian Novella." I, for one, would pay good money to see it. :-)

Stephen Sekula at 2015-08-20T15:53:44Z

Stephen Michael Kellat, Christopher Allan Webber likes this.

The same friend previously referenced from #mediagoblin brought up that this is a "fix the distros" vs "fix the package managers" discussion (well, kind of, it also might be "fix the packages"). But I think there may be another way forward that is kind of both related to functional package managers: if people were developing using nix/guix they could use the "universal virtualenv" type features each provide (eg "guix environment", and I forget what the nix equiv is) to do their development, at which point the package is probably also passing the test of being reasonably generally packageable / deployable as well.

It's also true that NixOS/GuixSD are more likely to be able to survive packaging applications designed with npm in mind than more traditional distros like Debian / Fedora can... since they are building derivatives which are purely based on their inputs, multiple versions of dependencies can be handled, it's just a bit of a pain in the butt.

But I think a package built with Nix/Guix as a starting point for a development environment is less likely to end up in the state of an NPM package which may potentially have significantly old dependencies in its tree, as it's fairly easy to identify and keep things up to date, and "sticking with old versions" is not considered a feature or something that happens as much without being noticed.

That said, I still value Debian / Fedora, and I would like modern applications to be able to be installed on them!

Christopher Allan Webber at 2015-08-20T16:31:10Z

Stephen Michael Kellat, David Thompson likes this.