Should I check in node_modules to git when creating a node.js app on Heroku?
My biggest concern with not checking
node_modules into git is that 10 years down the road, when your production application is still in use, npm may not be around. Or npm might become corrupted; or the maintainers might decide to remove the library that you rely on from their repository; or the version you use might be trimmed out.
This can be mitigated with repo managers like maven, because you can always use your own local Nexus or Artifactory to maintain a mirror with the packages that you use. As far as I understand, such a system doesn't exist for npm. The same goes for client-side library managers like Bower and Jamjs.
If you've committed the files to your own git repo, then you can update them when you like, and you have the comfort of repeatable builds and the knowledge that your app won't break because of some third-party action.
I followed the basic getting started instructions for node.js on Heroku here:
These instruction don't tell you to create a .gitignore node_modules, and therefore imply that node_modules should be checked in to git. When I include node_modules in git my getting started application ran correctly.
When I followed the more advanced example at:
It instructed me to add node_modules to .gitignore. So I removed node_modules from git, added it to .gitignore, then re-deployed. This time the deployed failed like so:
-----> Heroku receiving push -----> Node.js app detected -----> Resolving engine versions Using Node.js version: 0.8.2 Using npm version: 1.0.106 -----> Fetching Node.js binaries -----> Vendoring node into slug -----> Installing dependencies with npm Error: npm doesn't work with node v0.8.2 Required: firstname.lastname@example.org || 0.5 || 0.6 at /tmp/node-npm-5iGk/bin/npm-cli.js:57:23 at Object.<anonymous> (/tmp/node-npm-5iGk/bin/npm-cli.js:77:3) at Module._compile (module.js:449:26) at Object.Module._extensions..js (module.js:467:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Module.require (module.js:362:17) at require (module.js:378:17) at Object.<anonymous> (/tmp/node-npm-5iGk/cli.js:2:1) at Module._compile (module.js:449:26) Error: npm doesn't work with node v0.8.2 Required: email@example.com || 0.5 || 0.6 at /tmp/node-npm-5iGk/bin/npm-cli.js:57:23 at Object.<anonymous> (/tmp/node-npm-5iGk/bin/npm-cli.js:77:3) at Module._compile (module.js:449:26) at Object.Module._extensions..js (module.js:467:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Module.require (module.js:362:17) at require (module.js:378:17) at Object.<anonymous> (/tmp/node-npm-5iGk/cli.js:2:1) at Module._compile (module.js:449:26) Dependencies installed -----> Discovering process types Procfile declares types -> mongod, redis, web -----> Compiled slug size is 5.0MB -----> Launching... done, v9
Running "heroku ps" confirms the crash. Ok, no problem, so I rolled back the change, add node_module back to the git repository and removed it from .gitignore. However, even after reverting, I still get the same error message on deploy but now the application is running correctly again. Running "heroku ps" tells me the application is running.
So my question is what's the right way to do this? Include node_modules or not? And why would I still be getting the error message when I rollback? My guess is the git repository is in a bad state on the Heroku side?
I am using this solution:
- Create separate repository that holds
node_modules. If you have native modules that should be build for specific platform then create separate repository for each platform.
- Attach these repositories to your project repository with
git submodule add .../your_project_node_modules_windows.git node_modules_windows
git submodule add .../your_project_node_modules_linux_x86_64 node_modules_linux_x86_64
- Create link from platform-specific
node_modulesdirectory and add
- Commit submodule repository changes.
- Commit your project repository changes.
So you can easily switch between
node_modules on different platforms (for example if you are developing on OS X and deploying to Linux).
I was going to leave this after this comment: Should I check in node_modules to git when creating a node.js app on Heroku?
But was formatting it weird. If you don't have identical machines and are checking in node_modules, do a .gitignore on the native extensions. Our .gitignore looks like:
# Ignore native extensions in the node_modules folder (things changed by npm rebuild) node_modules/**/*.node node_modules/**/*.o node_modules/**/*.a node_modules/**/*.mk node_modules/**/*.gypi node_modules/**/*.target node_modules/**/.deps/ node_modules/**/build/Makefile node_modules/**/**/build/Makefile
Test this by first checking everything in, and then have another dev do the following:
rm -rf node_modules git checkout -- node_modules npm rebuild git status
Ensure that no files changed.
Instead of checking in node_modules, make a package.json file for your app.
The package.json file specifies the dependencies of your application. Heroku can then tell npm to install all of those dependencies. The tutorial you linked to contains a section on package.json files.
I have been using both committing node_modules folder and shrink-wrapping. Both solutions did not make me happy.
In short: committed node_modules adds too much noise to repository.
And shrinkwrap.json is not easy to manage and there is no guarantee that some shrink-wrapped project will build in a few years.
I found that Mozilla was using a separate repository for one of their projects https://github.com/mozilla-b2g/gaia-node-modules
So it did not take me long to implement this idea in a node CLI tool https://github.com/bestander/npm-git-lock
Just before every build add
npm-git-lock --repo [firstname.lastname@example.org:your/dedicated/node_modules/git/repository.git]
It will calculate hash of your package.json and will either check out node_modules content from a remote repo, or, if it is a first build for this package.json, will do a clean
npm install and push the results to the remote repo.