New users of pnpm frequently ask me about the weird structure of
node_modules that pnpm creates. Why is it not flat? Where are all the sub-dependencies?
I am going to assume that readers of the article are already familiar with flat
node_modulescreated by npm and Yarn. If you don't understand why npm 3 had to start using flat
node_modulesin v3, you can find some prehistory in Why should we use pnpm?.
So why is pnpm's
node_modules unusual? Let's create two directories and run
npm add express in one of them and
pnpm add express in the other one. Here's the top of what you get in the first directory's
.bin accepts array-flatten body-parser bytes content-disposition cookie-signature cookie debug depd destroy ee-first encodeurl escape-html etag express
You can see the whole directory here.
And this is what you get in the
node_modules created by pnpm:
.pnpm .modules.yaml express
You can check it here.
So where are all the dependencies? There is only one directory in the
.pnpm and a symlink called
express. Well, we installed only
express, so that is the only package that your application has to have access to
Read more about why pnpm's strictness is a good thing here
Let's see what is inside
▾ node_modules ▸ .pnpm ▾ express ▸ lib History.md index.js LICENSE package.json Readme.md .modules.yaml
express has no
node_modules? Where are all the dependencies of
The trick is that
express is just a symlink. When Node.js resolves dependencies, it uses their real locations, so it does not preserve symlinks. But where is the real location of
express, you might ask?
OK, so now we know the purpose of the
.pnpm/ stores all the packages in a flat directory structure, so every package can be found in a directory named by this pattern:
We call it the virtual store directory.
This flat structure avoids the long path issues that were caused by the nested
node_modules created by npm v2 but keeps packages isolated unlike the flat
node_modules created by npm v3,4,5,6 or Yarn v1.
Now let's look into the real location of
▾ express ▸ lib History.md index.js LICENSE package.json Readme.md
Is it a scam? It still lacks
node_modules! The second trick of pnpm's
node_modules structure is that the dependencies of packages are on the same directory level on which the real location of the dependent package. So dependencies of
express are not in
.email@example.com/node_modules/express/node_modules/ but in .firstname.lastname@example.org/node_modules/:
▾ node_modules ▾ .pnpm ▸ email@example.com ▸ firstname.lastname@example.org ... ▾ email@example.com ▾ node_modules ▸ accepts ▸ array-flatten ▸ body-parser ▸ content-disposition ... ▸ etag ▾ express ▸ lib History.md index.js LICENSE package.json Readme.md
All the dependencies of
express are symlinks to appropriate directories in
node_modules/.pnpm/. Placing dependencies of
express one level up allows avoiding circular symlinks.
So as you can see, even though pnpm's
node_modules structure seems unusual at first:
- it is completely Node.js compatible
- packages are nicely grouped with their dependencies
The structure is a little bit more complex for packages with peer dependencies but the idea is the same: using symlinks to create a nesting with a flat directory structure.