Javascript Tools: A Story in Disgrace

Why am I even writing about this?

Building JavaScript apps is overly complex right now

Among other things The State of JS 2016 asked developers if they disagreed (1), were neutral (3), or agreed (5) with the following statement: Building JavaScript apps is overly complex right now


A full 59% of developers agree that Building JavaScript apps is overly complex right now. Only 16% disagree.

Javascript fatigue is real, no matter how hard some people try to shrug it off (see this for an example).

We’re told: move fast and break things. For once, just for once, I would like to stop breaking things and stick to something that works. Half of the issues here is something I’ve encountered in the past two weeks alone, when trying to create a project from scratch.

Developer Experience

Developer Experience is not a lost art. It’s an art that has never been discovered. It was briefly discussed a few years ago and then forgotten. These days if you ever hear it mentioned, it’s usually in the context of APIs.

However (emphasis mine):

As software consumes the world around us, good design in our tools is a growing competitive advantage

Our tools shape our work, and great tools change the shape of our industry.

We talk a lot about the importance of a strong engineering team, but not enough about the design of our tools and the impact it has on the quality of the products we build. We should be talking about DX more, and it’s not enough to talk about UX alone.

Steve Boak

Javascript tools are quite literally death by a thousand cuts. The whole experience of working with and building for Javascript is, at the very least, an excercise in frustration. The landscape is utterly hostile to developers. With experience you learn to navigate it, somewhat safely. Is it an experience you need, though?

Falsehoods programmers believe about Javascript tools

  1. Tools work when you run them
  2. Tools can be configured
  3. In JSON
  4. In Javascript
  5. Can be pointed to different config file
  6. Don’t use hidden/special files like .something.rc
  7. Tools fail if their config is incorrect
  8. Tools warn you about invalid values in configs
  9. Tools ignore invalid values in config
  10. Tools use defaults instead of invalid values in config
  11. Tools don’t ignore valid values in config
  12. Well, at least tools report non-config errors
  13. At least fatal errors?
  14. Tools propagate errors from their plugins or sub-tools
  15. At least fatal errors? I asked that one already, didn’t I?
  16. You can tell if an error originated in the tool, in a plugin or in a sub-tool
  17. At least, errors are clearly stated on screen/in logs
  18. At least, with reasonable and relevant information
  19. Tools can be invoked from command line
  20. Tools can be run on a list of files
  21. With glob patterns
  22. Minor versions of tools, or their plugins, or their sub-parts keep backwards compatibility
  23. Tools fail if none of their requirements are satisfied
  24. Tools fail if some of their requirements are not satisfied
  25. Errors or warnings if this happens
  26. There is only one way to do things
  27. There is more than one way to do things
  28. These many ways lead to the same result
  29. Your tool will be relevant 5 years from now
  30. Ok, a year from now

So, let’s talk tools.


npm is the ubiquitous javascript tool. You may still run into bower occasionally, but that battle seems to have been lost.

npm is:

It probably does other tasks, but these are the most important ones. It does its job quite well, and I cannot recommend this post highly enough. Still, there are gotchas.

Run whatever I think you want, not what you want

This is relatively a minor WTF, but it’s there nonetheless:

If the specified configuration param resolves unambiguously to a known configuration parameter, then it is expanded to that configuration parameter. For example:

npm ls --par
# same as:
npm ls --parseable

In the example above --par is an invalid parameter. npm will not silently ignore it (which would be bad). npm will silently expand it to whatever parameter or combination of parameters it fuzzy matched.

npm has been notoriously bad at detecting actual errors. For example, until version 3.x (!) it would not fail if its configuration file was invalid.

Depend on whatever I you think you want, not what you want

Let’s consider npm install --save. This installs a dependency and adds it to the dependency list in your project’s package.json. And by saving I mean “take the list of dependencies, sort it alphabetically, and write it back to disk”.

This would not be a problem, save for this:

The npm client installs dependencies into the node_modules directory non-deterministically. This means that based on the order dependencies are installed, the structure of a node_modules directory could be different from one person to another. These differences can cause “works on my machine” bugs that take a long time to hunt down.

Yarn: A new package manager for JavaScript

Brilliant, isn’t it?

Remove all the things

Well, you know/remember this one: Rage-quit: Coder unpublished 17 lines of JavaScript and “broke the Internet”

Many were quick to attribute the problem to the horrible programmer culture of JavaScript, where people have forgotten how to program. It’s not the case. JavaScript community has whole-heartedly adopted Unix’s philosophy of “one package has to do one thing, and do it well”, but may have taken it to extremes.

The problem, or rather problems (plural) are all laid out here:, so I’m not going to re-iterate.

npm and npm’s registry are essential to developers and to developer experience. The way some/many of arising issues are handled by npm’s organisation are clearly detrimental to developer experience.

Your package name doesn’t matter. Until it matters

First, this story.

So after a search for various of keywords I found out that the module name npmjs was still available in the registry. In the four years that the registry existed nobody took the effort in registering it. This module was and is a perfect fit for my module.

On the 22th I received an email from Isaac, the CEO of npm Inc (which recently raised more than 2M in funding for his company) and creator of npm with a question:

Can you please choose another name for this module? It’s extremely confusing. Thanks.

It didn’t even matter what how right or wrong I was for using npmjs as a module name Isaac had clearly already decided to destroy the module as he stated there wasn’t any negotiation and that it would be deleted no matter what.

Arnout Kazemier

Sounds bad, doesn’t it? Ok, what about this story then (emphasis mine)?

For a few minutes today the package “fs” was unpublished from the registry in response to a user report that it was spam. It has been restored.

More detail: the “fs” package is a non-functional package. It simply logs the word “I am fs” and exits. There is no reason it should be included in any modules. However, something like 1000 packages do mistakenly depend on “fs”, probably because they were trying to use a built-in node module called “fs”.

Incident Report for npm, Inc.
  1. Should you even allow publishing a module that has the same name as an intenal one?
  2. If npmjs is confusing, how is fs not confusing?
  3. If thousands of packages depend on it, how can you remove it considering the SNAFU you had just several months prior?

npm devs are clueless? (added 2016-12-27)

Event though npm is a rather nice package manager, its devs often behave like they haven’t got a clue as to what’s happening, or how development should happen.

Performance drop by 40% or more

At one point npm implemented a new progress bar which slowed down installation speeds by 40% and more. See related issue. Worse still, it was now enabled by default (disabled by default in the previous version).

You’ve got to love some comments from the npm dev team:

I’ve been aware of this as an issue for a while and the fix was literally 10 minutes or so of effort, but it hadn’t bubbled up in priority as I hadn’t realized how big an impact it was having.

Or, in a related issue (this appeared after the release):

Profiling would be grand… Put together a minimal benchmark to work against… Ideally I’d like this benchmark to be 2.x and 3.x compatible so we can directly compare different parts.

We break your stable branch, we’re not going to fix it

Look at this issue. Here’s the problem:

I leave this with no comments

Semantic versioning all the way! Not!

NPM swears by semantic versioning. The CLI will even warn you if your packages do not conform to the SemVer.

Meanwhile, npm project itself couldn’t care less about semantic versioning.

You may wish to check any of the changelogs for any release, including patch releases (for example, 4.0.2).

And this really leads to the following question:

Can we even trust our core tools?

This all leads to rather horrible questions:

Also, see the rationale behind Yarn, Facebook’s alternative package manager.

Build tools

The number of build tools (subsets and supersets of Makefiles and npm, or combination of all of the above) for Javascript is simply staggering.

Grunt, Gulp, Broccoli, Brunch, Jake, Mimosa, Webpack, Bower…

They are all broken in more ways than one. Let me just quote from my own experience (read the whole article for more than just this one snippet):

Let’s step back for a second, and consider:

  • Grunt does not transform code, it’s a “task runner” The task it runs is called browserify
  • Well, the problem with the browserify task is that the task runner cannot run it. It needs a plugin called grunt-browserify to do that
  • Oh, and browserify has to run babel on the source code before it can do anything with it.
  • And the problem is that browserify cannot run babel directly. It needs babelify to work

All in all the whole toolchain to produce a Javascript file is grunt -> grunt-browserify -> browserify -> babelify -> babel. And someone in that chain decides that babel missing all of its specified plugins is not a reason to throw an error, stop and report.

Javascript: Fatigue vs. Stockholm syndrome

Are these problems fixed now? I don’t know. I no longer even care if they are fixed or not. I got myself new shiny better toys to play with. Or did I?


Webpack is almost the latest and greatest in a web-developer’s life (there’s a more latest now, called Rollup).

On the surface it does the following: take all the modules that your app has, figure out dependencies between them, bundle them up in a single file that you can serve to your website.

All webpack understands is ES5 and some common module structures: CommonJS, AMD etc. It can invoke so-called loaders whose job is to take a file and produce an ES5 output which Webpack will then take and bundle.

As a result, you can somewhat easily depend on anything: modules written in ES6, or in TypeScript, or in Coffeescript, or… You can even depend on CSS files or PNGs. If there’s a loader for that type of file, Webpack can bundle it.


Also as a result, Webpack’s configuration is inane. And I don’t say it lightly.

Search github for “boilerplate”, and you will come away with easily hundreds of “this is configuration you need to get you started” because it is very nearly impossible to configure webpack.

Webpack is not alone to blame for this. It’s also the fractured tools, the fractured libraries etc.

   test: /(\.css)/,
   loader: "css-loader?module&localIdentName=[path][name]--[local]--[hash:base64:5]"

Yes, the above is configuration for css-loader. Yes, it is a string that contains URL-like structure where parameters you pass in are, well, URL query parameters. Because reasons.

There’s a reason for that, obviously. There always is.

First, you can pipe loaders for a file type:

   test: /(\.css)/,
   loader: "style-loader!css-loader?importLoaders=1!autoprefixer-loader"

This configuration pipes a .css file through style-loader then css-loader with options then autoprefixer-loader.

Second, you can do the same thing from within your code because why the fuck would you not want to do that?


Did you know you could exclude things from a loader, if, well, there’s such an option?

   test: /(\.css)/,
   loader: ExtractTextPlugin.extract('style', 'css?-autoprefixer!postcss')

This is a plugin. Acting as a loader. It has a pre-loader, style. And a loader. Which is a combination of two loaders, css and postcss. Oh, and we exclude autoprefixer (plugin? feature?) from the css loader.

Yes, if there’s just one loader, you can provide its parameters as a regular JSON structure (feast your eyes on the query parameters section). However, this is yet another impendancy mismatch you have to deal with when trying to figure out what, where and how to configure all the moving parts.

You go an look for docs on a *-loader, and you run into anything: config as strings, config as objects, a mix of it. And if something goes wrong, you are left alone, there’s no way to know what failed.

But honsetly. How much time do you have to spend to come up with this: ExtractTextPlugin.extract('style', 'css?-autoprefixer!postcss')?

Learn to love the stacktrace

Let’s pretend you are a C++ developer. You use a Makefile to invoke the gcc compiler on your source code. If there is an error in your source code, you will see the relevant error.

For some reason you will never see the stacktraces from either the make tool or from gcc

Not so in Javascript. Because internal stacktraces from the build tools are the bread and butter of everyday JS development.


> webpack --config webpack.config.js --progress --colors -w -d

Hash: 6f816ab5f143490174a0
Version: webpack 1.13.2
Time: 3176ms
     Asset     Size  Chunks             Chunk Names
    app.js  1.11 MB       0  [emitted]  main  1.25 MB       0  [emitted]  main
   [0] multi main 28 bytes {0} [built]
    + 226 hidden modules

ERROR in ./js/app/fsm/payment-flow-fsm.ts
Module parse failed: /Users/dmitriid/Projects/project/node_modules/awesome-typescript-loader/dist/entry.js!/Users/dmitriid/Projects/project/js/app/fsm/payment-flow-fsm.ts Unexpected token (3:18)
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected token (3:18)
    at Parser.pp$4.raise (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:2221:15)
    at Parser.pp.unexpected (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:603:10)
    at Parser.pp$3.parseExprAtom (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:1822:12)
    at Parser.pp$3.parseExprSubscripts (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:1715:21)
    at Parser.pp$3.parseMaybeUnary (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:1692:19)
	<...skip 20 or so lines...>
    at Parser.parse (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:516:17)
    at Object.parse (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/acorn/dist/acorn.js:3098:39)
	<...skip another 20 or so lines...>
    at nextLoader (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/webpack-core/lib/NormalModuleMixin.js:290:3)
    at /Users/dmitriid/Projects/project/node_modules/webpack/node_modules/webpack-core/lib/NormalModuleMixin.js:259:5
    at Storage.finished (/Users/dmitriid/Projects/project/node_modules/webpack/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:38:16)
    at /Users/dmitriid/Projects/project/node_modules/webpack/node_modules/enhanced-resolve/node_modules/graceful-fs/graceful-fs.js:78:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)
 @ ./js/app/index.tsx 9:25-58

ERROR in [default] /Users/dmitriid/Projects/project/js/app/fsm/payment-flow-fsm.ts:3:15
Expression expected.

The relevant ticket for this is webpack/webpack#1245. Note no one even asks the most obvious question: “why in the seven hells would I need internal stacktraces if I’m not a webpack/plugin developer?” Well, not until yours truly came along.

Can you understand what actually failed?

So, I’m running this:

> webpack --config webpack.config.js --progress --colors "-w" "-d"

Hash: 578f6adad579fede3e98
Version: webpack 1.9.6
Time: 3224ms
     Asset     Size  Chunks             Chunk Names
    app.js  1.12 MB       0  [emitted]  main  1.27 MB       0  [emitted]  main
   [0] multi main 28 bytes {0} [built]
    + 227 hidden modules

ERROR in ./js/app/fsm/state-manager.ts
Module not found: Error: Cannot resolve module 'machina' in /Users/dmitriid/Projects/js/app/fsm
 @ ./js/app/fsm/state-manager.ts 2:14-32

Can you immediately tell me if it’s webpack or typescript failing?

Ok, how about this:

> webpack --config webpack.config.js --progress --colors "-w" "-d"

Hash: ab8d7ccc3d611479ca81
Version: webpack 1.9.6
Time: 4464ms
     Asset     Size  Chunks             Chunk Names
    app.js   2.4 MB       0  [emitted]  main  2.77 MB       0  [emitted]  main
   [0] multi main 28 bytes {0} [built]
    + 341 hidden modules

ERROR in [default] /Users/dmitriid/Projects/js/app/fsm/state-manager.ts:1:25
Cannot find module 'machina'.

As you can clearly see, the first one is a webpack error. The second one is TypeScript error. Relevant issue: webpack/webpack#2878 (there are probably others).

All your options are belong to us

Specifically, the watch option. Could be others. I don’t know and don’t care at this point.

So, if you provide watch: true in Webpack configuration, Webpack will:

Enter watch mode, which rebuilds on file change.
Webpack documetation

This is, unsurprisingly, not entirely correct. See, if you ever decide to create your own build script, and invoke webpack trough its node.js API, you will see that:


Because, I guess, reasons. And, surprisingly, the “short” version with webpack(config, callback) works as expected. Who’d a thunk it.

Forget continuous builds (added 2016-12-27)

Just read through this issue. Tl;dr: if webpack fails, it exits with a status code of 0. Because reasons.

And yes, despite this being a majot bug in the main version currenlty used, it has not been fixed in the two years since it was reported. Because reasons.

Nothing works out of the box anymore

Install and run? Sane defaults? These things are becoming a rare beast in the Javascript world. It seems that nothing works out of the box anymore.

In JS world, sadly, going through hoops and withholding crucial information from the developer is now the accepted norm.


The only job that Babel does is compiling a next version of JavaScript to the current version of Javascript.

I mean this is what says on its site:

Babel is a JavaScript compiler.

Use next generation JavaScript, today.

Babel transforms your JavaScript

You put JavaScript in

[1,2,3].map(n => n + 1);

And get JavaScript out

[1,2,3].map(function(n) {
  return n + 1;

THIS IS A LIE. A lie so blatant that the next line on the site says it’s a lie:

Start by installing the Babel CLI and a preset

npm install --save-dev babel-cli babel-preset-latest

Create a .babelrc file in your project (or use your package.json)

  "presets": ["latest"]

Understand this: out of the box the javascript compiler does not do a single thing. You have to install a number of plugins/presets before it even does anything.

Moreover, if there are no presets and no plugins installed, babel will not even complain about it. It will just … do nothing.

Given this index.js file:

[1, 2, 3].map(n => n + 1);

Running freshly installed babel will not even warn you, and will do nothing:

> node_modules/.bin/babel index.js
[1, 2, 3].map(n => n + 1);

Only if you install a preset and specify it, you get back what you need.

Babel has the “latest” preset which is:

This is a special preset that will contain all yearly presets so user’s won’t need to specify each one individually.

The answer is yes in any world other than Javascript.


I’ve decided to talk about Typescript, because why not. More often than not Javascript is only used as a target language. Multiple other languages exist that compile/transpile into Javascript while promising nicer features, syntax, tools and so on and so forth.

Typescript is a superset of Javascript developed by Microsoft. It introduces type checks and various niceties into the language.

It includes a nice fast compiler which removes the need for Babel, but, obviously, it introduces a whole host of other problems.


Not a typescript issue per se, but still.

When invoking from Webpack, you will deal with either ts-loader or awesome-typescript-loader.

The former one is recommended in typescript docs. The latter one is used in 99% of Webpack boilerplates. Good luck figuring out how they are different, what features they support or don’t support etc.

I haven’t had much experience with ts-loader (yet?), but I’ve alredy run into the following with awesome-typescript-loader.

Who cares about your configs. Part I

Project from scratch. Forgot to create tsconfig.json. No errors whatsoever, obvously. Webpack fails because it cannot parse the non-processed .ts file.

Why did .ts compilation fail? Was it due to a missing tsconfig.json. Wouldn’t it be just so nice if the tools involved could report this?

Who cares about your configs. Part II

When trying to clean up configs I decided to move tsconfig.json to a config directory. Provided path to this file as tsconfig option as per README.

Provided the following invalid option to the compiler in the tsconfig:

  "compilerOptions": {
    "target": "absdefg",

The config above was accepted, and silently ignored. Everything got compiled. Was the config file even picked up? Well, webpack didn’t fail, so probably it was (see Part I). Who ignored the error? TS compiler? awesome-config-loader? No one knows, and it is impossible to find out.

Third-party modules, do you speak it? (upd. 2016-11-16)

Sooner or later you will have to import modules written in or transpiled to Javascript. Unless you develop a library with no external dependencies (quite possible) or something that only depends on other libraries written in Typescript (highly unlikely).

So, you will find yourself typing something like this into your Typescript code:

import machina from 'machina';

Which will fail:

Cannot find module 'machina'.

See, Typescript needs type definitions describing the stuff your import. And here’s a type definition that will work just fine:

declare module machina;

You will ask however, “Is that it?”. Yes, that is it.

Edit (Nov 16): This will finally be fixed in TypeScript 2.1.3

Because reasons. Go ahead and try to make sense of the ambient modules section in the docs.

Types, types everywhere

Libraries are developed in whatever language authors prefer. To provide proper static type checking Typescript needs more than the stub module definitions. It needs actual type definitions for libraries.

Thanks to countless contributors to DefinitelyTyped there are quite a few definitions Typescript can use.

Well,…for some definition of “can”.

> npm install @types/superagent

[default] /Users/dmitriid/Projects/keyflow/keyflow-website/app/node_modules/@types/superagent/index.d.ts:83:30
Cannot find name 'Promise'.

See, despite the fact that this is not a single isolated problem, there are no solutions to this.

Well, except one: maybe try a newer version of npm, namely npm 3.x.

I personally have a problem with this. node.js has this thing called Node LTS, Long Term Support. And at the time of this writing it was:

> node -v
> npm -v

I know, I know. I probably shouldn’t run cutting edge stuff on non-cutting-edge platforms, yada yada.

The problem is there, the problem exists. And, obviously, it exists for some modules, and for others (@types/react works, @types/superagent doesn’t, ad nauseam). Because reasons.

Wrapping up

I’m not sure this warrants a conclusion. I’ve stopped detailing my experience as I was approaching the 4000 word mark. However, there are so many more things that break, run amok, break in unpredictable ways, etc. etc. etc.

I’ll leave you with these quotes:

…never have I worked in an ecosystem where the knowledge attained while becoming a master of the craft goes out of date so rapidly, or where solutions are quite so brittle.

The JS world’s obsession with small tools mean that they combine in endless permutations, causing endless issues to debug.

When the tower of abstractions falls down in a steaming pile and you need to figure out what’s gone wrong and fix it, you then end up sinking hours and hours into it (all the more because you don’t really understand what’s going on, because you didn’t set it up from scratch).

Or you waste a month figuring out how to plug all the tools together. If I have a complex project I know full well I’m going to want code coverage, a proper module system, minification, etc. etc. The initial time investment to investigate the options here and get it all working is faintly ridiculous compared to ecosystems like Java or .NET (or even C++, for that matter).

I’m not even going to talk about the cavalier attitude various popular parts of the ecosystem (e.g. react-router) have towards API stability.

It’s deeply irksome, at best.

Alastair Maw

…the JavaScript community suffers from a very serious case of NIH syndrome, compounded by a neglect for long term sustainability of software projects.

Don’t get me wrong, every single language in the 20 or so years of web development has gone through the framework phase, where people would experiment with solutions for every one of those problems.

The difference is that every one of those languages very quickly converged into good solutions and then made those good solutions into effective tools to get things done.

The JavaScript community, otoh, doesn’t seem to get to the converging part…

Daniel Ruoso

It’s been said that “Javascript fatigue” appears because developers are lazy.

It’s been said that “Javascript fatigue” is because developers don’t want to learn anything new.

These arguments are null and void…

  • there’s nothing lazy in trying to make your build tool work
  • there are exactly zero useful things to learn from that experience

The time I spent trying to figure out the exact motions of all the moving parts I could spend on learning something genuinely new.

Instead, I now have a build toolchain that I have exactly zero confidence in (because it will break unexpectedly at the very next update of any of the fourteen hundred moving parts in it).

Yours truly