The JS ecosystem is exploding

People have been talking about JavaScript fatigue lately, referring to how we’re continuously having to learn about libraries that don’t usually last very long.
I think this discussion evidences how good programmers cannot be lazy people. You have to get a liking for learning new things. Its not that big of a deal, being forced into it: at worst, you’ll become better at identifying good libraries and clean software architecture. Embrace change, and it will make you a better developer.

There is another kind of JavaScript fatigue which doesn’t have as many positive elements to it, and it hasn’t received as much attention. I want to talk about it because I believe it is talking a toll on the JS ecosystem in particular.
Richard Stallman envisioned free software as an environment where people shared their code and other people would improve on it and this would gradually drive progress in the software world in more ways than a closed source environment ever could. I think there are some requirements for such an ecosystem to work properly. A certain balance has to be maintained.

The Cathedral and the Bazaar, which should be a required reading for any open source developer, also hinted at this problem when saying that good programmers know what to write, but great ones know what to rewrite (and reuse).

JavaScript as a language does not have the correct focus that is necessary for that to happen. It is simply far too easy to create a new library or package, and since anyone can do it in a few seconds now, this generates a lot of noise. The language has even evolved to make it as easy as possible to wrap a mess of DOM-manipulating stateful code in a couple of export lines and ship the module to the world. It is just too easy now to share poor code.

Think about the process that you undergo when you want to find a package that solves your problem. You search GitHub, or the npm registry, see what is popular for that, take a look at comparisons between the top candidates, maybe check which ones have unit tests, good code size, reasonable architecture, some minimal documentation, responsive project leaders... it can take a hours, even days, to decide.

The forgotten edge cases

Recently I needed a library to build query strings, just a small one so that I wouldn’t have to include jQuery just for that. After a couple of hours of research, I had found several candidates, but it quickly became evident that even though they were working and updated libraries, none of them was able to handle nested objects and arrays properly, in the way that jQuery could.

If you’re familiar with the 80/20 rule: edge cases are the 20 of the job that takes 80% of the time, and unfortunately most programmers are not going to put in the effort if most of the benefit of releasing their library has already been garnered. Writing complete and bug-free programs is challenging. Sharing poor code is not. Slap a cool animation on your Readme file and everyone will assume your project is awesome.

There are upwards of 200,000 JS projects out there, with language features that focus on increasing that number, and tooling that makes it even easier still. It’s easy to share bad code, and hard to identify and improve it.

Oh, so you’ve spent a couple of days fixing that bug on the library you downloaded? Great job, and I hope you remembered to manually check how active the project is, and if they accept PRs, and if a pending PR doesn’t already fix the bug. Project maintainers don’t have enough tools to make their jobs easy and, unfortunately, the tools that they have are often complex and off-putting.

Consider how difficult it is to keep track of which forks are interesting, even though making a new fork only takes the click of a button.

In truth, some tools have been created which aim to facilitate writing good code, such as linters, code checkers, and so on, but they are not nearly as numerous, and building tools to parse ASTs can be tough. It’s an uphill battle.

An ode to productivity

Programmers love being productive. Once you learn that a couple of lines of code can save you hours of work, you might fall into a trap of believing that productivity is the end goal and any library or framework that makes you solve problems faster must be great. In fact, many people believe this, and tools that simplify your work will often become fads with many people swearing by them.
This can go so far as to reach project managers and recruiters and get them to actively seek people who have experience with the current trendy framework, because all the top tech companies are using it, and their developers swear by it, so clearly it must be good.

This drive towards productivity is the cause for the existence of an increasing number of build tools, code generation tools, and services which aim to make shipping as easy as possible, and nobody will question if you should be shipping your package at all.

The fragmented nature of large open source ecosystems has become evident since the advent of Android. It’s a far-reaching issue and it is also self-reinforcing because languages with smaller communities such as Go and Nim don’t yet suffer from it, and that makes them more appealing, leading to fragmentation on an even larger scale as developers rally to greener pastures.

Fragmentation is like a virus driven by the human passion for creation. People love to share what they make, and too much of it generates noise and makes it hard to see what’s really useful.

Remember Google’s Polymer? Angular 1? Express? Perhaps the creators of these kinds of tester projects which get abandoned if they don’t pan out should be more transparent about their intentions. After all, misleading thousands of users does not generate a positive attitude toward open source, and can lead to bitterness and more fragmentation. [Edit: I’ve been called out for this statement, but just look at how Google is abusing open source to keep other companies under their belt (downright scary article)]

Instead of nurturing narcissistic language ambassadors that drop their projects like they change fedora hats every time they get a new idea, let’s focus on organized efforts and building a sense of community. Let’s make tools that help reduce noise. Let’s make it harder to share crap and easier to share the good stuff.

Human progress is not going to happen by default, so don’t assume that more open means better. An open environment is good so long as it has active monitoring and group effort.

Would you like to know more?

Sign up for the newsletter and get new articles weekly!