nealford.com • Why Everyone (Eventually) Hates (or Leaves) Maven

originally published: January 22, 2013

My colleague Martin Fowler recently posted a Bliki entry about InternalReprogrammability, lamenting the diminishing mind share of tools that allow extremely fine grained customizability, such as Emacs and Smalltalk. I agree whole heartedly with his sentiment; I covered some of the same ground in my Abstraction Distractions keynote, but I attempted to burrow a level deeper.

In my keynote, I defined two types of extensibility/programability abstractions prevalent in the development world: composable and contextual. Plug-in based architectures are excellent examples of the contextual abstraction. The plug-in API provides a plethora of data structures and other useful context developers inherit from or summon via already existing methods. But to use the API, a developer must understand what that context provides, and that understanding is sometimes expensive. I frequently ask developers how often they make non-trivial changes to the way their editor/IDE behaves beyond the preferences dialog. Heavy IDE users do this much less frequently because extending a tool like Eclipse takes an immense amount of knowledge. Consider the complexity of implementing the example that motivated Martin’s blog – the desire for the editor to automatically add a blank line in certain contexts – in a traditional IDE. The knowledge and effort required for a seemingly trivial change prevents the change from occurring, leaving the developer with a perpetually dull tool. Contextual tools aren’t bad things at all – Eclipse and IntelliJ wouldn’t exist without that approach. Contextual tools provide a huge amount of infrastructure that developers don’t have to build. Once mastered, the intricacies of Eclipse’s API provide access to enormous encapsulated power…and there’s the rub: how encapsulated?

In the late 1990’s, 4GLs were all the rage, and they exemplified the contextual approach. The built the context into the language itself: dBASE, FoxPro, Clipper, Paradox, PowerBuilder, Microsoft Access, and similar ilk all had database-inspired facilities directly in the language and tooling. Ultimately, 4GLs fell from grace because of Dietzler’s Law, which I defined in my book Productive Programmer, based on experiences by my colleague Terry Dietzler, who ran the Access projects for my employer at the time:

Dietzler’s Law for Access

Every Access project will eventually fail because, while 80% of what the user wants is fast and easy to create, and the next 10% is possible with difficulty, ultimately the last 10% is impossible because you can’t get far enough underneath the built-in abstractions, and users always want 100% of what they want.

Ultimately Dietzler’s Law killed the market for 4GLs. While they made it easy to build simple things fast, they didn’t scale to meet the demands of the real world. We all returned to general purpose languages.

Composable systems tend to consist of finer grained parts that are expected to be wired together in specific ways. Powerful exemplars of this abstraction show up in *-nix shells with the ability to chain disparate behaviors together to create new things. A famous story from 1992 illustrates just how powerful these abstractions are. Donald Knuth was asked to write a program to solve this text handling problem: read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies. He wrote a program consisting of more than ten pages of Pascal, designing (and documenting) a new algorithm along the way. Then, Doug McIlroy demonstrated a shell script that would easily fit within a Twitter post that solved the problem more simply, elegantly, and understandably (if you understand shell commands):

tr -cs A-Za-z '\n' |
tr A-Z a-z |
sort |
uniq -c |
sort -rn |
sed ${1}q

I suspect that even the designers of Unix shells are often surprised at the inventive uses developers have wrought with their simple but powerfully composable abstractions.

Contextual systems provide more scaffolding, better “out of the box” behavior, and contextual intelligence via that scaffolding. Thus, contextual systems tend to ease the friction of initial use by doing more for you. Huge global data structures sometimes hide behind inheritance in these systems, creating a huge footprint that shows up in derived extensions via their parents. Composable systems have less implicit behavior and initial ease of use but tend to provide more granular building blocks that lead to more eventual power. Well designed composable systems provide narrow local context within well encapsulated modules. For example, to make changes in Emacs such as the one Martin describes requires a bit of knowledge about buffers (which he’s working in) but little else: the knowledge required is local and small in scope.

These abstractions apply to tools and frameworks as well, particularly tools that must scale in their power and sophistication along with projects, like build tools. By hard-won lesson, composable build tools scale (in time, complexity, and usefulness) better than contextual ones. Contextual tools like Ant and Maven allow extension via a plug-in API, making extensions the original authors envisioned easy. However, trying to extend it in ways not designed into the API range in difficultly from hard to impossible, Dietzler’s Law Redux. This is especially true in tools where critical parts of how they function, like the ordering of tasks, is inaccessible without hacking.

Which is why every project eventually hates Maven. Maven is a classic contextual tool: it is opinionated, rigid, generic, and dogmatic, which is exactly what is needed at the beginning of a project. Before anything exists, it’s nice for something to impose a structure, and to make it trivial to add behavior via plug-ins and other pre-built niceties. But over time, the project becomes less generic and more like a real, messy project. Early on, when no one knows enough to have opinions about things like lifecycle, a rigid system is good. Over time, though, project complexity requires developers to spawn opinions, and tools like Maven don’t care.

Tools build atop languages tend to be more composable. My all time favorite build language for personal and project work (almost without regard to the project technology stack) is Rake, the build tool in the Ruby world. It is a fantastic combination of simplicity and power. When I first migrated from Ant to Rake, I started poking around the Rake documentation to find out what tasks were available in Rake, simliar to the giant list of tasks (and extensions) familiar in the Ant world. At first I was disgusted by the lack of documentation until I realized there wasn’t any for a reason: you can do anything you need within Rake tasks, because it’s just Ruby code. Rake has added some nice helpers for file list manipulation, but Rake mostly just handles tasks dependencies and housekeeping, getting out of the way of developers.

People will acuse me of Maven-bashing, but I’m actually not – I’m trying to foster understanding for when it’s useful. No tool works perfectly in every context, and much grief visits projects who try to use tools outside their expiration date. Maven is perfect for starting new projects: it ensures consistency and provides a huge bang for the buck in terms of already existing functionality. But because something starts strong doesn’t mean that it scales well (in fact, almost always the opposite is true). The real trick is to use Maven until the day it starts fighting you, then find an alternative. Once you start fighting with Maven, it’ll never return to the rosy days when your relationship was young. Fortunately, at least one Maven “Get out of Jail Free” card exists in Gradle, which still understands the Maven stuff you already have, but it’s language rather than plug-in based, implemented as a Groovy domain specific language, making it more composable than Maven.

Many contextualized systems eventually become more composable by redesigning them as DSLs. Consider the 4GLs from the 90s. Ruby on Rails and similar frameworks are just like those 4GLS, with a critical distinction: they are implemented as internal DSLs atop a general purpose language. When developers in those environments hit the upper percentages of Dietzler’s Law, they can drop below the framework back to the underlying general purpose language. Rake and Gradle are both DSLs, and I’ve come to believe that scripting builds is far too specific and unique to each project to use contextualized tools.

I tend to prefer composable tools. They tend to have a steeper learning curve but deliver more power and scalability over time, which is why I’m a huge Emacs fan, and why Martin’s post on InternalReprogrammability struck a cord. Contextual tools are fantastic for the proper use; I use IntelliJ for Java coding, but Emacs for pretty much everything else, and I tend to seek out composable tools when there’s an option.