Autotools Considered Harmful

Programmers quickly learn the importance of good tools. The right programming environment, libraries, editors, and so forth can mean the difference between a two-day project and a two-month project.

For the most part, Linux has the best developer tools around. However, in one important respect, Linux suffers, and that is in its usage of autotools.

In this document, I'm going to argue that autools is not a blessing, but a curse. It is difficult for developers to understand, not portable, slow to execute, piggish with disk space, and generally a square wheel. Finally, Autotools could and should be replaced with CMake with relatively little effort.

About Autotools

Autotools, also known as the GNU build system, is a suite of of programming tools designed to build software. It is similar in spirit to the Make program. Autotools was designed to be portable to many different operating systems.

The Standard that Wasn't

There is no standard way of using autotools itself. Because it's a suite of programs, different developers can choose to use different bits and pieces. Add version skew and platform differences to this mixture, and you have a recipe for confusion.

I know of at least three different ways of using autotools:

The first thing you have to do when analyzing the build system of a project that uses autotools is finding which of these three variants they're actually using. Obviously, you don't want to start editing files which turn out to be automatically generated. To complicate matters even further, there is no consensus on whether or not to check in the generated files or not.

The generated files are often several megabytes in size, so maybe the project managers who don't want to check them in have a point. It's too bad for the poor saps who just want to build the project from the git tree, though.

The Learning Curve

In order to fully understand autotools, you'll need to learn how to use the following different tools: Many other build systems ask programmers to learn a domain-specific language. But how many other build systems ask programmers to learn about dozens of different UNIX shell variants? How many other build systems use a programming language like m4?

m4 is a macro processing language. The level of abstraction is similar to the C preprocessor-- in other words, low. Does it seem strange to use a language with a lower level of abstraction than C to perform traditional scripting tasks? Get used to the pain, because with autoconf, you'll be using m4 constantly.

Portability

It's time to talk about portability. After all, portability was the whole reason the autotools system was invented. We might be suffering a little bit of pain here and there, but it's all worth it to gain portable programs, right?

Unfortunately, the portability story for autotools looks bleak. The executive summary is that you are in charge of portability.

That's right, you. You are responsible for writing portable shell scripts, that can run on ash, bash, ksh, pdksh, zsh, and whatever shells may have shipped on AIX, ancient versions of Solaris, Mac OS X, and so forth. But don't worry, the autotools manual will provide you with a "Zoology of shells"so that you can get started on your never-ending task of supporting all past and future shells.

Here's a handy tip: sed '10q' is a great alternative to head on systems that don't have the latter command installed. Also, aspirin is cheaper when you buy it in bulk!

Technically, you can use autotools on Microsoft Windows, but no sane person would ever try it. Projects that use autotools that need to be portable to Windows generally have a second, parallel build system for Windows. John Calcote, author of "A brief introduction to the GNU autotools," writes "while it can be done, the tradeoffs are way too significant to justify shoe-horning a Windows project into the Autotools build paradigm."

No C or C++ build system can completely eliminate the hassles of writing portable programs. But autotools actually hurts the portability of projects that use it, which has to be some kind of record in build system perversity.

Wasting the computer's time

Autotools wastes the computer's time. You might expect that on modern systems this would not be an issue, but frankly, it is.

Just to put some concrete numbers behind this, I created a simple project to test autotools. There are about 34 total lines of C code in this project, but once I initialize autotools, the directory contains 4.1 megabytes. Running configure on this empty project takes 6.5 seconds.

From personal experience, I always have to wait several seconds after typing make foo before even seeing the compiler run. It's a waste of both CPU power and my time as a developer.

By using automake in a non-recursive way, you can acheive vaguely acceptable performance. But that means that you have to combine together all of your build system directives into one giant file, giving up modularity and clean design. As programmers, would we accept having to put all of our source into one giant file, because the compiler was too dumb to efficiently handle multiple files? No. And we shouldn't expect the same shabby treatment from our build systems.

The build-time performance of libtool is just inexcusable. Josh Triplett estimates that "for many... libraries and other packages, more than half of the build time goes into running the libtool shell script." Libtool, as he goes on to explain, is "an 8500 line, 250 kilobyte shell script, which runs every single time you compile a source file."

Not content with causing poor build time performance, libtool also causes unecessarily long startup times, by replacing your binary with a shell script that loads the "real" binary and libraries.

Lack of mindshare

Most successful open source tools get used by many different organizations. The Linux kernel has made its way into everything from home routers, to mobile phones, to supercomputers. Companies, universities, and other organizations recognized the value of the technology and put it to good use. The git source control system was picked up by a lot of companies, especially web developers.

Autotools, on the other hand, has barely made a ripple in the corporate world. Neither Google, nor Facebook, nor Amazon, nor any other major company has publicly committed to using autotools internally. Google even went so far as to roll their own build system for Android. No doubt, some people will argue that they did this to avoid the GPL. However, the build system that they ended up coming up with depends on Makefile extensions that exist only in GNU Make, a GPLed program.

Most developers on open source projects have a casual familiarity with autotools. But the number of people who would claim to be good with autotools is vanishingly small.

When speaking about the KDE experience with autotools, Alexander Neundorf wrote:
In KDE3 development, we ended up with an autotools-based build system that most of our developers did not fully understand. I estimate that no more than 10 people were able to fix any major problem occurring with it, first and foremost our "buildsystem-guru" Stephan Kulow. Over the years, these guys must have been contacted by hundreds of co-developers with requests for help to fix build problems in specific applications and modules.

The Alternatives

One comment I hear frequently about autotools is that they may be bad, but they are the only viable option. But the reality is that alternatives exist-- many of which are faster, easier to understand, more portable, and altogether more usable than autotools.

Most Linux developers don't accept the argument that they should use Windows because "it's the standard" on the desktop. So why should we use a build system that we know is bad? And who exactly decided that autotools was "the standard"? It certainly wasn't Linus Torvalds, who designed his own build system for the Linux kernel. It wasn't the KDE project, which migrated from autotools to CMake. It wasn't MySQL, which also is in the process of migrating from autotools to CMake. It's not any of the big software companies-- they don't use autotools.

Autotools is the standard only in the minds of those who believe it is the standard. It's sad to see developers wasting their time learning an obscure macro language, or debugging problems related to libtool, when they could be improving open source software. It's sad to see projects that claim to be portable, but in fact are not, because autoconf required the developers to write code that was portable to 10 different shells, and they only managed 9. It's sad to see potential contributors get discouraged by the complexity of dealing with a Makefile.am file.

There are many viable alternatives to autotools. My favorite is CMake. I would encourage anyone starting a new project to try out CMake. It really will save you time and a lot of frustration.