Automagic dependencies, what they are and how to fix them

1.  Introduction

What are automagic dependencies?

The so-called automagic dependencies are shallow dependencies of a software recognized at build or runtime and changes the way the software works. The name automagic is a pun referred to the use of GNU autotools, that produces most of the cases of automagic dependencies.

Software usually have two kind of dependencies: mandatory dependencies and optional dependencies. The first kind of dependencies are needed to use the software (that might be a library or a program), and cannot be missing in the system while building or running the program (depending whether they are build or runtime dependencies). Optional dependencies are the ones that can be disabled, usually at buildtime (but sometimes at runtime, too).

Optional dependencies are usually up to the user (or the builder) to enable or disable, the classical example is brought by the --enable-foo or --with-bar options at ./configure call (those parameters are used to enable dependencies that are off by default, but there are cases when the dependencies are on by default so you have --disable-foo and --without-bar).

But with build systems that tries to understand what is present in the system they are building it, sometimes dependencies get automagic. This means that the build system doesn't give the builder a way to decide if he wants something enabled, so the dependency added, but they just enable it when they find it. This is the wrong behavior.

Why automagic dependencies are wrong

In the case of binary-based distributions, like RPM or DEB based ones, automagic dependencies does not change anything: if the user has something installed and is building by hand, it's usually what he wants to enable, while if it's the maintainer, he'll just have to add a dependency over the packages required to run the binaries he has created.

Different is for source-based distributions like Gentoo Linux (and variants). As source-based distributions doesn't detach user and devel packages, the build systems might find more stuff than the user required, and will try to link to all it know about. This is the major cause of binary linking breaking after a depclean.

To simplify, when an automagic dependency is not stated as mandatory in an ebuild, but rather has a flag that just adds or remove the dependencies on a given package, if this package is present in the system building the software with automagic dependencies, but then it's removed, the software will break, requiring to run revdep-rebuild to fix the linking. It's also possible that an user really don't want some dependency enabled because he know it's likely to break from time to time, or because he's going to create a binary package for another machine where the dependency might not be present (or might not work at all).

When a package has automagic dependencies there are only two things that can be done: the first is to state the dependency as mandatory, no matter what the users put in their USE variable, but that might mean that some support that people don't want is always enable and its dependencies pulled in; the other is to fix the build system to be able to disable at build time the dependency also if it's present on the system.

Most of the time upstream developers don't really think of adding support for disabling automagic dependencies as they don't feel them as actual problems: they do have all of them installed, or the ones they need, and they usually build with all of them. Luckily, most of upstream developers also don't mind adding options to disable them if patches are provided (sometimes also without patches, but of course it might be more welcome if already prepared patches are sent), but that's not always the case, for example Wine's upstream don't want to add support for enabling or disabling features in ./configure call as they want the software to always use as much as options as possible.

2.  Fixing automagic dependencies

Autotools

Most of the automagic dependencies, like the name suggests, are due to (bad) use of GNU autotools (autoconf to be exact). There are two main cases where automagic dependencies are brought in: the first is the "lazy devs" case, where the dependencies does not have a ./configure parameter at all, they are just checked with AC_CHECK_LIB or the pkg-config macro PKG_CHECK_MODULES, that allows to run specific code when a library (or a package) is present or not; the other case is the "silly argument" case, where a --disable-foo or --without-bar parameter is actually accepted by ./configure, but it's not checked correctly.

The first case is actually simple to fix, it's just matter of adding a AC_ARG_WITH (or AC_ARG_ENABLE) call and then check for the corresponding variable before doing the test. It's useful to know that the first parameter passed to the above macro actually names a variable that gets loaded by autoconf without having to add the extra parameters for action to execute when the parameter is present and when it's not, the variable is named $enable_foo or $with_bar, depending on which of the two macros are called.

AC_ARG_WITH([foo], AS_HELP_STRING([--without-foo], [Build without foo library (default: test)]))

if test "x$with_foo" != "xno"; then
  PKG_CHECK_MODULES([FOO], [foo >= 0.1])
fi

AC_ARG_WITH([foo], AS_HELP_STRING([--with-foo], [Build with foo library (default: disabled)]))

if test "x$with_foo" == "xyes"; then
  PKG_CHECK_MODULES([FOO], [foo >= 0.1])
fi

When the parameter is present but it's not honored, it might be simple as well as complex to fix the dependency. It might just be a test that's not properly written, so it has to be changed in something alike to the tests above, or it might be a complete screw-up in calls of AC_ARG_WITH macros. In those cases, it's better to check the code carefully and contact upstream if a screw up seems likely.

There's actually another way to workaround, without fixing (and patching), the automagic dependencies generated by AC_CHECK_LIB, and it's by playing around with the cache values used by autoconf. This method is actually deprecated because it does not fix the issue to the roots and might create problems if upstream changes the tests a bit using a different cache variable name. Also, in this way fixes can't be sent upstream for integration in later versions.

CMake

Automagic dependencies may occur in CMake-based build systems where PKG_CHECK_MODULES is called without the REQUIRED parameter unconditionally. Fixing this is quite easy, as it only involves introducing an option to build system and executing PKG_CHECK_MODULES, depending on its value.

OPTION(ENABLE_FOO "Enable foo library" ON)
...
IF (ENABLE_FOO)
  PKG_CHECK_MODULES (FOO foo>=0.1)
ENDIF (ENABLE_FOO)
...
IF (ENABLE_FOO)
  IF (FOO_FOUND)
  ...
  ELSE (FOO_FOUND)
  ...
  ENDIF (FOO_FOUND)
ENDIF (ENABLE_FOO)

Other build systems

Automagic dependencies can be created also with custom build systems that are used by some software. Unfortunately, being custom, those build systems are usually difficult to tweak, and there's no way to describe a general approach to fix them.

The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license.