Gentoo Logo

Gentoo Java Packaging Guide

Content:

1.  Background

Before going into details of how Java is handled on Gentoo, it is worthwhile to take a few moments to discuss current trends in the Java programming world.

Java Build Systems

There are a few build systems commonly used for Java projects. Apache's ant (Another Neat Tool) is by far the most common. It is a task oriented system. This means that you give Ant instructions on what tasks to perform and how to perform them to build your project. For example, compile these source files using this class path, create a jar of these classes, and so on. It is fairly easy to get up and running with Ant. Unfortunately, it leaves a lot of room for variation, so Ant scripts are terribly inconsistent from one project to the next.

Maven is another Apache project which has been gaining popularity. In contrast to ant, it is a project based system. This means that you give maven information about your project, and once that's established, you can do many things without any further configuration. For example, you tell maven your project's name and where your source is, and you can then tell maven "Hey maven, make me a jar and some javadocs too!" Another feature of maven is its ability take care of making a project's dependencies available by downloading them from mirrors.

You can also find a number of packages using the classic combination of autoconf and automake. They will mostly be found on projects that interact with existing C and C++ libraries. Unfortunately, automake and autoconf are used mostly dealing with the non-Java bits, leaving the Java bits wedged where it can fit.

Lastly, you may find custom scripts which will attempt to run javac, jar, and javadoc itself, or it may just be a wrapper for ant to properly prepare the build environment.

Bundled Dependencies

One of the features of Java has always been "compile once, run everywhere." A consequence of this is that Java developers traditionally bundle all library dependencies with their project's distribution. While this might be convenient for the end-user, it rapidly becomes problematic from a packager's point of view.

For example, project A depends on C, and project B depends on C, and you have project A and B installed, then you'd have two copies of C installed to support it. And if there's new release of C with a bug or security fix , you would want both A and B to take advantage of it.

2.  Java on Gentoo

This section will give you more insight into how Gentoo handles Java. You should be familiar with the Java User Guide before proceeding.

Virtual Machines (VMs)

As discussed in the User Guide, there are several VMs available from portage.

Testing all packages to ensure they build and run with every VM is a huge undertaking, and there simply are not enough resources to guarantee every package will build and run with every VM.

We now maintain a list of "supported virtual machines" for each architecture. These are the VMs we will test the packages against before committing changes to portage. When you emerge a package, it will by default try to use the best "supported virtual machine."

Of course, Gentoo and Linux in general are about choice, so if you prefer a different VM over the "supported virtual machines", you can easily use that VM instead. However, you should also know that bugs reported with one of the non-"supported virtual machine" will get a low priority if it isn't present using a "supported virtual machine".

Configuring which VM to Use

You can choose which VM to use on a per-user basis, and which VM to use for the system (ie when running things as root). Both of these are configured using java-config.

A user's current VM is represented by a symlink located at ~/.gentoo/java-config-2/current-user-vm, if existing. This symlink points to the JAVA_HOME of the chosen VM. Similarly, the system VM is represented by a symlink at /etc/java-config-2/current-system-vm.

The current VM can also be changed on the fly. This can be accomplished setting the environment GENTOO_VM to contain the name of a VM that java-config knows about.

Java Tools

The traditional Java tools, ie, java, javac, javadoc, etc, are all located in /usr/bin. They are actually all symlinks to the run-java-tool script. This script will call the appropriate tool, depending on how the script is invoked, from the current VM. GENTOO_VM is first checked, then the user VM, and lastly the system VM.

Build-time VM Switching

As outlined in the User Guide, and mentioned briefly earlier, the VM will switch at build time to accommodate the needs of the package. The VM to use is first determined by JAVA_PKG_FORCE_VM, then /etc/java-config-2/build/jdk.conf, and lastly the system VM.

Bytecode Compatibility

The default behavior of javac is to compile bytecode that will compatible with the current VM version and higher (ie forward compatible). This becomes particularly problematic when trying to use a lower versioned VM. For example, source compiled with 1.4 will be compatible with 1.4 and 1.5 and later, and source compiled with 1.5 will only be compatible with 1.5 and later, but not 1.4. This makes it particularly difficult to revert to an earlier VM.

It is possible to specify which VM to compile for to provide the best compatibility.

At build time, the DEPEND and RDEPEND will determine what VM to compile for based on virtual/jdk and virtual/jre. Additionally, this can be controlled by the environment variables JAVA_PKG_WANT_SOURCE and JAVA_PKG_WANT_TARGET.

There is a wrapper for javac, ejavac, which will use the appropriate VM's javac, and then specify the appropriate -target and -source. For projects that use ant, the build.xml can be translated to specify the appropriate -target and -source.

3.  Gentoo Java Packaging Policy

In addition to other Gentoo policies, there are a few unique to Java packages, or that need special attention.

Why build from source?

We should strive to only accept ebuilds that build from source code. For reasons why, please refer to Why Build From Source.

Filesystem Layout

In general, the directory policies are handled for you by the helper functions in the java-utils-2 eclass.

These functions adhere to the following path name conventions:

  • /usr/share/${PN}-${SLOT}/package.env contains information about the package
  • .jar files created from source code are installed to /usr/share/${PN}-${SLOT}/lib/
  • .jar pre-built files not compiled by the ebuild are installed to /opt/${PN}-${SLOT}/lib
  • Javadoc documentation is installed to /usr/share/doc/${PF}/html/api/
  • Source archives are installed to /usr/share/${PN}-${SLOT}/source/
  • User-executable scripts are installed to /usr/bin
  • System-wide environment files are in installed to /usr/env.d/java
  • User-specific environment files can be put into ${HOME}/.gentoo/env.d/

Slotting

Libraries should be slotted according to the API they provide. If two version have the same API, or if a new version is fully compatible with the previous version, then they should be in the same SLOT.

Java libraries have a tendency to sometimes break API and ABI between minor revisions, ie from 2.1.1 to 2.1.2. As a result, it is necessary to slot early, and slot often.

For applications, it is mostly sufficient to keep only the latest version. If the application comes in series, such as Eclipse, we want to keep the latest revision in each series. Very old series may eventually be dropped completely.

Dependencies

Packages should not use bundled jars. Instead, they should make use of jars from already installed packages.

When depending on a package, take care that you depend on a sufficiently recent version, and explicitly ensure at building time that the providing package gives you the correct interface, i.e. the correct SLOT.

4.  Writing the ebuild

General Guidelines

In addition to standard Gentoo ebuild guidelines, there are a number specific for Java packages:

  • In RDEPEND, use ">=virtual/jre-[minimal-version]". If you need a JDK for normal operation, like www-servers/tomcat used to, then you should use ">=virtual/jdk-[minimal-version]". The jre atom MUST have a version.
  • In DEPEND, use ">=virtual/jdk-[minimal-version]", unless the ebuild is not compiling source. The jdk atom MUST have a version.
  • For packages that use Ant to build, try to DEPEND on just dev-java/ant-core when possible instead of dev-java/ant. dev-java/ant-core is automatically added to DEPEND if you inherit java-ant-2. If the package makes use of 'optional' ant tasks, you'll need to DEPEND on dev-java/ant or add WANT_ANT_TASKS="ant-task" before inherit. The latter is of course preferred. see the Ant Guide for details.
  • For packages that are distributed as zip, you need to DEPEND on app-arch/unzip to unpack
  • Always use slot dependencies (EAPI 1 or later) as jars are installed to slot specific paths. Your DEPEND and RDEPEND should look like dev-java/some-library:1.2 or dev-java/another:0.
  • Avoid using bundled .jar files at all costs for source-based packages. Instead, they should use system installed versions with the help of our eclass functions.
  • If you only need the path to installed libraries, you can use java-pkg_getjar(s). Don't call java-config directly, because it will not be recorded as a dependency in the package env.
  • Always provide an easily understandable reason after 'die', so that end-users will provide the maintainers with sensible feedback if the build should fail.
  • Avoid cluttering up the environment by adding environment files to /etc/env.d/. Instead, store your env file in /etc/env.d/java/, and then have user scripts source its environment file when it launches. Otherwise, developers, who regularly override CLASSPATH, CATALINA_HOME and other env vars, will have problems running regular apps. If you use the launcher it will also automatically source the appropriate env file.
  • Make sure you always compile with correct a source/target. This is important to ensure future and backwards compatibility. If the packages use ant, this can be done for you automatically. See java-ant-2.eclass. If not you will have to patch it to pass $(java-pkg_javac-args) to javac.
  • Do no install jars which contain versions in their filename, ie castor-0.9.7.jar. Use java-pkg_newjar to rename and install versioned jars to not contain the version in their filename.

USE flags

If a manual or other extensive documentation is available, it should be installed using the doc USE flag. If the build system can, you should build javadocs also using the doc USE flag. If it does not, you should consider patching it to do so, and provide the patch upstream. HTML documentation should be installed using dohtml and javadocs using java-pkg_dojavadoc.

If the program provides examples, in the form of source code or some other format, and you think they may be worthwhile for some users to have, install them optionally with the examples use flag using the java-pkg_doexamples function.

If you want to go all the way, add the source USE flag that installs the complete source code as a .zip file. Use java-pkg_dosrc for this purpose. This allows IDEs such as Eclipse and NetBeans to do complete source-level debugging. You will need to also DEPEND="source? ( app-arch/zip )" or add JAVA_PKG_IUSE="source" before inherit. Using JAVA_PKG_IUSE means that we can remove the app-arch/zip requirement in the future and use for example jar provided by virtual/jdk.

If your package comes with unit tests, you can enable these using the test FEATURE, in src_test. If you need extra dependencies for the testing you can pull these in with the test useflag (for example dev-java/junit). We will no longer use the junit use flag for this.

Typical Examples

Without further ado, here are a few examples.

Code Listing 4.1: Example: Pure java package


EAPI="2"
JAVA_PKG_IUSE="doc examples source"

inherit eutils java-pkg-2 java-ant-2

DESCRIPTION="Fictional example ebuild."
HOMEPAGE="http://www.gentoo.org/"
SRC_URI="mirror://gentoo/${P}-src.tar.gz"

LICENSE="Apache-2.0"
SLOT="0"
KEYWORDS="~x86 ~sparc ~ppc ~amd64 ppc64"
IUSE=""

COMMON_DEP="
        dev-java/xerces:2
        >=dev-java/log4j-1.2.8:0"

RDEPEND=">=virtual/jre-1.4
        ${COMMON_DEP}"

DEPEND=">=virtual/jdk-1.4
        ${COMMON_DEP}"

S=${WORKDIR}/${P}-src

java_prepare() {
        cd "${S}/lib"
        rm -v *.jar || die

        java-pkg_jar-from xerces-2
        java-pkg_jar-from log4j log4j.jar log4j-1.2.8.jar
}

src_install() {
        java-pkg_newjar target/${P}-dev.jar ${PN}.jar

        use doc && java-pkg_dojavadoc dist/api
        use source && java-pkg_dosrc src/java/org
        use examples && java-pkg_doexamples src/java/examples
}

Code Listing 4.2: Example: Optional java support


EAPI="2"

inherit eutils java-pkg-opt-2

DESCRIPTION="Fictional example ebuild"
HOMEPAGE="http://www.gentoo.org/"
SRC_URI="mirror://gentoo/${P}.tar.gz"

LICENSE="LGPL-2.1"
SLOT="0"
KEYWORDS="~amd64 ~ia64 ~ppc ~ppc64 ~sparc ~x86"
IUSE="java doc nls"

DEPEND="java? ( >=virtual/jdk-1.4 )"
RDEPEND="java? ( >=virtual/jre-1.4 )"

java_prepare() {
        epatch "${FILESDIR}/${P}.patch"
}

src_compile() {
        local myconf="$(use_enable java)"
        if use java; then
                myconf="${myconf} --with-javac-args=\"$(java-pkg_javac-args)\""
        fi

        econf $(use_enable nls) ${myconf} || die

        emake || die
}

src_install() {
        make install DESTDIR=${D} || die

        if use java; then
                java-pkg_newjar build/java/${P}.jar ${PN}.jar

                if use doc; then
                        java-pkg_dohtml -r doc/java
                fi
        fi
}

5.  Java Eclass Reference

Overview

Currently there are six Java related eclasses.

Eclass Usage
java-pkg-2.eclass Any and all java based packages
java-pkg-opt-2.eclass Package that have optional java support
java-ant-2.eclass Ant based packages (see also the Ant Guide)
java-utils-2.eclass Inherited by the java-pkg* eclasses and contains all the functions and dark voodoo
java-vm-2.eclass Helper functions for packages that provide a VM
java-osgi.eclass Helper functions for packages that need to be OSGi compliant (special manifest inside the jar)

java-pkg-2.eclass

This is the eclass you should use for any package using Java. It inherits java-utils-2, and gives you all the needed function. It also depends on the correct version of java-config the eclass needs to do its work. It also exports the pkg_setup phase, where it switchs the vm and setup the environment for your VM.

java-pkg-opt-2.eclass

This is the eclass you should use for packages with optional java support. It does the same as java-pkg-2.eclass but only when the 'java' USE flag is on. The USE flag that is used to enable Java features can be changed by setting JAVA_PKG_OPT_USE before inheriting this eclass.

java-vm-2.eclass

This eclass should be inherited by all packages that provide a VM. If no system-vm can be found it will set the one currently being merged as the system-vm. It also has a function to install the env file and create all necessary symlinks.

  • set_java_env
    • Takes the env file, fills with with the appropriate variables (ie name, version, etc) and places it in /etc/env.d/java, then creates the jvm symlink in /usr/lib/jvm/
    • Takes the base env file as argument
  • install_mozilla_plugin
    • Creates a symlink for the mozilla plugin
    • Takes the path to the oij plugin

java-ant-2.eclass

You should inherit this eclass when your package uses ant. This eclass automatically rewrites the build.xml unless JAVA_PKG_BSFIX is set to off. fex: Add source/target attributes to javac calls.

The rewriting is done in java-ant-2_src_configure for EAPI 2 or eant for earlier EAPIs.

Some variables you can set from your ebuild. (Usually not needed)

  • JAVA_PKG_BSFIX
    • After src_unpack, should we try to 'fix' ant build files to include (the correct) target and source attributes
    • Default: on
  • JAVA_PKG_BSFIX_ALL
    • Should we try to fix all build files we can find
    • Default: yes
  • JAVA_PKG_BSFIX_NAME
    • The name or names(space separated) of the build xml we should fix
    • Default: build.xml
  • JAVA_PKG_BSFIX_TARGET_TAGS
    • Tags to add the target attribute to
    • Default: javac xjavac javac.preset
  • JAVA_PKG_BSFIX_SOURCE_TAGS
    • Tags to add the source attribute to
    • Default: javadoc javac xjavac javac.preset

java-osgi.eclass

You should inherit this eclass when your package needs to be OSGi compliant. This eclass contains functions similar to those those in java-utils-2, but that create a jar containing a manifest with special values. This manifest will allow the jar to be used as an OSGi bundle in applications based on OSGi, for example Eclipse.

If a package is using this eclass, it means that another Gentoo package based on OSGi needs it. Thus, version bumps of the package should still use the install functions from this eclass.

Some of the functions in this eclass request a Manifest file. In this case, be careful about the file you provide. It's usually best to copy the Manifest from a safe source, eg. from Eclipse's bundled jars for example. Note that upstream Eclipse creates those files manually.

  • java-osgi_dojar
    • Similar to java-pkg_dojar. Installs a jar, and records it in the package env. Make sure the jar name does not contain a version. The manifest in the jar will be filled with headers populated from the arguments given to this function.
    • Takes four parameters:
      • path to the jar (like in java-pkg_dojar)
      • bundle symbolic name
      • bundle name
      • export-package-header. This is the most important thing, you must provide a valid OSGi export header. Refer to the OSGi documentation for help with this.
  • java-osgi_newjar
    • Similar to java-pkg_newjar. Use this if you need to rename the jar. The manifest in the jar will be filled with headers populated from the arguments given to this function.
    • Takes four or five parameters:
      • path to the jar (like in java-pkg_newjar)
      • (Optional) name of the renamed jar. If absent, it will be ${PN}.jar
      • bundle symbolic name
      • bundle name
      • export-package-header. This is the most important thing, you must provide a valid OSGi export header. Refer to the OSGi documentation for help with this.
  • java-osgi_dojar-fromfile
    • Similar to java-osgi_dojar, except that instead of creating the manifest from the given arguments, it takes a path to a Manifest file.
    • Takes three or four parameters:
      • (Optional) --no-auto-version. This option disables automatic rewriting of the version in the Manifest file. If not present, the Gentoo package version will be written to the Manifest.
      • path to the jar
      • path to the Manifest file
      • bundle name
  • java-osgi_newjar-fromfile
    • Similar to java-osgi_newjar, except that instead of creating the manifest from the given arguments, it takes a path to a Manifest file.
    • Takes from three to five parameters:
      • (Optional) --no-auto-version. This option disables automatic rewriting of the version in the Manifest file. If not present, the Gentoo package version will be written to the Manifest.
      • path to the jar
      • (Optional) name of the renamed jar. If absent, it will be ${PN}.jar
      • path to the Manifest file
      • bundle name

6.  java-utils-2.eclass

eant/ejavac

Since a lot of people prefer to compile their packages with jikes or eclipse-ecj, and it was becoming a hassle to add these in every ebuild we provide 2 wrapper scripts to call either ant with that compiler or that compiler itself.

It will read the file /etc/java-config/compilers.conf and try the entries front to end until it find an available(installed) compiler, and use it. It can contain any combination of: 'ecj' 'jikes' and 'javac'.

You should only use this if the package is known to work all 3 of the compilers, if later on a problem does occur with one of them, you can set JAVA_PKG_FILTER_COMPILER to it in the ebuild, and it will no longer try to use it.

It also listens to the JAVA_PKG_FORCE_COMPILER environment variable to force one of them, this can be useful when testing a new ebuild with every compiler.

Install functions

  • java-pkg_dojar
    • Installs a jar, and records it in the package env. Make sure the jar name does not contain a version.
    • Takes one or more paths to a jars
    • will die on errors
  • java-pkg_newjar
    • If you need to rename the jar, since we don't allow versions in the jar name, then calls _dojar.
    • If you only pass one argument it will name it ${PN}.jar
    • will die on errors
  • java-pkg_dowar
    • Installs a war into a packages webapps directory.
    • War is not recorded within package.env and is not integrated with java-config or any servlet container (e.g. Tomcat)
    • Takes one or more paths to a war
    • will die on errors
  • java-pkg_dohtml
    • Deprecated, use dohtml now.
  • java-pkg_dojavadoc
    • Installs javadoc documentation and records it to package.env
    • Takes only one argument namely the directory of the javadoc root.
    • Will die if the argument is not a directory or does not exist.
  • java-pkg_doexamples
    • Installs examples to the standard location
    • Takes multiple arguments.
    • If given only one directory, installs dir/* into examples/.
    • Will die on errors
  • java-pkg_addcp
    • Sometimes you need specials things on the package's classpath
    • Classpath string
  • java-pkg_regjar
    • record an already installed jar in he packages env
    • Takes one or more paths to a jars. Will strip ${D}
    • will die on error
  • java-pkg_doso
    • Install a jni library, and register its location it the package env
    • Takes one or more path to a library
    • will die on error
  • java-pkg_regso
    • record an already installed library in the package env
    • takes one or more paths to a library
    • will die on error
  • java-pkg_jarinto
    • Change the location java-pkg_dojar installs to
  • java-pkg_sointo
    • Change the location java-pkg_doso installs to
  • java-pkg_dosrc
    • Install a zip containing the source, so it can used in IDE's like Eclipse or Netbeans.
    • Takes a list of paths to the base of the source directories like com or org.
    • Will die on error
  • java-pkg_dolauncher
    • Makes a wrapper script to launch/start this package.
      This wrapper will (attempt to) automatically figure out the dependency and build up the classpath and library path when needed. It will also make sure it uses a vm that is capable of running the package and every dependency.
    • You can call the function without any arguments if the ebuild only installs one jar. In that case the function uses ${PN} as the binary name and the jar is launched using the Main-class attribute (the same as running with java -jar). Otherwise use the following arguments.
      • --main The class to start
      • --jar The jar to launch (this will ignore classpath)
      • --java_args Extra arguments to pass to java
      • --pkg_args Extra arguments to pass to the package
      • --pwd Directory it should launch the app from
      • -into Directory where it should put the launch wrapper
      • -pre Location of a (bash) script to include and run prior to launching

Query functions

  • java-pkg_jar-from
    • Creates symlinks to the jars of a package in the cwd
    • Can be called with
      • A comma separated list of packages
      • A single package and the jar name you want from that package
      • A single package, the jar name, and the name of symlink
    • Will die on errors
  • java-pkg_getjars
    • echos the classpath, and record the dependency
    • Takes a comma separated list of packages
    • Does not die on errors, returns 1
  • java-pkg_getjar
    • echos the classpath of the jar, and records the dependency
    • A single package, and the name of the jar
    • Does not die on errors, returns 1

Other functions

  • eant
    • Wrapper to call ant and use the preferred compiler
    • Will pass any args to ant
    • Will die on error
    • Affected by the ANT_TASKS and related variables - see the Ant Guide for details.
  • ejavac
    • Wrapper to call the preferred compiler
    • Will pass any args to javac
    • Will die on error
  • ejunit
    • Wrapper start junit.textui.TestRunner
    • Will pass on any args
    • Appends junit and full recorded deptree to -cp or -classpath
    • Will die on error
  • java-pkg_get-source
    • Get the -source value
  • java-pkg_get-target
    • Get the -target value
  • java-pkg_javac-args
    • Get the arguments that should be passed to javac
  • java-pkg_switch-vm
    • Attempts to switch to the best vm
  • java-pkg_ensure-vm-version-sufficient
    • die if the current vm cannot build this package
    • Takes a version: 1.4 1.5
  • java-pkg_ensure-vm-version-eq
    • die if the current vm's version isn't equal to ${1}
    • Takes a version: 1.4 1.5
  • java-pkg_is-vm-version-eq
    • test is the active vm's version equals $1
    • Takes a version: 1.4 1.5
  • java-pkg_ensure-vm-version-ge
    • die if the current vm's version isn't at least ${1}
    • Takes a version: 1.4 1.5
  • java-pkg_is-vm-version-ge
    • test if the active vm's version is at least $1
    • Takes a version: 1.4 1.5
  • java-pkg_register-dependency
    • Register runtime dependency on a package
    • Useful for binary packages and things loaded by custom classloading
    • Takes a package list: xerces-2,xalan
    • Or one jar from a package: ant-core ant.jar
  • java-pkg_register-optional-dependency
    • Same as register-dependency but the package is not expected to be installed at runtime

External variables

Some variables that can come in handy, you are not allowed to use these from an ebuild.

  • JAVA_PKG_ALLOW_VM_CHANGE
    • Allow the eclass to switch the vm at merge time. If you set this to no, it will die when the active vm isn't sufficient
    • Default: yes
  • JAVA_PKG_WANT_TARGET
    • Ignore what DEPEND claims it needs, and use the value of this var for -target
    • Default: unset
  • JAVA_PKG_WANT_SOURCE
    • Ignore what DEPEND claims it needs, and use the value of this var for -source
    • Default: unset
  • JAVA_PKG_FORCE_VM
    • Force a specific jdk at build time.
    • Default: unset
  • JAVA_PKG_FORCE_COMPILER
    • Force a specific compiler to use at build time.
    • Default: unset
  • JAVA_PKG_DEBUG
    • Turn on very verbose output in for example build.xml rewriting and ant.
    • Default: unset
  • JAVA_PKG_STRICT
    • Enables additional checks in Java eclasses. This variable must be set when developing Java ebuilds.
    • Default: unset

7.  External Resources



Print

Page updated January 18, 2008

Summary: This document serves two purposes. The first is to discuss the specifics of how Gentoo handles the Java platform. The second is to discuss how to package Java packages for Gentoo. This document assumes you are familiar with the Java User Guide.

Karl Trygve Kalleberg
Author

Joshua Nichols
Author

Petteri Räty
Author

Donate to support our development efforts.

Copyright 2001-2014 Gentoo Foundation, Inc. Questions, Comments? Contact us.