Gentoo Logo

Disclaimer : This handbook has been replaced by a newer version and is not maintained anymore.


[ << ] [ < ] [ Home ] [ > ] [ >> ]


3. Portage Features

Content:

3.a. Portage Features

Portage has several additional features that makes your Gentoo experience even better. Many of these features rely on certain software tools that improve performance, reliability, security, ...

To enable or disable certain Portage features you need to edit /etc/portage/make.conf's FEATURES variable which contains the various feature keywords, separated by white space. In several cases you will also need to install the additional tool on which the feature relies.

Not all features that Portage supports are listed here. For a full overview, please consult the make.conf man page:

Code Listing 1.1: Consulting the make.conf man page

$ man make.conf

To find out what FEATURES are default set, run emerge --info and search for the FEATURES variable or grep it out:

Code Listing 1.2: Finding out the FEATURES that are already set

$ emerge --info | grep ^FEATURES=

3.b. Distributed Compiling

Using distcc

distcc is a program to distribute compilations across several, not necessarily identical, machines on a network. The distcc client sends all necessary information to the available distcc servers (running distccd) so they can compile pieces of source code for the client. The net result is a faster compilation time.

You can find more information about distcc (and how to have it work with Gentoo) in our Gentoo Distcc Documentation.

Installing distcc

Distcc ships with a graphical monitor to monitor tasks that your computer is sending away for compilation. If you use Gnome then put 'gnome' in your USE variable. However, if you don't use Gnome and would still like to have the monitor then you should put 'gtk' in your USE variable.

Code Listing 2.1: Installing distcc

# emerge distcc

Activating Portage Support

Add distcc to the FEATURES variable inside /etc/portage/make.conf. Next, edit the MAKEOPTS variable to your liking. A known guideline is to fill in "-jX" with X the number of CPUs that run distccd (including the current host) plus one, but you might have better results with other numbers.

Now run distcc-config and enter the list of available distcc servers. For a simple example we assume that the available DistCC servers are 192.168.1.102 (the current host), 192.168.1.103 and 192.168.1.104 (two "remote" hosts):

Code Listing 2.2: Configuring distcc to use three available distcc servers

# distcc-config --set-hosts "192.168.1.102 192.168.1.103 192.168.1.104"

Don't forget to run the distccd daemon as well:

Code Listing 2.3: Starting the distccd daemons

# rc-update add distccd default
# /etc/init.d/distccd start

3.c. Caching Compilation

About ccache

ccache is a fast compiler cache. When you compile a program, it will cache intermediate results so that, whenever you recompile the same program, the compilation time is greatly reduced. The first time you run ccache, it will be much slower than a normal compilation. Subsequent recompiles should be faster. ccache is only helpful if you will be recompiling the same application many times; thus it's mostly only useful for software developers.

If you are interested in the ins and outs of ccache, please visit the ccache homepage.

Warning: ccache is known to cause numerous compilation failures. Sometimes ccache will retain stale code objects or corrupted files, which can lead to packages that cannot be emerged. If this happens (if you receive errors like "File not recognized: File truncated"), try recompiling the application with ccache disabled (FEATURES="-ccache" in /etc/portage/make.conf) before reporting a bug. Unless you are doing development work, do not enable ccache.

Installing ccache

To install ccache, run emerge ccache:

Code Listing 3.1: Installing ccache

# emerge ccache

Activating Portage Support

Open /etc/portage/make.conf and add ccache to the FEATURES variable. Next, add a new variable called CCACHE_SIZE and set it to "2G":

Code Listing 3.2: Editing CCACHE_SIZE in /etc/portage/make.conf

CCACHE_SIZE="2G"

To check if ccache functions, ask ccache to provide you with its statistics. Because Portage uses a different ccache home directory, you need to set the CCACHE_DIR variable as well:

Code Listing 3.3: Viewing ccache statistics

# CCACHE_DIR="/var/tmp/ccache" ccache -s

The /var/tmp/ccache location is Portage' default ccache home directory; if you want to alter this setting you can set the CCACHE_DIR variable in /etc/portage/make.conf.

However, if you would run ccache, it would use the default location of ${HOME}/.ccache, which is why you needed to set the CCACHE_DIR variable when asking for the (Portage) ccache statistics.

Using ccache for non-Portage C Compiling

If you would like to use ccache for non-Portage compilations, add /usr/lib/ccache/bin to the beginning of your PATH variable (before /usr/bin). This can be accomplished by editing .bash_profile in your user's home directory. Using .bash_profile is one way to define PATH variables.

Code Listing 3.4: Editing .bash_profile

PATH="/usr/lib/ccache/bin:/opt/bin:${PATH}"

3.d. Binary Package Support

Creating Prebuilt Packages

Portage supports the installation of prebuilt packages. Even though Gentoo does not provide prebuilt packages by itself Portage can be made fully aware of prebuilt packages.

To create a prebuilt package you can use quickpkg if the package is already installed on your system, or emerge with the --buildpkg or --buildpkgonly options.

If you want Portage to create prebuilt packages of every single package you install, add buildpkg to the FEATURES variable.

More extended support for creating prebuilt package sets can be obtained with catalyst. For more information on catalyst please read the Catalyst Frequently Asked Questions.

Installing Prebuilt Packages

Although Gentoo doesn't provide one, you can create a central repository where you store prebuilt packages. If you want to use this repository, you need to make Portage aware of it by having the PORTAGE_BINHOST variable point to it. For instance, if the prebuilt packages are on ftp://buildhost/gentoo:

Code Listing 4.1: Setting PORTAGE_BINHOST in /etc/portage/make.conf

PORTAGE_BINHOST="ftp://buildhost/gentoo"

When you want to install a prebuilt package, add the --getbinpkg option to the emerge command alongside of the --usepkg option. The former tells emerge to download the prebuilt package from the previously defined server while the latter asks emerge to try to install the prebuilt package first before fetching the sources and compiling it.

For instance, to install gnumeric with prebuilt packages:

Code Listing 4.2: Installing the gnumeric prebuilt package

# emerge --usepkg --getbinpkg gnumeric

More information about emerge's prebuilt package options can be found in the emerge man page:

Code Listing 4.3: Reading the emerge man page

$ man emerge

3.e. Fetching Files

Parallel fetch

When you are emerging a series of packages, Portage can fetch the source files for the next package in the list even while it is compiling another package, thus shortening compile times. To make use of this capability, add "parallel-fetch" to your FEATURES. Note that this is on by default, so you shouldn't need to specifically enable it.

Userfetch

When Portage is run as root, FEATURES="userfetch" will allow Portage to drop root privileges while fetching package sources. This is a small security improvement.

3.f. Pulling Validated Portage Tree Snapshots

As an administrator, you can opt to only update your local Portage tree with a cryptographically validated Portage tree snapshot as released by the Gentoo infrastructure. This ensures that no rogue rsync mirror is adding unwanted code or packages in the tree you are downloading.

To configure Portage, first create a truststore in which you download and accept the keys of the Gentoo Infrastructure responsible for signing the Portage tree snapshots. Of course, if you want to, you can validate this GPG key as per the proper guidelines (like checking the key fingerprint). You can find the list of GPG keys used by the release engineering team on their project page.

Code Listing 6.1: Creating a truststore for Portage

# mkdir -p /etc/portage/gpg
# chmod 0700 /etc/portage/gpg
(... Substitute the keys with those mentioned on the release engineering site ...)
# gpg --homedir /etc/portage/gpg --keyserver subkeys.pgp.net --recv-keys 0xDB6B8C1F96D8BF6D
# gpg --homedir /etc/portage/gpg --edit-key 0xDB6B8C1F96D8BF6D trust

Next, edit /etc/portage/make.conf and enable support for validating the signed Portage tree snapshots (using FEATURES="webrsync-gpg") and disabling updating the Portage tree using the regular emerge --sync method.

Code Listing 6.2: Updating make.conf

FEATURES="webrsync-gpg"
PORTAGE_GPG_DIR="/etc/portage/gpg"

Code Listing 6.3: Updating repos.conf

# Make sure sync-type and sync-uri are commented out
# sync-type = rsync
# sync-uri = ...

That's it. Next time you run emerge-webrsync, only the snapshots with a valid signature will be expanded on your file system.


[ << ] [ < ] [ Home ] [ > ] [ >> ]


Print

View all

Page updated April 12, 2014

Summary: Discover the features Portage has, such as support for distributed compiling, ccache and more.

Sven Vermeulen
Author

Grant Goodyear
Author

Roy Marples
Author

Daniel Robbins
Author

Chris Houser
Author

Jerry Alexandratos
Author

Seemant Kulleen
Gentoo x86 Developer

Tavis Ormandy
Gentoo Alpha Developer

Jason Huebel
Gentoo AMD64 Developer

Guy Martin
Gentoo HPPA developer

Pieter Van den Abeele
Gentoo PPC developer

Joe Kallar
Gentoo SPARC developer

John P. Davis
Editor

Pierre-Henri Jondot
Editor

Eric Stockbridge
Editor

Rajiv Manglani
Editor

Jungmin Seo
Editor

Stoyan Zhekov
Editor

Jared Hudson
Editor

Colin Morey
Editor

Jorge Paulo
Editor

Carl Anderson
Editor

Jon Portnoy
Editor

Zack Gilburd
Editor

Jack Morgan
Editor

Benny Chuang
Editor

Erwin
Editor

Joshua Kinard
Editor

Tobias Scherbaum
Editor

Xavier Neys
Editor

Gerald J. Normandin Jr.
Reviewer

Donnie Berkholz
Reviewer

Ken Nowack
Reviewer

Lars Weiler
Contributor

Donate to support our development efforts.

Copyright 2001-2014 Gentoo Foundation, Inc. Questions, Comments? Contact us.